2024-09-10T10:25:50.174 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T10:25:50.200 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T10:25:50.303 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034 branch: wip-rishabh-testing-20240909.165402-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7899034' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH/nW15y5b7YMkNw39rcyB+jkgBn0L5xLKSJ3ftiHSmVf0XmzvpmcY8uLm1b3t6boNF6JDY90o0yhZJDTY3WWwQ= smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOzH9aaXdGI6S7Uma+hPKRZcEAfJftmoc7Unav8/EeNNY591fGiozHCwHFA/+Tzm9m+t0nKllwhBtcAJE2FaJfA= smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhRWxNbN2dgH0OTWhBvwi2r1Z/w6kzHiZZ8S8G8Wl2s2xteohiW7MYGu6PYE1+H8QgfCFoItPag8I+VfVKDie0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T10:25:50.304 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T10:25:50.305 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T10:25:50.305 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T10:25:50.306 INFO:teuthology.task.internal:Checking packages... 2024-09-10T10:25:50.328 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T10:25:50.328 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T10:25:50.329 INFO:teuthology.packaging:ref: None 2024-09-10T10:25:50.329 INFO:teuthology.packaging:tag: None 2024-09-10T10:25:50.329 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:25:50.329 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:25:50.329 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:25:50.470 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T10:25:50.472 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T10:25:50.480 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T10:25:50.480 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T10:25:50.503 INFO:teuthology.task.internal:Saving configuration 2024-09-10T10:25:50.523 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T10:25:50.532 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T10:25:50.550 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 10:20:12.650419', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH/nW15y5b7YMkNw39rcyB+jkgBn0L5xLKSJ3ftiHSmVf0XmzvpmcY8uLm1b3t6boNF6JDY90o0yhZJDTY3WWwQ='} 2024-09-10T10:25:50.566 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 10:20:12.648953', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOzH9aaXdGI6S7Uma+hPKRZcEAfJftmoc7Unav8/EeNNY591fGiozHCwHFA/+Tzm9m+t0nKllwhBtcAJE2FaJfA='} 2024-09-10T10:25:50.582 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 10:20:12.651727', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPhRWxNbN2dgH0OTWhBvwi2r1Z/w6kzHiZZ8S8G8Wl2s2xteohiW7MYGu6PYE1+H8QgfCFoItPag8I+VfVKDie0='} 2024-09-10T10:25:50.582 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T10:25:50.591 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T10:25:50.591 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T10:25:50.591 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T10:25:50.591 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T10:25:50.677 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fba0e8375b0>, signals=[15]) 2024-09-10T10:25:50.677 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T10:25:50.685 INFO:teuthology.task.internal:Opening connections... 2024-09-10T10:25:50.685 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T10:25:50.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:25:50.805 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-09-10T10:25:50.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:25:50.878 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T10:25:50.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:25:50.950 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T10:25:50.958 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-09-10T10:25:50.976 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-09-10T10:25:50.976 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-09-10T10:25:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-09-10T10:25:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-09-10T10:25:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-09-10T10:25:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-09-10T10:25:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-09-10T10:25:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-09-10T10:25:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T10:25:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-09-10T10:25:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-09-10T10:25:51.034 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T10:25:51.035 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-09-10T10:25:51.035 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T10:25:51.035 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T10:25:51.035 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T10:25:51.035 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-09-10T10:25:51.050 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-09-10T10:25:51.069 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-09-10T10:25:51.070 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-09-10T10:25:51.127 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-09-10T10:25:51.127 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-09-10T10:25:51.127 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-09-10T10:25:51.127 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-09-10T10:25:51.127 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-09-10T10:25:51.127 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-09-10T10:25:51.127 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T10:25:51.128 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-09-10T10:25:51.128 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-09-10T10:25:51.128 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T10:25:51.128 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-09-10T10:25:51.128 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T10:25:51.128 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T10:25:51.128 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T10:25:51.128 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-09-10T10:25:51.142 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-09-10T10:25:51.161 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-09-10T10:25:51.161 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-09-10T10:25:51.219 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-09-10T10:25:51.219 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-09-10T10:25:51.219 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T10:25:51.220 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T10:25:51.221 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-09-10T10:25:51.242 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T10:25:51.254 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T10:25:51.277 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T10:25:51.277 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-09-10T10:25:51.280 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-09-10T10:25:51.282 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-09-10T10:25:51.299 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T10:25:51.306 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T10:25:51.306 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-09-10T10:25:51.337 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-09-10T10:25:51.339 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-09-10T10:25:51.575 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T10:25:51.652 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T10:25:51.668 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T10:25:51.668 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-10T10:25:51.668 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-10T10:25:51.669 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T10:25:51.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:25:51.686 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-10T10:25:51.744 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-503.el9.x86_64 2024-09-10T10:25:51.744 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-503.el9.x86_64 2024-09-10T10:25:51.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-09-10T10:25:55.248 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-10T10:25:58.382 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-10T10:26:03.999 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.5 MB 00:01 2024-09-10T10:26:05.616 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 19 kB 00:00 2024-09-10T10:26:06.621 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T10:26:12.268 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-10T10:26:14.352 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T10:26:14.352 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T10:26:14.455 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-10T10:26:14.469 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-10T10:26:14.470 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-10T10:26:14.471 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-09-10T10:26:14.471 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:26:14.471 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 79 M 2024-09-10T10:26:14.471 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 126 M 2024-09-10T10:26:14.471 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-10T10:26:14.580 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 284 kB/s | 23 kB 00:00 2024-09-10T10:26:17.038 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.2 MB/s | 16 MB 00:02 2024-09-10T10:26:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 7.4 MB/s | 29 MB 00:03 2024-09-10T10:26:19.130 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 7.5 MB/s | 35 MB 00:04 2024-09-10T10:26:19.130 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:26:19.130 INFO:teuthology.orchestra.run.smithi007.stdout:Total 17 MB/s | 79 MB 00:04 2024-09-10T10:26:19.846 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-10T10:26:20.217 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-10T10:26:20.218 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-10T10:26:21.204 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-10T10:26:21.208 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-10T10:26:23.771 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-10T10:26:24.464 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T10:26:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:26:27.349 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:26:27.490 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:26:32.617 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:26:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:26:37.784 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:27:17.128 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:27:17.168 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:27:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:27:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T10:27:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T10:27:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T10:27:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T10:27:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:27:18.515 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-10T10:27:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T10:27:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T10:27:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T10:27:18.515 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T10:27:18.516 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:27:18.516 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-10T10:27:18.877 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T10:27:18.966 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T10:27:18.967 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T10:27:18.967 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T10:27:18.967 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T10:27:18.968 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T10:27:18.968 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-10T10:27:18.968 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T10:27:18.968 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T10:27:19.027 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T10:27:19.027 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T10:27:19.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-09-10T10:27:19.063 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-09-10T10:27:19.063 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-09-10T10:27:19.063 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-09-10T10:27:19.063 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 82.el9 2024-09-10T10:27:19.063 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8274670 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T10:27:19.064 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-09-10T10:27:19.065 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-09-10T10:27:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:27:19.065 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T10:27:19.065 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T10:27:19.065 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T10:27:19.065 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-09-10T10:27:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:27:19.065 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T10:27:19.066 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T10:27:19.066 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-505.el9.x86_64 2024-09-10T10:27:19.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T10:27:19.718 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-09-10T10:27:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T10:27:21.073 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-09-10T10:27:21.074 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-10T10:27:21.089 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.JItNy1EQJw 2024-09-10T10:27:21.089 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JItNy1EQJw 2024-09-10T10:27:21.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.JItNy1EQJw 2024-09-10T10:27:21.292 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.JItNy1EQJw is 6KB 2024-09-10T10:27:21.341 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.JItNy1EQJw 2024-09-10T10:27:21.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T10:27:21.418 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T10:27:21.418 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T10:27:21.418 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T10:27:21.418 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T10:27:21.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T10:27:21.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-09-10T10:27:51.584 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T10:27:51.585 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T10:27:51.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:28:51.588 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T10:29:00.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T10:29:00.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:29:19.080 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-10T10:29:19.458 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T10:29:19.458 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-10T10:29:19.458 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-10T10:29:19.474 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-505.el9.x86_64 2024-09-10T10:29:19.474 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T10:29:19.474 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T10:29:19.474 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T10:29:20.476 DEBUG:teuthology.parallel:result is None 2024-09-10T10:29:20.476 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T10:29:20.498 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T10:29:20.499 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T10:29:20.502 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T10:29:20.504 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T10:29:20.524 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T10:29:20.532 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T10:29:20.578 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T10:29:20.579 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T10:29:20.582 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T10:29:20.585 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T10:29:20.687 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T10:29:20.694 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T10:29:20.695 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T10:29:20.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:29:20.712 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T10:29:20.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:29:20.730 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T10:29:20.732 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T10:29:20.755 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T10:29:20.804 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:29:20.812 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:29:20.822 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:29:20.827 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:29:20.846 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:29:20.903 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:29:20.905 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T10:29:20.912 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T10:29:20.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T10:29:20.914 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T10:29:20.916 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T10:29:20.946 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T10:29:20.954 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T10:29:20.954 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T10:29:20.981 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T10:29:20.984 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T10:29:21.004 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:29:21.070 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:29:21.127 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:29:21.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:29:21.251 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:29:21.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T10:29:21.332 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:29:21.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:29:21.443 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:29:21.488 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:29:21.532 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:29:21.532 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T10:29:21.601 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:29:21.646 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:29:21.707 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:29:21.751 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:29:21.794 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:29:21.794 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T10:29:21.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-09-10T10:29:21.863 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-09-10T10:29:21.865 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-09-10T10:29:21.901 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:29:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:29:21.935 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:29:22.376 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T10:29:22.384 INFO:teuthology.task.internal:Starting timer... 2024-09-10T10:29:22.384 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T10:29:22.431 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T10:29:22.462 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T10:29:22.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-09-10T10:29:22.526 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-09-10T10:29:22.528 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-09-10T10:29:22.586 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-09-10T10:29:22.588 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-09-10T10:29:22.633 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-09-10T10:29:22.635 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T10:29:22.635 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-09-10T10:29:22.660 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-09-10T10:29:22.660 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-09-10T10:29:22.683 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-09-10T10:29:22.683 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-09-10T10:29:22.707 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-09-10T10:29:22.707 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi133.front.sepia.ceph.com': 'permissive'} 2024-09-10T10:29:22.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:29:22.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:29:22.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:29:22.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:29:22.763 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:29:22.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:29:22.789 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T10:29:22.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-09-10T10:29:22.824 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-09-10T10:29:22.858 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-09-10T10:29:22.894 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T10:29:22.902 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T10:29:22.913 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T10:29:22.914 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi103.front.sepia.ceph.com,smithi133.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T10:33:28.029 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com')] 2024-09-10T10:33:28.031 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T10:33:28.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:33:28.113 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-10T10:33:28.199 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T10:33:28.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-10T10:33:28.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:33:28.276 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-10T10:33:28.356 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-10T10:33:28.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T10:33:28.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:33:28.437 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-10T10:33:28.517 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T10:33:28.517 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T10:33:28.528 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T10:33:28.528 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T10:33:28.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:33:28.531 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T10:33:28.532 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:33:28.534 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T10:33:28.534 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:33:28.560 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T10:33:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T10:33:28.574 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T10:33:28.582 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T10:33:28.588 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T10:33:28.599 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-09-10T10:33:28.602 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T10:33:28.610 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-09-10T10:33:28.610 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-09-10T10:33:28.622 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-09-10T10:33:28.623 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T10:33:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-09-10T10:33:28.637 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T10:33:28.638 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-09-10T10:33:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T10:33:28.653 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T10:33:28.654 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T10:33:28.666 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T10:33:28.697 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:33:28.701 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:33:28.701 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-10T10:33:28.701 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.701 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.701 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.701 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.709 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:33:28.713 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:33:28.713 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-10T10:33:28.713 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.713 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.713 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.713 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.722 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:33:28.725 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:33:28.725 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-09-10T10:33:28.725 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.726 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.726 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.726 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:33:28.727 INFO:teuthology.run_tasks:Running task install... 2024-09-10T10:33:29.015 DEBUG:teuthology.task.install:project ceph 2024-09-10T10:33:29.016 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T10:33:29.016 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T10:33:29.016 INFO:teuthology.task.install:Using flavor: default 2024-09-10T10:33:29.021 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T10:33:29.021 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T10:33:29.022 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T10:33:29.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:33:29.024 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T10:33:29.025 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:33:29.025 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T10:33:29.026 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:33:29.176 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T10:33:29.176 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T10:33:29.192 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T10:33:29.192 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T10:33:29.195 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T10:33:29.196 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T10:33:29.289 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T10:33:29.289 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:33:29.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T10:33:29.301 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T10:33:29.301 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:33:29.301 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T10:33:29.312 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T10:33:29.312 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:33:29.312 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T10:33:29.318 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T10:33:29.319 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T10:33:29.328 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T10:33:29.329 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T10:33:29.340 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T10:33:29.340 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T10:33:29.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T10:33:29.396 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T10:33:29.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T10:33:29.469 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T10:33:29.471 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T10:33:29.485 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T10:33:29.535 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-09-10T10:33:29.536 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-09-10T10:33:29.537 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-09-10T10:33:29.538 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-09-10T10:33:29.550 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-09-10T10:33:29.552 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-09-10T10:33:29.762 INFO:teuthology.orchestra.run.smithi133.stdout:45 files removed 2024-09-10T10:33:29.778 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-09-10T10:33:29.789 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T10:33:29.794 INFO:teuthology.orchestra.run.smithi007.stdout:45 files removed 2024-09-10T10:33:29.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T10:33:29.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T10:33:30.338 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-10T10:33:30.358 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-10T10:33:30.382 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 321 kB/s | 84 kB 00:00 2024-09-10T10:33:30.621 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T10:33:30.642 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T10:33:30.674 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-10T10:33:30.897 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T10:33:30.926 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-10T10:33:30.958 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-10T10:33:31.492 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-10T10:33:32.414 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.2 MB 00:01 2024-09-10T10:33:33.435 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-10T10:33:34.075 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-10T10:33:34.626 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.2 MB 00:03 2024-09-10T10:33:36.502 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-10T10:33:39.277 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-10T10:33:40.109 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-10T10:33:41.036 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-10T10:33:41.560 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-10T10:33:41.907 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-10T10:33:42.330 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T10:33:42.336 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-10T10:33:44.062 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-10T10:33:45.134 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-10T10:33:47.671 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T10:33:47.770 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T10:33:50.066 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:33:50.067 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:33:50.080 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T10:33:50.081 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T10:33:50.082 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T10:33:50.083 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T10:33:50.085 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T10:33:50.087 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T10:33:50.088 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T10:33:50.099 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:33:50.100 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:33:50.111 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T10:33:50.112 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T10:33:50.113 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T10:33:50.114 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T10:33:50.115 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T10:33:50.118 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T10:33:50.119 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T10:33:50.152 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-09-10T10:33:50.173 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================== 2024-09-10T10:33:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-09-10T10:33:50.173 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================== 2024-09-10T10:33:50.173 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-09-10T10:33:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T10:33:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T10:33:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T10:33:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T10:33:50.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T10:33:50.174 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T10:33:50.175 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T10:33:50.176 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T10:33:50.177 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T10:33:50.178 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T10:33:50.178 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T10:33:50.178 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T10:33:50.178 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T10:33:50.178 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T10:33:50.178 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T10:33:50.178 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T10:33:50.178 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T10:33:50.179 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T10:33:50.180 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T10:33:50.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T10:33:50.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T10:33:50.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T10:33:50.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T10:33:50.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:33:50.186 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-09-10T10:33:50.187 INFO:teuthology.orchestra.run.smithi133.stdout:====================================================================================== 2024-09-10T10:33:50.187 INFO:teuthology.orchestra.run.smithi133.stdout:Install 148 Packages 2024-09-10T10:33:50.187 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 8 Packages 2024-09-10T10:33:50.187 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:33:50.187 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-10T10:33:50.188 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 226 M 2024-09-10T10:33:50.188 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-09-10T10:33:50.205 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-10T10:33:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-10T10:33:50.205 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-10T10:33:50.205 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-10T10:33:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T10:33:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T10:33:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T10:33:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T10:33:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T10:33:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T10:33:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T10:33:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T10:33:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T10:33:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T10:33:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T10:33:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T10:33:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T10:33:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T10:33:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T10:33:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T10:33:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T10:33:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T10:33:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T10:33:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T10:33:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T10:33:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T10:33:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T10:33:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T10:33:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T10:33:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T10:33:50.219 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-09-10T10:33:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T10:33:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:33:50.219 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-10T10:33:50.219 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-10T10:33:50.219 INFO:teuthology.orchestra.run.smithi007.stdout:Install 148 Packages 2024-09-10T10:33:50.219 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 8 Packages 2024-09-10T10:33:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:33:50.220 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 226 M 2024-09-10T10:33:50.220 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-10T10:33:50.646 INFO:teuthology.orchestra.run.smithi133.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-10T10:33:50.764 INFO:teuthology.orchestra.run.smithi007.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-10T10:33:50.879 INFO:teuthology.orchestra.run.smithi133.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 3.8 MB/s | 897 kB 00:00 2024-09-10T10:33:50.971 INFO:teuthology.orchestra.run.smithi133.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.5 MB/s | 145 kB 00:00 2024-09-10T10:33:50.990 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T10:33:51.055 INFO:teuthology.orchestra.run.smithi133.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-10T10:33:51.163 INFO:teuthology.orchestra.run.smithi133.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 14 MB/s | 1.5 MB 00:00 2024-09-10T10:33:51.339 INFO:teuthology.orchestra.run.smithi133.stdout:(6/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 6.5 MB/s | 2.4 MB 00:00 2024-09-10T10:33:51.431 INFO:teuthology.orchestra.run.smithi007.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 1.3 MB/s | 897 kB 00:00 2024-09-10T10:33:51.456 INFO:teuthology.orchestra.run.smithi133.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-10T10:33:51.498 INFO:teuthology.orchestra.run.smithi007.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 2.1 MB/s | 145 kB 00:00 2024-09-10T10:33:51.648 INFO:teuthology.orchestra.run.smithi133.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 19 MB/s | 21 MB 00:01 2024-09-10T10:33:51.689 INFO:teuthology.orchestra.run.smithi133.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 607 kB/s | 25 kB 00:00 2024-09-10T10:33:52.073 INFO:teuthology.orchestra.run.smithi133.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 17 MB/s | 11 MB 00:00 2024-09-10T10:33:52.132 INFO:teuthology.orchestra.run.smithi133.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 3.8 MB/s | 228 kB 00:00 2024-09-10T10:33:52.190 INFO:teuthology.orchestra.run.smithi133.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 465 kB/s | 27 kB 00:00 2024-09-10T10:33:52.198 INFO:teuthology.orchestra.run.smithi007.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 3.4 MB/s | 2.4 MB 00:00 2024-09-10T10:33:52.240 INFO:teuthology.orchestra.run.smithi133.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 634 kB/s | 32 kB 00:00 2024-09-10T10:33:52.274 INFO:teuthology.orchestra.run.smithi007.stdout:(5/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-09-10T10:33:52.316 INFO:teuthology.orchestra.run.smithi133.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 9.8 MB/s | 751 kB 00:00 2024-09-10T10:33:52.391 INFO:teuthology.orchestra.run.smithi133.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.1 MB/s | 164 kB 00:00 2024-09-10T10:33:52.466 INFO:teuthology.orchestra.run.smithi133.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.6 MB/s | 127 kB 00:00 2024-09-10T10:33:52.550 INFO:teuthology.orchestra.run.smithi133.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 5.9 MB/s | 506 kB 00:00 2024-09-10T10:33:52.782 INFO:teuthology.orchestra.run.smithi007.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 2.6 MB/s | 1.5 MB 00:00 2024-09-10T10:33:52.959 INFO:teuthology.orchestra.run.smithi133.stdout:(18/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 10 MB/s | 17 MB 00:01 2024-09-10T10:33:53.101 INFO:teuthology.orchestra.run.smithi133.stdout:(19/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 9.7 MB/s | 5.3 MB 00:00 2024-09-10T10:33:53.125 INFO:teuthology.orchestra.run.smithi133.stdout:(20/156): python3-ceph-argparse-19.1.1-305.g1cf 271 kB/s | 45 kB 00:00 2024-09-10T10:33:53.151 INFO:teuthology.orchestra.run.smithi133.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 2.7 MB/s | 136 kB 00:00 2024-09-10T10:33:53.174 INFO:teuthology.orchestra.run.smithi007.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 5.2 MB/s | 4.7 MB 00:00 2024-09-10T10:33:53.185 INFO:teuthology.orchestra.run.smithi133.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.7 MB/s | 161 kB 00:00 2024-09-10T10:33:53.210 INFO:teuthology.orchestra.run.smithi133.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 5.3 MB/s | 320 kB 00:00 2024-09-10T10:33:53.235 INFO:teuthology.orchestra.run.smithi133.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.9 MB/s | 301 kB 00:00 2024-09-10T10:33:53.260 INFO:teuthology.orchestra.run.smithi133.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T10:33:53.285 INFO:teuthology.orchestra.run.smithi133.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T10:33:53.312 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:33:53.312 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:33:53.326 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T10:33:53.327 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T10:33:53.328 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T10:33:53.329 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T10:33:53.331 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T10:33:53.334 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T10:33:53.335 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T10:33:53.344 INFO:teuthology.orchestra.run.smithi133.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-10T10:33:53.398 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T10:33:53.422 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T10:33:53.423 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T10:33:53.424 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T10:33:53.425 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T10:33:53.425 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T10:33:53.425 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T10:33:53.425 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T10:33:53.425 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T10:33:53.425 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T10:33:53.425 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-09-10T10:33:53.425 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T10:33:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T10:33:53.427 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T10:33:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T10:33:53.429 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T10:33:53.430 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T10:33:53.430 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T10:33:53.430 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T10:33:53.430 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T10:33:53.430 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T10:33:53.430 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T10:33:53.430 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T10:33:53.433 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T10:33:53.434 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T10:33:53.435 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T10:33:53.436 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T10:33:53.437 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T10:33:53.438 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-09-10T10:33:53.438 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T10:33:53.438 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:33:53.438 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-10T10:33:53.438 INFO:teuthology.orchestra.run.smithi103.stdout:====================================================================================== 2024-09-10T10:33:53.438 INFO:teuthology.orchestra.run.smithi103.stdout:Install 148 Packages 2024-09-10T10:33:53.438 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 8 Packages 2024-09-10T10:33:53.438 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:33:53.439 INFO:teuthology.orchestra.run.smithi133.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 382 kB/s | 32 kB 00:00 2024-09-10T10:33:53.439 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 226 M 2024-09-10T10:33:53.439 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-10T10:33:53.478 INFO:teuthology.orchestra.run.smithi133.stdout:(29/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 14 MB/s | 3.1 MB 00:00 2024-09-10T10:33:53.503 INFO:teuthology.orchestra.run.smithi133.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 1.9 MB/s | 146 kB 00:00 2024-09-10T10:33:53.770 INFO:teuthology.orchestra.run.smithi133.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 12 MB/s | 3.6 MB 00:00 2024-09-10T10:33:53.854 INFO:teuthology.orchestra.run.smithi133.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.9 MB/s | 249 kB 00:00 2024-09-10T10:33:53.883 INFO:teuthology.orchestra.run.smithi103.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 30 kB/s | 6.5 kB 00:00 2024-09-10T10:33:54.108 INFO:teuthology.orchestra.run.smithi103.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 3.9 MB/s | 897 kB 00:00 2024-09-10T10:33:54.192 INFO:teuthology.orchestra.run.smithi103.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.7 MB/s | 145 kB 00:00 2024-09-10T10:33:54.279 INFO:teuthology.orchestra.run.smithi133.stdout:(33/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 19 MB/s | 49 MB 00:02 2024-09-10T10:33:54.304 INFO:teuthology.orchestra.run.smithi133.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 109 kB/s | 49 kB 00:00 2024-09-10T10:33:54.329 INFO:teuthology.orchestra.run.smithi133.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 330 kB/s | 16 kB 00:00 2024-09-10T10:33:54.342 INFO:teuthology.orchestra.run.smithi103.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 8.0 MB/s | 5.4 MB 00:00 2024-09-10T10:33:54.422 INFO:teuthology.orchestra.run.smithi133.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 8.0 MB/s | 7.4 MB 00:00 2024-09-10T10:33:54.447 INFO:teuthology.orchestra.run.smithi133.stdout:(37/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 2.0 MB/s | 285 kB 00:00 2024-09-10T10:33:54.498 INFO:teuthology.orchestra.run.smithi133.stdout:(38/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 4.5 MB/s | 766 kB 00:00 2024-09-10T10:33:54.501 INFO:teuthology.orchestra.run.smithi103.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-10T10:33:54.514 INFO:teuthology.orchestra.run.smithi133.stdout:(39/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 199 kB/s | 18 kB 00:00 2024-09-10T10:33:54.575 INFO:teuthology.orchestra.run.smithi007.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 5.4 MB/s | 21 MB 00:03 2024-09-10T10:33:54.601 INFO:teuthology.orchestra.run.smithi103.stdout:(6/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 5.8 MB/s | 2.4 MB 00:00 2024-09-10T10:33:54.625 INFO:teuthology.orchestra.run.smithi007.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 505 kB/s | 25 kB 00:00 2024-09-10T10:33:54.631 INFO:teuthology.orchestra.run.smithi133.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 434 kB/s | 80 kB 00:00 2024-09-10T10:33:54.715 INFO:teuthology.orchestra.run.smithi133.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 955 kB/s | 191 kB 00:00 2024-09-10T10:33:54.765 INFO:teuthology.orchestra.run.smithi133.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-10T10:33:54.790 INFO:teuthology.orchestra.run.smithi133.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-10T10:33:54.816 INFO:teuthology.orchestra.run.smithi133.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T10:33:54.818 INFO:teuthology.orchestra.run.smithi103.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 15 MB/s | 4.7 MB 00:00 2024-09-10T10:33:54.841 INFO:teuthology.orchestra.run.smithi133.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T10:33:54.866 INFO:teuthology.orchestra.run.smithi133.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T10:33:54.900 INFO:teuthology.orchestra.run.smithi133.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-10T10:33:54.983 INFO:teuthology.orchestra.run.smithi133.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-09-10T10:33:55.035 INFO:teuthology.orchestra.run.smithi103.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 15 MB/s | 21 MB 00:01 2024-09-10T10:33:55.084 INFO:teuthology.orchestra.run.smithi133.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-10T10:33:55.093 INFO:teuthology.orchestra.run.smithi103.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 433 kB/s | 25 kB 00:00 2024-09-10T10:33:55.100 INFO:teuthology.orchestra.run.smithi007.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 5.5 MB/s | 11 MB 00:01 2024-09-10T10:33:55.109 INFO:teuthology.orchestra.run.smithi133.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 853 kB/s | 107 kB 00:00 2024-09-10T10:33:55.134 INFO:teuthology.orchestra.run.smithi133.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-10T10:33:55.168 INFO:teuthology.orchestra.run.smithi133.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-10T10:33:55.193 INFO:teuthology.orchestra.run.smithi133.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-10T10:33:55.200 INFO:teuthology.orchestra.run.smithi007.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 2.2 MB/s | 228 kB 00:00 2024-09-10T10:33:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-10T10:33:55.243 INFO:teuthology.orchestra.run.smithi133.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-10T10:33:55.269 INFO:teuthology.orchestra.run.smithi133.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T10:33:55.294 INFO:teuthology.orchestra.run.smithi133.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-10T10:33:55.300 INFO:teuthology.orchestra.run.smithi007.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 271 kB/s | 27 kB 00:00 2024-09-10T10:33:55.319 INFO:teuthology.orchestra.run.smithi133.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-09-10T10:33:55.367 INFO:teuthology.orchestra.run.smithi007.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 475 kB/s | 32 kB 00:00 2024-09-10T10:33:55.369 INFO:teuthology.orchestra.run.smithi133.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-10T10:33:55.394 INFO:teuthology.orchestra.run.smithi103.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 18 MB/s | 11 MB 00:00 2024-09-10T10:33:55.420 INFO:teuthology.orchestra.run.smithi133.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-10T10:33:55.445 INFO:teuthology.orchestra.run.smithi133.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 326 kB/s | 24 kB 00:00 2024-09-10T10:33:55.461 INFO:teuthology.orchestra.run.smithi103.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 3.3 MB/s | 228 kB 00:00 2024-09-10T10:33:55.470 INFO:teuthology.orchestra.run.smithi133.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-09-10T10:33:55.495 INFO:teuthology.orchestra.run.smithi133.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-10T10:33:55.509 INFO:teuthology.orchestra.run.smithi007.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 5.2 MB/s | 751 kB 00:00 2024-09-10T10:33:55.520 INFO:teuthology.orchestra.run.smithi133.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T10:33:55.528 INFO:teuthology.orchestra.run.smithi103.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 407 kB/s | 27 kB 00:00 2024-09-10T10:33:55.546 INFO:teuthology.orchestra.run.smithi133.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 771 kB/s | 38 kB 00:00 2024-09-10T10:33:55.571 INFO:teuthology.orchestra.run.smithi133.stdout:(66/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-10T10:33:55.576 INFO:teuthology.orchestra.run.smithi007.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.4 MB/s | 164 kB 00:00 2024-09-10T10:33:55.595 INFO:teuthology.orchestra.run.smithi103.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 475 kB/s | 32 kB 00:00 2024-09-10T10:33:55.643 INFO:teuthology.orchestra.run.smithi007.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.9 MB/s | 127 kB 00:00 2024-09-10T10:33:55.663 INFO:teuthology.orchestra.run.smithi133.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 35 MB/s | 4.9 MB 00:00 2024-09-10T10:33:55.687 INFO:teuthology.orchestra.run.smithi103.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 8.0 MB/s | 751 kB 00:00 2024-09-10T10:33:55.739 INFO:teuthology.orchestra.run.smithi133.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-09-10T10:33:55.762 INFO:teuthology.orchestra.run.smithi103.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.1 MB/s | 164 kB 00:00 2024-09-10T10:33:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 4.0 MB/s | 506 kB 00:00 2024-09-10T10:33:55.781 INFO:teuthology.orchestra.run.smithi133.stdout:(69/156): postgresql-13.16-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-10T10:33:55.806 INFO:teuthology.orchestra.run.smithi133.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-10T10:33:55.829 INFO:teuthology.orchestra.run.smithi103.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.9 MB/s | 127 kB 00:00 2024-09-10T10:33:55.848 INFO:teuthology.orchestra.run.smithi133.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.8 MB/s | 910 kB 00:00 2024-09-10T10:33:55.881 INFO:teuthology.orchestra.run.smithi133.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-09-10T10:33:55.906 INFO:teuthology.orchestra.run.smithi133.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-10T10:33:55.938 INFO:teuthology.orchestra.run.smithi103.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 4.6 MB/s | 506 kB 00:00 2024-09-10T10:33:55.940 INFO:teuthology.orchestra.run.smithi133.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-10T10:33:56.015 INFO:teuthology.orchestra.run.smithi133.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-09-10T10:33:56.040 INFO:teuthology.orchestra.run.smithi133.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-10T10:33:56.066 INFO:teuthology.orchestra.run.smithi133.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-10T10:33:56.091 INFO:teuthology.orchestra.run.smithi133.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-10T10:33:56.116 INFO:teuthology.orchestra.run.smithi133.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-10T10:33:56.130 INFO:teuthology.orchestra.run.smithi103.stdout:(18/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 17 MB 00:01 2024-09-10T10:33:56.150 INFO:teuthology.orchestra.run.smithi133.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-10T10:33:56.175 INFO:teuthology.orchestra.run.smithi133.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-09-10T10:33:56.180 INFO:teuthology.orchestra.run.smithi103.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 905 kB/s | 45 kB 00:00 2024-09-10T10:33:56.251 INFO:teuthology.orchestra.run.smithi133.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-10T10:33:56.276 INFO:teuthology.orchestra.run.smithi133.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-10T10:33:56.280 INFO:teuthology.orchestra.run.smithi103.stdout:(20/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-10T10:33:56.306 INFO:teuthology.orchestra.run.smithi103.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 1.1 MB/s | 136 kB 00:00 2024-09-10T10:33:56.309 INFO:teuthology.orchestra.run.smithi133.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-10T10:33:56.335 INFO:teuthology.orchestra.run.smithi133.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T10:33:56.339 INFO:teuthology.orchestra.run.smithi103.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.7 MB/s | 161 kB 00:00 2024-09-10T10:33:56.360 INFO:teuthology.orchestra.run.smithi133.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-10T10:33:56.381 INFO:teuthology.orchestra.run.smithi103.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 4.1 MB/s | 320 kB 00:00 2024-09-10T10:33:56.385 INFO:teuthology.orchestra.run.smithi133.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-10T10:33:56.407 INFO:teuthology.orchestra.run.smithi103.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 4.4 MB/s | 301 kB 00:00 2024-09-10T10:33:56.410 INFO:teuthology.orchestra.run.smithi133.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T10:33:56.436 INFO:teuthology.orchestra.run.smithi133.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-10T10:33:56.449 INFO:teuthology.orchestra.run.smithi103.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.5 MB/s | 99 kB 00:00 2024-09-10T10:33:56.461 INFO:teuthology.orchestra.run.smithi133.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T10:33:56.474 INFO:teuthology.orchestra.run.smithi103.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-10T10:33:56.494 INFO:teuthology.orchestra.run.smithi133.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 923 kB/s | 54 kB 00:00 2024-09-10T10:33:56.549 INFO:teuthology.orchestra.run.smithi103.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.2 MB/s | 170 kB 00:00 2024-09-10T10:33:56.570 INFO:teuthology.orchestra.run.smithi133.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-10T10:33:56.595 INFO:teuthology.orchestra.run.smithi133.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-10T10:33:56.608 INFO:teuthology.orchestra.run.smithi103.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 545 kB/s | 32 kB 00:00 2024-09-10T10:33:56.620 INFO:teuthology.orchestra.run.smithi133.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-10T10:33:56.635 INFO:teuthology.orchestra.run.smithi007.stdout:(18/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 6.2 MB/s | 5.3 MB 00:00 2024-09-10T10:33:56.645 INFO:teuthology.orchestra.run.smithi133.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-10T10:33:56.670 INFO:teuthology.orchestra.run.smithi133.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-10T10:33:56.675 INFO:teuthology.orchestra.run.smithi103.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.1 MB/s | 146 kB 00:00 2024-09-10T10:33:56.696 INFO:teuthology.orchestra.run.smithi133.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-10T10:33:56.730 INFO:teuthology.orchestra.run.smithi133.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-10T10:33:56.733 INFO:teuthology.orchestra.run.smithi103.stdout:(30/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 3.1 MB 00:00 2024-09-10T10:33:56.754 INFO:teuthology.orchestra.run.smithi133.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 990 kB/s | 58 kB 00:00 2024-09-10T10:33:56.780 INFO:teuthology.orchestra.run.smithi133.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T10:33:56.805 INFO:teuthology.orchestra.run.smithi133.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T10:33:56.810 INFO:teuthology.orchestra.run.smithi007.stdout:(19/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 4.2 MB/s | 17 MB 00:04 2024-09-10T10:33:56.817 INFO:teuthology.orchestra.run.smithi103.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 25 MB/s | 3.6 MB 00:00 2024-09-10T10:33:56.827 INFO:teuthology.orchestra.run.smithi007.stdout:(20/156): python3-ceph-argparse-19.1.1-305.g1cf 235 kB/s | 45 kB 00:00 2024-09-10T10:33:56.830 INFO:teuthology.orchestra.run.smithi133.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-10T10:33:56.884 INFO:teuthology.orchestra.run.smithi103.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 3.7 MB/s | 249 kB 00:00 2024-09-10T10:33:56.886 INFO:teuthology.orchestra.run.smithi007.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.7 MB/s | 161 kB 00:00 2024-09-10T10:33:56.911 INFO:teuthology.orchestra.run.smithi007.stdout:(22/156): python3-ceph-common-19.1.1-305.g1cf81 1.3 MB/s | 136 kB 00:00 2024-09-10T10:33:56.926 INFO:teuthology.orchestra.run.smithi103.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 1.2 MB/s | 49 kB 00:00 2024-09-10T10:33:56.985 INFO:teuthology.orchestra.run.smithi103.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 282 kB/s | 16 kB 00:00 2024-09-10T10:33:56.986 INFO:teuthology.orchestra.run.smithi007.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 3.1 MB/s | 320 kB 00:00 2024-09-10T10:33:57.012 INFO:teuthology.orchestra.run.smithi007.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 2.9 MB/s | 301 kB 00:00 2024-09-10T10:33:57.052 INFO:teuthology.orchestra.run.smithi103.stdout:(35/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.2 MB/s | 285 kB 00:00 2024-09-10T10:33:57.054 INFO:teuthology.orchestra.run.smithi007.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.5 MB/s | 99 kB 00:00 2024-09-10T10:33:57.064 INFO:teuthology.orchestra.run.smithi133.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-10T10:33:57.079 INFO:teuthology.orchestra.run.smithi007.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-10T10:33:57.114 INFO:teuthology.orchestra.run.smithi133.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T10:33:57.144 INFO:teuthology.orchestra.run.smithi103.stdout:(36/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 8.2 MB/s | 766 kB 00:00 2024-09-10T10:33:57.148 INFO:teuthology.orchestra.run.smithi133.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-09-10T10:33:57.163 INFO:teuthology.orchestra.run.smithi007.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.0 MB/s | 170 kB 00:00 2024-09-10T10:33:57.173 INFO:teuthology.orchestra.run.smithi133.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-10T10:33:57.203 INFO:teuthology.orchestra.run.smithi103.stdout:(37/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 314 kB/s | 18 kB 00:00 2024-09-10T10:33:57.221 INFO:teuthology.orchestra.run.smithi007.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 547 kB/s | 32 kB 00:00 2024-09-10T10:33:57.272 INFO:teuthology.orchestra.run.smithi007.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T10:33:57.282 INFO:teuthology.orchestra.run.smithi133.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-10T10:33:57.307 INFO:teuthology.orchestra.run.smithi133.stdout:(108/156): lua-devel-5.4.4-4.el9.x86_64.rpm 44 kB/s | 22 kB 00:00 2024-09-10T10:33:57.333 INFO:teuthology.orchestra.run.smithi133.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-09-10T10:33:57.366 INFO:teuthology.orchestra.run.smithi133.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-10T10:33:57.391 INFO:teuthology.orchestra.run.smithi133.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-10T10:33:57.417 INFO:teuthology.orchestra.run.smithi133.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 446 kB/s | 49 kB 00:00 2024-09-10T10:33:57.467 INFO:teuthology.orchestra.run.smithi133.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-10T10:33:57.479 INFO:teuthology.orchestra.run.smithi007.stdout:(30/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 7.3 MB/s | 3.1 MB 00:00 2024-09-10T10:33:57.492 INFO:teuthology.orchestra.run.smithi133.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T10:33:57.517 INFO:teuthology.orchestra.run.smithi133.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-09-10T10:33:57.543 INFO:teuthology.orchestra.run.smithi133.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-09-10T10:33:57.568 INFO:teuthology.orchestra.run.smithi133.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-10T10:33:57.593 INFO:teuthology.orchestra.run.smithi133.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-10T10:33:57.618 INFO:teuthology.orchestra.run.smithi133.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-10T10:33:57.645 INFO:teuthology.orchestra.run.smithi103.stdout:(38/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 19 MB/s | 49 MB 00:02 2024-09-10T10:33:57.671 INFO:teuthology.orchestra.run.smithi133.stdout:(120/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-09-10T10:33:57.702 INFO:teuthology.orchestra.run.smithi133.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-09-10T10:33:57.722 INFO:teuthology.orchestra.run.smithi007.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 7.9 MB/s | 3.6 MB 00:00 2024-09-10T10:33:57.728 INFO:teuthology.orchestra.run.smithi103.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 7.4 MB/s | 7.4 MB 00:00 2024-09-10T10:33:57.736 INFO:teuthology.orchestra.run.smithi133.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-09-10T10:33:57.753 INFO:teuthology.orchestra.run.smithi133.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 722 kB/s | 11 kB 00:00 2024-09-10T10:33:57.754 INFO:teuthology.orchestra.run.smithi103.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 145 kB/s | 80 kB 00:00 2024-09-10T10:33:57.778 INFO:teuthology.orchestra.run.smithi133.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-09-10T10:33:57.789 INFO:teuthology.orchestra.run.smithi007.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 3.7 MB/s | 249 kB 00:00 2024-09-10T10:33:57.803 INFO:teuthology.orchestra.run.smithi133.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-10T10:33:57.829 INFO:teuthology.orchestra.run.smithi133.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-10T10:33:57.839 INFO:teuthology.orchestra.run.smithi007.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 989 kB/s | 49 kB 00:00 2024-09-10T10:33:57.854 INFO:teuthology.orchestra.run.smithi133.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-10T10:33:57.879 INFO:teuthology.orchestra.run.smithi133.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-10T10:33:57.890 INFO:teuthology.orchestra.run.smithi007.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 330 kB/s | 16 kB 00:00 2024-09-10T10:33:57.896 INFO:teuthology.orchestra.run.smithi103.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-10T10:33:57.904 INFO:teuthology.orchestra.run.smithi133.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T10:33:57.921 INFO:teuthology.orchestra.run.smithi103.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 995 kB/s | 191 kB 00:00 2024-09-10T10:33:57.930 INFO:teuthology.orchestra.run.smithi133.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T10:33:57.955 INFO:teuthology.orchestra.run.smithi133.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T10:33:57.965 INFO:teuthology.orchestra.run.smithi007.stdout:(35/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 3.7 MB/s | 285 kB 00:00 2024-09-10T10:33:57.971 INFO:teuthology.orchestra.run.smithi103.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T10:33:57.997 INFO:teuthology.orchestra.run.smithi133.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-10T10:33:58.013 INFO:teuthology.orchestra.run.smithi103.stdout:(44/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.1 MB/s | 801 kB 00:00 2024-09-10T10:33:58.022 INFO:teuthology.orchestra.run.smithi133.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-10T10:33:58.039 INFO:teuthology.orchestra.run.smithi103.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-10T10:33:58.048 INFO:teuthology.orchestra.run.smithi133.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-10T10:33:58.073 INFO:teuthology.orchestra.run.smithi133.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T10:33:58.082 INFO:teuthology.orchestra.run.smithi007.stdout:(36/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 6.4 MB/s | 766 kB 00:00 2024-09-10T10:33:58.166 INFO:teuthology.orchestra.run.smithi103.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-10T10:33:58.167 INFO:teuthology.orchestra.run.smithi007.stdout:(37/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 441 kB/s | 18 kB 00:00 2024-09-10T10:33:58.167 INFO:teuthology.orchestra.run.smithi133.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-10T10:33:58.167 INFO:teuthology.orchestra.run.smithi133.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-09-10T10:33:58.184 INFO:teuthology.orchestra.run.smithi133.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 986 kB/s | 59 kB 00:00 2024-09-10T10:33:58.206 INFO:teuthology.orchestra.run.smithi103.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 649 kB/s | 125 kB 00:00 2024-09-10T10:33:58.207 INFO:teuthology.orchestra.run.smithi133.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-10T10:33:58.233 INFO:teuthology.orchestra.run.smithi133.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 737 kB/s | 36 kB 00:00 2024-09-10T10:33:58.241 INFO:teuthology.orchestra.run.smithi007.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 9.7 MB/s | 7.4 MB 00:00 2024-09-10T10:33:58.249 INFO:teuthology.orchestra.run.smithi133.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-10T10:33:58.258 INFO:teuthology.orchestra.run.smithi007.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 597 kB/s | 80 kB 00:00 2024-09-10T10:33:58.275 INFO:teuthology.orchestra.run.smithi133.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T10:33:58.281 INFO:teuthology.orchestra.run.smithi103.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-09-10T10:33:58.300 INFO:teuthology.orchestra.run.smithi133.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-10T10:33:58.325 INFO:teuthology.orchestra.run.smithi133.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-09-10T10:33:58.333 INFO:teuthology.orchestra.run.smithi007.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-10T10:33:58.340 INFO:teuthology.orchestra.run.smithi103.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-10T10:33:58.376 INFO:teuthology.orchestra.run.smithi133.stdout:(145/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-09-10T10:33:58.382 INFO:teuthology.orchestra.run.smithi103.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-09-10T10:33:58.383 INFO:teuthology.orchestra.run.smithi007.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-09-10T10:33:58.401 INFO:teuthology.orchestra.run.smithi133.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-10T10:33:58.407 INFO:teuthology.orchestra.run.smithi103.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-09-10T10:33:58.417 INFO:teuthology.orchestra.run.smithi007.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-10T10:33:58.434 INFO:teuthology.orchestra.run.smithi133.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-09-10T10:33:58.441 INFO:teuthology.orchestra.run.smithi103.stdout:(52/156): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-09-10T10:33:58.451 INFO:teuthology.orchestra.run.smithi007.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-09-10T10:33:58.466 INFO:teuthology.orchestra.run.smithi103.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-09-10T10:33:58.485 INFO:teuthology.orchestra.run.smithi007.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-09-10T10:33:58.485 INFO:teuthology.orchestra.run.smithi133.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-10T10:33:58.500 INFO:teuthology.orchestra.run.smithi103.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-10T10:33:58.510 INFO:teuthology.orchestra.run.smithi007.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-10T10:33:58.525 INFO:teuthology.orchestra.run.smithi103.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-10T10:33:58.535 INFO:teuthology.orchestra.run.smithi007.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T10:33:58.550 INFO:teuthology.orchestra.run.smithi103.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T10:33:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-10T10:33:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-10T10:33:58.609 INFO:teuthology.orchestra.run.smithi103.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-10T10:33:58.702 INFO:teuthology.orchestra.run.smithi133.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 12 MB/s | 3.2 MB 00:00 2024-09-10T10:33:58.760 INFO:teuthology.orchestra.run.smithi133.stdout:(150/156): librados2-19.1.1-305.g1cf819b7.el9.x 9.5 MB/s | 3.4 MB 00:00 2024-09-10T10:33:58.776 INFO:teuthology.orchestra.run.smithi103.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-10T10:33:58.810 INFO:teuthology.orchestra.run.smithi103.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-09-10T10:33:58.827 INFO:teuthology.orchestra.run.smithi103.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 484 kB/s | 24 kB 00:00 2024-09-10T10:33:58.844 INFO:teuthology.orchestra.run.smithi133.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 489 kB/s | 41 kB 00:00 2024-09-10T10:33:58.853 INFO:teuthology.orchestra.run.smithi103.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-10T10:33:58.886 INFO:teuthology.orchestra.run.smithi103.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-10T10:33:58.928 INFO:teuthology.orchestra.run.smithi103.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 926 kB/s | 38 kB 00:00 2024-09-10T10:33:58.954 INFO:teuthology.orchestra.run.smithi103.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-10T10:33:58.960 INFO:teuthology.orchestra.run.smithi133.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.1 MB/s | 249 kB 00:00 2024-09-10T10:33:59.011 INFO:teuthology.orchestra.run.smithi133.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 5.8 MB/s | 1.8 MB 00:00 2024-09-10T10:33:59.111 INFO:teuthology.orchestra.run.smithi133.stdout:(154/156): NetworkManager-1.49.90-1.el9.x86_64. 3.7 MB/s | 2.3 MB 00:00 2024-09-10T10:33:59.154 INFO:teuthology.orchestra.run.smithi103.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.2 MB/s | 3.0 MB 00:00 2024-09-10T10:33:59.270 INFO:teuthology.orchestra.run.smithi133.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.4 MB/s | 1.4 MB 00:00 2024-09-10T10:33:59.419 INFO:teuthology.orchestra.run.smithi007.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 780 kB/s | 689 kB 00:00 2024-09-10T10:33:59.587 INFO:teuthology.orchestra.run.smithi103.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-10T10:33:59.753 INFO:teuthology.orchestra.run.smithi133.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.2 MB/s | 2.4 MB 00:00 2024-09-10T10:33:59.759 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:33:59.760 INFO:teuthology.orchestra.run.smithi133.stdout:Total 24 MB/s | 226 MB 00:09 2024-09-10T10:33:59.863 INFO:teuthology.orchestra.run.smithi103.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.2 MB/s | 910 kB 00:00 2024-09-10T10:33:59.911 INFO:teuthology.orchestra.run.smithi007.stdout:(49/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 9.2 MB/s | 49 MB 00:05 2024-09-10T10:33:59.930 INFO:teuthology.orchestra.run.smithi103.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-10T10:33:59.953 INFO:teuthology.orchestra.run.smithi007.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 677 kB/s | 943 kB 00:01 2024-09-10T10:34:00.222 INFO:teuthology.orchestra.run.smithi103.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.8 MB/s | 4.9 MB 00:01 2024-09-10T10:34:00.280 INFO:teuthology.orchestra.run.smithi103.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-09-10T10:34:00.339 INFO:teuthology.orchestra.run.smithi103.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-10T10:34:00.478 INFO:teuthology.orchestra.run.smithi007.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 38 kB/s | 22 kB 00:00 2024-09-10T10:34:00.512 INFO:teuthology.orchestra.run.smithi007.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 98 kB/s | 107 kB 00:01 2024-09-10T10:34:00.565 INFO:teuthology.orchestra.run.smithi103.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-10T10:34:00.606 INFO:teuthology.orchestra.run.smithi103.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T10:34:00.704 INFO:teuthology.orchestra.run.smithi103.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.5 MB/s | 245 kB 00:00 2024-09-10T10:34:00.796 INFO:teuthology.orchestra.run.smithi103.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-10T10:34:00.803 INFO:teuthology.orchestra.run.smithi007.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 369 kB/s | 313 kB 00:00 2024-09-10T10:34:00.838 INFO:teuthology.orchestra.run.smithi103.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-09-10T10:34:00.880 INFO:teuthology.orchestra.run.smithi103.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-09-10T10:34:00.955 INFO:teuthology.orchestra.run.smithi103.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-10T10:34:00.991 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-09-10T10:34:00.997 INFO:teuthology.orchestra.run.smithi103.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 836 kB/s | 35 kB 00:00 2024-09-10T10:34:01.021 INFO:teuthology.orchestra.run.smithi007.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-09-10T10:34:01.143 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-09-10T10:34:01.143 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-09-10T10:34:01.329 INFO:teuthology.orchestra.run.smithi007.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-09-10T10:34:01.513 INFO:teuthology.orchestra.run.smithi007.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-10T10:34:01.572 INFO:teuthology.orchestra.run.smithi007.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-10T10:34:01.673 INFO:teuthology.orchestra.run.smithi103.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.3 MB/s | 5.8 MB 00:01 2024-09-10T10:34:01.756 INFO:teuthology.orchestra.run.smithi103.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-10T10:34:01.856 INFO:teuthology.orchestra.run.smithi103.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-10T10:34:01.915 INFO:teuthology.orchestra.run.smithi103.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T10:34:01.957 INFO:teuthology.orchestra.run.smithi103.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-10T10:34:02.074 INFO:teuthology.orchestra.run.smithi103.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-09-10T10:34:02.080 INFO:teuthology.orchestra.run.smithi007.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 214 kB/s | 160 kB 00:00 2024-09-10T10:34:02.105 INFO:teuthology.orchestra.run.smithi007.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 76 kB/s | 45 kB 00:00 2024-09-10T10:34:02.199 INFO:teuthology.orchestra.run.smithi103.stdout:(87/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-09-10T10:34:02.224 INFO:teuthology.orchestra.run.smithi103.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-09-10T10:34:02.267 INFO:teuthology.orchestra.run.smithi103.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-10T10:34:02.309 INFO:teuthology.orchestra.run.smithi103.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-10T10:34:02.334 INFO:teuthology.orchestra.run.smithi103.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-09-10T10:34:02.376 INFO:teuthology.orchestra.run.smithi103.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T10:34:02.418 INFO:teuthology.orchestra.run.smithi103.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-10T10:34:02.455 INFO:teuthology.orchestra.run.smithi007.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 758 kB/s | 662 kB 00:00 2024-09-10T10:34:02.501 INFO:teuthology.orchestra.run.smithi103.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-10T10:34:02.543 INFO:teuthology.orchestra.run.smithi103.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-09-10T10:34:02.585 INFO:teuthology.orchestra.run.smithi103.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-10T10:34:02.639 INFO:teuthology.orchestra.run.smithi007.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 46 kB/s | 24 kB 00:00 2024-09-10T10:34:02.661 INFO:teuthology.orchestra.run.smithi103.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-10T10:34:02.694 INFO:teuthology.orchestra.run.smithi103.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-09-10T10:34:02.753 INFO:teuthology.orchestra.run.smithi103.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.8 MB/s | 304 kB 00:00 2024-09-10T10:34:02.778 INFO:teuthology.orchestra.run.smithi103.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-10T10:34:02.820 INFO:teuthology.orchestra.run.smithi103.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 949 kB/s | 64 kB 00:00 2024-09-10T10:34:02.837 INFO:teuthology.orchestra.run.smithi103.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-09-10T10:34:02.946 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-09-10T10:34:02.953 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-09-10T10:34:03.031 INFO:teuthology.orchestra.run.smithi007.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 612 kB/s | 581 kB 00:00 2024-09-10T10:34:03.056 INFO:teuthology.orchestra.run.smithi007.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 79 kB/s | 47 kB 00:00 2024-09-10T10:34:03.413 INFO:teuthology.orchestra.run.smithi103.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2024-09-10T10:34:03.445 INFO:teuthology.orchestra.run.smithi007.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 365 kB/s | 292 kB 00:00 2024-09-10T10:34:03.471 INFO:teuthology.orchestra.run.smithi103.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-10T10:34:03.504 INFO:teuthology.orchestra.run.smithi103.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-10T10:34:03.522 INFO:teuthology.orchestra.run.smithi103.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-10T10:34:03.661 INFO:teuthology.orchestra.run.smithi103.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-10T10:34:03.675 INFO:teuthology.orchestra.run.smithi103.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T10:34:03.695 INFO:teuthology.orchestra.run.smithi103.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-10T10:34:03.696 INFO:teuthology.orchestra.run.smithi007.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 61 kB/s | 38 kB 00:00 2024-09-10T10:34:03.724 INFO:teuthology.orchestra.run.smithi103.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T10:34:03.751 INFO:teuthology.orchestra.run.smithi103.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-10T10:34:03.782 INFO:teuthology.orchestra.run.smithi103.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 922 kB/s | 863 kB 00:00 2024-09-10T10:34:03.782 INFO:teuthology.orchestra.run.smithi007.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 254 kB/s | 188 kB 00:00 2024-09-10T10:34:03.818 INFO:teuthology.orchestra.run.smithi103.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-10T10:34:03.842 INFO:teuthology.orchestra.run.smithi103.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-09-10T10:34:03.867 INFO:teuthology.orchestra.run.smithi103.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T10:34:03.903 INFO:teuthology.orchestra.run.smithi103.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-10T10:34:03.918 INFO:teuthology.orchestra.run.smithi103.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-10T10:34:03.951 INFO:teuthology.orchestra.run.smithi103.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-09-10T10:34:03.977 INFO:teuthology.orchestra.run.smithi103.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T10:34:04.001 INFO:teuthology.orchestra.run.smithi103.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-10T10:34:04.035 INFO:teuthology.orchestra.run.smithi103.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T10:34:04.060 INFO:teuthology.orchestra.run.smithi103.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T10:34:04.086 INFO:teuthology.orchestra.run.smithi103.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-10T10:34:04.111 INFO:teuthology.orchestra.run.smithi103.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-10T10:34:04.136 INFO:teuthology.orchestra.run.smithi103.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 934 kB/s | 23 kB 00:00 2024-09-10T10:34:04.161 INFO:teuthology.orchestra.run.smithi103.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-10T10:34:04.187 INFO:teuthology.orchestra.run.smithi103.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 777 kB/s | 20 kB 00:00 2024-09-10T10:34:04.238 INFO:teuthology.orchestra.run.smithi103.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.6 MB/s | 2.0 MB 00:00 2024-09-10T10:34:04.254 INFO:teuthology.orchestra.run.smithi103.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-10T10:34:04.280 INFO:teuthology.orchestra.run.smithi103.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T10:34:04.297 INFO:teuthology.orchestra.run.smithi103.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-10T10:34:04.339 INFO:teuthology.orchestra.run.smithi103.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-10T10:34:04.364 INFO:teuthology.orchestra.run.smithi103.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-10T10:34:04.389 INFO:teuthology.orchestra.run.smithi103.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-10T10:34:04.415 INFO:teuthology.orchestra.run.smithi103.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T10:34:04.440 INFO:teuthology.orchestra.run.smithi103.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-10T10:34:04.466 INFO:teuthology.orchestra.run.smithi103.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-10T10:34:04.491 INFO:teuthology.orchestra.run.smithi103.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T10:34:04.524 INFO:teuthology.orchestra.run.smithi103.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-10T10:34:04.541 INFO:teuthology.orchestra.run.smithi007.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.5 MB/s | 4.9 MB 00:01 2024-09-10T10:34:04.550 INFO:teuthology.orchestra.run.smithi103.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-09-10T10:34:04.575 INFO:teuthology.orchestra.run.smithi103.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T10:34:04.600 INFO:teuthology.orchestra.run.smithi103.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T10:34:04.626 INFO:teuthology.orchestra.run.smithi103.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T10:34:04.651 INFO:teuthology.orchestra.run.smithi103.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-10T10:34:04.685 INFO:teuthology.orchestra.run.smithi103.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-10T10:34:04.718 INFO:teuthology.orchestra.run.smithi103.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-10T10:34:04.757 INFO:teuthology.orchestra.run.smithi007.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-09-10T10:34:04.769 INFO:teuthology.orchestra.run.smithi103.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-10T10:34:04.924 INFO:teuthology.orchestra.run.smithi007.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-09-10T10:34:04.986 INFO:teuthology.orchestra.run.smithi103.stdout:(148/156): librados2-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-10T10:34:05.053 INFO:teuthology.orchestra.run.smithi103.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-10T10:34:05.466 INFO:teuthology.orchestra.run.smithi007.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 983 kB/s | 910 kB 00:00 2024-09-10T10:34:05.508 INFO:teuthology.orchestra.run.smithi007.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 185 kB/s | 137 kB 00:00 2024-09-10T10:34:05.511 INFO:teuthology.orchestra.run.smithi103.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 4.4 MB/s | 2.3 MB 00:00 2024-09-10T10:34:05.562 INFO:teuthology.orchestra.run.smithi103.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 817 kB/s | 41 kB 00:00 2024-09-10T10:34:05.646 INFO:teuthology.orchestra.run.smithi103.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 3.0 MB/s | 1.8 MB 00:00 2024-09-10T10:34:05.671 INFO:teuthology.orchestra.run.smithi103.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.2 MB/s | 249 kB 00:00 2024-09-10T10:34:05.963 INFO:teuthology.orchestra.run.smithi103.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.2 MB/s | 2.4 MB 00:00 2024-09-10T10:34:06.067 INFO:teuthology.orchestra.run.smithi007.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 5.1 MB/s | 5.8 MB 00:01 2024-09-10T10:34:06.121 INFO:teuthology.orchestra.run.smithi103.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 2.8 MB/s | 1.4 MB 00:00 2024-09-10T10:34:06.184 INFO:teuthology.orchestra.run.smithi007.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 129 kB/s | 87 kB 00:00 2024-09-10T10:34:06.367 INFO:teuthology.orchestra.run.smithi007.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-10T10:34:06.843 INFO:teuthology.orchestra.run.smithi007.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 93 kB/s | 61 kB 00:00 2024-09-10T10:34:07.151 INFO:teuthology.orchestra.run.smithi007.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 313 kB/s | 245 kB 00:00 2024-09-10T10:34:07.226 INFO:teuthology.orchestra.run.smithi007.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-09-10T10:34:07.618 INFO:teuthology.orchestra.run.smithi007.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 320 kB/s | 248 kB 00:00 2024-09-10T10:34:07.697 INFO:teuthology.orchestra.run.smithi103.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.0 MB/s | 16 MB 00:05 2024-09-10T10:34:07.702 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:34:07.703 INFO:teuthology.orchestra.run.smithi103.stdout:Total 16 MB/s | 226 MB 00:14 2024-09-10T10:34:07.735 INFO:teuthology.orchestra.run.smithi007.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 44 kB/s | 26 kB 00:00 2024-09-10T10:34:07.769 INFO:teuthology.orchestra.run.smithi007.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-09-10T10:34:08.294 INFO:teuthology.orchestra.run.smithi007.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 62 kB/s | 35 kB 00:00 2024-09-10T10:34:08.344 INFO:teuthology.orchestra.run.smithi007.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 237 kB/s | 172 kB 00:00 2024-09-10T10:34:08.895 INFO:teuthology.orchestra.run.smithi007.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-10T10:34:08.911 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-10T10:34:09.045 INFO:teuthology.orchestra.run.smithi007.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 334 kB/s | 250 kB 00:00 2024-09-10T10:34:09.063 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-10T10:34:09.063 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-10T10:34:09.204 INFO:teuthology.orchestra.run.smithi007.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 259 kB/s | 222 kB 00:00 2024-09-10T10:34:09.353 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-09-10T10:34:09.440 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T10:34:09.478 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T10:34:09.532 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T10:34:09.545 INFO:teuthology.orchestra.run.smithi007.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 119 kB/s | 77 kB 00:00 2024-09-10T10:34:09.565 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T10:34:09.591 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T10:34:09.633 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T10:34:09.646 INFO:teuthology.orchestra.run.smithi007.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 70 kB/s | 42 kB 00:00 2024-09-10T10:34:09.964 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T10:34:09.979 INFO:teuthology.orchestra.run.smithi007.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 348 kB/s | 269 kB 00:00 2024-09-10T10:34:10.067 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:34:10.189 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:34:10.218 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:34:10.309 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:34:10.339 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:34:10.397 INFO:teuthology.orchestra.run.smithi007.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 187 kB/s | 159 kB 00:00 2024-09-10T10:34:10.487 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T10:34:10.497 INFO:teuthology.orchestra.run.smithi007.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 328 kB/s | 279 kB 00:00 2024-09-10T10:34:10.544 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T10:34:10.572 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:34:10.668 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:34:10.698 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:34:10.755 INFO:teuthology.orchestra.run.smithi007.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 66 kB/s | 51 kB 00:00 2024-09-10T10:34:10.761 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:34:10.782 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:34:10.805 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:34:10.894 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-10T10:34:10.895 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-10T10:34:10.997 INFO:teuthology.orchestra.run.smithi007.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 89 kB/s | 54 kB 00:00 2024-09-10T10:34:11.000 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:34:11.126 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:34:11.239 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T10:34:11.322 INFO:teuthology.orchestra.run.smithi007.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 73 kB/s | 42 kB 00:00 2024-09-10T10:34:11.336 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:34:11.423 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:34:11.572 INFO:teuthology.orchestra.run.smithi007.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 77 kB/s | 44 kB 00:00 2024-09-10T10:34:11.666 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:34:11.783 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T10:34:11.875 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T10:34:11.905 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T10:34:11.956 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T10:34:11.989 INFO:teuthology.orchestra.run.smithi007.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-10T10:34:12.044 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T10:34:12.089 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T10:34:12.121 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T10:34:12.165 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T10:34:12.196 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T10:34:12.225 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T10:34:12.231 INFO:teuthology.orchestra.run.smithi007.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 243 kB/s | 221 kB 00:00 2024-09-10T10:34:12.265 INFO:teuthology.orchestra.run.smithi007.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 52 kB/s | 36 kB 00:00 2024-09-10T10:34:12.294 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T10:34:12.347 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T10:34:12.377 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T10:34:12.413 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T10:34:12.490 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T10:34:12.523 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T10:34:12.566 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T10:34:12.587 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T10:34:12.607 INFO:teuthology.orchestra.run.smithi007.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 77 kB/s | 47 kB 00:00 2024-09-10T10:34:12.656 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T10:34:12.702 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T10:34:12.775 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T10:34:12.882 INFO:teuthology.orchestra.run.smithi007.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 94 kB/s | 58 kB 00:00 2024-09-10T10:34:12.907 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T10:34:12.962 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T10:34:13.024 INFO:teuthology.orchestra.run.smithi007.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 388 kB/s | 304 kB 00:00 2024-09-10T10:34:13.341 INFO:teuthology.orchestra.run.smithi007.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 131 kB/s | 96 kB 00:00 2024-09-10T10:34:13.477 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T10:34:13.516 INFO:teuthology.orchestra.run.smithi007.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 100 kB/s | 64 kB 00:00 2024-09-10T10:34:13.557 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T10:34:13.587 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T10:34:13.621 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T10:34:13.625 INFO:teuthology.orchestra.run.smithi007.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 75 kB/s | 45 kB 00:00 2024-09-10T10:34:13.659 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T10:34:13.675 INFO:teuthology.orchestra.run.smithi007.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T10:34:13.701 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T10:34:13.709 INFO:teuthology.orchestra.run.smithi007.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T10:34:13.734 INFO:teuthology.orchestra.run.smithi007.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-09-10T10:34:13.851 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T10:34:13.851 INFO:teuthology.orchestra.run.smithi007.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-10T10:34:13.868 INFO:teuthology.orchestra.run.smithi007.stdout:(108/156): lua-devel-5.4.4-4.el9.x86_64.rpm 42 kB/s | 22 kB 00:00 2024-09-10T10:34:13.895 INFO:teuthology.orchestra.run.smithi007.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 635 kB/s | 26 kB 00:00 2024-09-10T10:34:13.907 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T10:34:13.927 INFO:teuthology.orchestra.run.smithi007.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T10:34:13.953 INFO:teuthology.orchestra.run.smithi007.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 579 kB/s | 49 kB 00:00 2024-09-10T10:34:13.962 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T10:34:13.986 INFO:teuthology.orchestra.run.smithi007.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-10T10:34:14.028 INFO:teuthology.orchestra.run.smithi007.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-10T10:34:14.034 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T10:34:14.053 INFO:teuthology.orchestra.run.smithi007.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-09-10T10:34:14.067 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T10:34:14.070 INFO:teuthology.orchestra.run.smithi007.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-09-10T10:34:14.098 INFO:teuthology.orchestra.run.smithi007.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 770 kB/s | 32 kB 00:00 2024-09-10T10:34:14.103 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T10:34:14.113 INFO:teuthology.orchestra.run.smithi007.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 336 kB/s | 14 kB 00:00 2024-09-10T10:34:14.138 INFO:teuthology.orchestra.run.smithi007.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-10T10:34:14.149 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T10:34:14.171 INFO:teuthology.orchestra.run.smithi007.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T10:34:14.185 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T10:34:14.196 INFO:teuthology.orchestra.run.smithi007.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-10T10:34:14.222 INFO:teuthology.orchestra.run.smithi007.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T10:34:14.228 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T10:34:14.247 INFO:teuthology.orchestra.run.smithi007.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T10:34:14.264 INFO:teuthology.orchestra.run.smithi007.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 650 kB/s | 11 kB 00:00 2024-09-10T10:34:14.270 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T10:34:14.281 INFO:teuthology.orchestra.run.smithi007.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-10T10:34:14.301 INFO:teuthology.orchestra.run.smithi007.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-10T10:34:14.315 INFO:teuthology.orchestra.run.smithi007.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-10T10:34:14.332 INFO:teuthology.orchestra.run.smithi007.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.1 MB/s | 20 kB 00:00 2024-09-10T10:34:14.390 INFO:teuthology.orchestra.run.smithi007.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-10T10:34:14.405 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T10:34:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-10T10:34:14.433 INFO:teuthology.orchestra.run.smithi007.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T10:34:14.447 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T10:34:14.475 INFO:teuthology.orchestra.run.smithi007.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.3 MB/s | 2.0 MB 00:00 2024-09-10T10:34:14.488 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T10:34:14.500 INFO:teuthology.orchestra.run.smithi007.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 864 kB/s | 58 kB 00:00 2024-09-10T10:34:14.525 INFO:teuthology.orchestra.run.smithi007.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 655 kB/s | 16 kB 00:00 2024-09-10T10:34:14.550 INFO:teuthology.orchestra.run.smithi007.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-10T10:34:14.576 INFO:teuthology.orchestra.run.smithi007.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-10T10:34:14.591 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T10:34:14.601 INFO:teuthology.orchestra.run.smithi007.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-10T10:34:14.626 INFO:teuthology.orchestra.run.smithi007.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-10T10:34:14.652 INFO:teuthology.orchestra.run.smithi007.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T10:34:14.677 INFO:teuthology.orchestra.run.smithi007.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T10:34:14.711 INFO:teuthology.orchestra.run.smithi007.stdout:(140/156): protobuf-compiler-3.14.0-13.el9.x86_ 723 kB/s | 863 kB 00:01 2024-09-10T10:34:14.736 INFO:teuthology.orchestra.run.smithi007.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 425 kB/s | 36 kB 00:00 2024-09-10T10:34:14.770 INFO:teuthology.orchestra.run.smithi007.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 806 kB/s | 74 kB 00:00 2024-09-10T10:34:14.795 INFO:teuthology.orchestra.run.smithi007.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T10:34:14.820 INFO:teuthology.orchestra.run.smithi007.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-09-10T10:34:14.839 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T10:34:14.846 INFO:teuthology.orchestra.run.smithi007.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-10T10:34:14.871 INFO:teuthology.orchestra.run.smithi007.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-09-10T10:34:14.878 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T10:34:14.896 INFO:teuthology.orchestra.run.smithi007.stdout:(147/156): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-10T10:34:14.904 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T10:34:14.929 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T10:34:14.947 INFO:teuthology.orchestra.run.smithi007.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T10:34:15.005 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T10:34:15.120 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T10:34:15.122 INFO:teuthology.orchestra.run.smithi007.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-10T10:34:15.189 INFO:teuthology.orchestra.run.smithi007.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-10T10:34:15.231 INFO:teuthology.orchestra.run.smithi007.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 983 kB/s | 41 kB 00:00 2024-09-10T10:34:15.273 INFO:teuthology.orchestra.run.smithi007.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.9 MB/s | 249 kB 00:00 2024-09-10T10:34:15.301 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T10:34:15.331 INFO:teuthology.orchestra.run.smithi007.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 23 MB/s | 1.4 MB 00:00 2024-09-10T10:34:15.339 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T10:34:15.373 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T10:34:15.421 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T10:34:15.452 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T10:34:15.483 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T10:34:15.516 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T10:34:15.550 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T10:34:15.591 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T10:34:15.616 INFO:teuthology.orchestra.run.smithi007.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.4 MB/s | 2.4 MB 00:00 2024-09-10T10:34:15.625 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T10:34:15.665 INFO:teuthology.orchestra.run.smithi007.stdout:(155/156): librados2-19.1.1-305.g1cf819b7.el9.x 4.3 MB/s | 3.4 MB 00:00 2024-09-10T10:34:15.712 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T10:34:15.756 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T10:34:15.795 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T10:34:15.897 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T10:34:15.946 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T10:34:16.657 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T10:34:16.710 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T10:34:16.743 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T10:34:16.841 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T10:34:16.898 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T10:34:16.943 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T10:34:17.030 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T10:34:17.063 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T10:34:17.254 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T10:34:17.269 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-10T10:34:17.277 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T10:34:17.354 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T10:34:17.384 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T10:34:17.502 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T10:34:17.593 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T10:34:17.627 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T10:34:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T10:34:17.695 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T10:34:17.992 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T10:34:18.026 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T10:34:18.109 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T10:34:18.145 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:34:18.250 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:34:18.280 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:34:18.354 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:34:18.384 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:34:18.533 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T10:34:18.589 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T10:34:18.617 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:34:18.721 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:34:18.743 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:34:18.806 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:34:18.828 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:34:18.851 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:34:19.046 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:34:19.181 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:34:19.315 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T10:34:19.319 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T10:34:19.391 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T10:34:19.415 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:34:19.417 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T10:34:19.449 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T10:34:19.496 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:34:19.727 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:34:19.757 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T10:34:19.796 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:34:19.833 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T10:34:19.886 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:34:19.917 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T10:34:19.929 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T10:34:19.953 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T10:34:19.968 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T10:34:20.002 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T10:34:20.067 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T10:34:20.112 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T10:34:20.144 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T10:34:20.188 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T10:34:20.220 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T10:34:20.248 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T10:34:20.325 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T10:34:20.387 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T10:34:20.420 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T10:34:20.431 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T10:34:20.457 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:34:20.462 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T10:34:20.511 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:34:20.538 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T10:34:20.541 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T10:34:20.572 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T10:34:20.616 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T10:34:20.645 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T10:34:20.714 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T10:34:20.768 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T10:34:20.842 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T10:34:20.979 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T10:34:21.040 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T10:34:21.562 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T10:34:21.636 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T10:34:21.666 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T10:34:21.717 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T10:34:21.763 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T10:34:21.805 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T10:34:21.973 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T10:34:22.025 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T10:34:22.080 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T10:34:22.154 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T10:34:22.185 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T10:34:22.221 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T10:34:22.268 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T10:34:22.312 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T10:34:22.356 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T10:34:22.398 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T10:34:22.534 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T10:34:22.734 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T10:34:22.775 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T10:34:22.879 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T10:34:22.941 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T10:34:22.980 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T10:34:23.006 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T10:34:23.031 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T10:34:23.055 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:34:23.088 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:34:23.105 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T10:34:23.232 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T10:34:23.297 INFO:teuthology.orchestra.run.smithi007.stdout:(156/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 385 kB/s | 3.2 MB 00:08 2024-09-10T10:34:23.305 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:34:23.305 INFO:teuthology.orchestra.run.smithi007.stdout:Total 6.8 MB/s | 226 MB 00:33 2024-09-10T10:34:23.305 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T10:34:23.333 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T10:34:23.368 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T10:34:23.407 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T10:34:23.439 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T10:34:23.468 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T10:34:23.511 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T10:34:23.545 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T10:34:23.586 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T10:34:23.621 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T10:34:23.631 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:34:23.666 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:34:23.708 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T10:34:23.751 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T10:34:23.757 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:34:23.757 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T10:34:23.757 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T10:34:23.757 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:23.783 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T10:34:23.784 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:34:23.893 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T10:34:23.951 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T10:34:24.591 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-10T10:34:24.673 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T10:34:24.725 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T10:34:24.757 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T10:34:24.763 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-10T10:34:24.763 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-10T10:34:24.855 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T10:34:24.921 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T10:34:24.966 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T10:34:25.052 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T10:34:25.085 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T10:34:25.278 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T10:34:25.308 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T10:34:25.385 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T10:34:26.030 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T10:34:26.151 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T10:34:27.120 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-10T10:34:27.121 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-10T10:34:27.363 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T10:34:27.456 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T10:34:27.484 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T10:34:27.517 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T10:34:27.831 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T10:34:27.880 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:34:27.970 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:34:28.001 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T10:34:28.046 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T10:34:28.526 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T10:34:28.567 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:34:28.637 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:34:28.660 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T10:34:30.978 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:34:31.015 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:34:31.562 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:34:31.602 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:34:31.700 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:34:31.700 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T10:34:31.700 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T10:34:31.700 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:31.737 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:34:33.649 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-10T10:34:33.760 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T10:34:33.805 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T10:34:33.870 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T10:34:33.911 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T10:34:33.936 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T10:34:33.998 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T10:34:34.345 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T10:34:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:34:34.587 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:34:34.622 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:34:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:34:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:34:34.948 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T10:34:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T10:34:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:34:35.133 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:34:35.169 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:34:35.253 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:34:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:34:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:34:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:34:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:34:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T10:34:35.875 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:34:35.875 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-09-10T10:34:35.875 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-09-10T10:34:35.875 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-09-10T10:34:35.875 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-09-10T10:34:35.875 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-09-10T10:34:35.875 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-09-10T10:34:35.875 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-09-10T10:34:35.875 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:35.925 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:34:35.998 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:34:36.046 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:34:36.062 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:34:36.062 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:36.062 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T10:34:36.062 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:34:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T10:34:36.429 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T10:34:36.471 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T10:34:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T10:34:36.589 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T10:34:36.641 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T10:34:36.681 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T10:34:36.724 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T10:34:36.757 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T10:34:36.784 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T10:34:36.865 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T10:34:36.925 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T10:34:36.954 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T10:34:36.998 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T10:34:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T10:34:37.109 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T10:34:37.163 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T10:34:37.189 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T10:34:37.263 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T10:34:37.313 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T10:34:37.399 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T10:34:37.544 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T10:34:37.549 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:34:37.575 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:34:37.576 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:37.576 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T10:34:37.576 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:34:37.576 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:34:37.576 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:37.599 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T10:34:37.808 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:34:37.831 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:34:37.832 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:37.832 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T10:34:37.832 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:34:37.832 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:34:37.832 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:38.158 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T10:34:38.246 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T10:34:38.267 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:34:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T10:34:38.290 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:34:38.291 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:38.291 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T10:34:38.291 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:34:38.291 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:34:38.291 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:38.325 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T10:34:38.363 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T10:34:38.397 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T10:34:38.405 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T10:34:38.453 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T10:34:38.509 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T10:34:38.563 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T10:34:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T10:34:38.689 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T10:34:38.694 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T10:34:38.742 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T10:34:38.763 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T10:34:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T10:34:38.819 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T10:34:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T10:34:38.884 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T10:34:38.908 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T10:34:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T10:34:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T10:34:39.034 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T10:34:39.046 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T10:34:39.093 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T10:34:39.149 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T10:34:39.177 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T10:34:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T10:34:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T10:34:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T10:34:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T10:34:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T10:34:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T10:34:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T10:34:39.513 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T10:34:39.609 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T10:34:39.639 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:34:39.667 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:34:39.699 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T10:34:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T10:34:39.761 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T10:34:39.779 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T10:34:39.790 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:34:39.815 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T10:34:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T10:34:39.898 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T10:34:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T10:34:40.106 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T10:34:40.229 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T10:34:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T10:34:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T10:34:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T10:34:40.570 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T10:34:40.626 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T10:34:40.665 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T10:34:40.774 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T10:34:40.826 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T10:34:40.837 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:34:40.867 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:34:41.595 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T10:34:41.642 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T10:34:41.676 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T10:34:41.779 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T10:34:41.831 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:34:41.832 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T10:34:41.867 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:34:41.886 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T10:34:41.969 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:34:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T10:34:42.020 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T10:34:42.138 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T10:34:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:34:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:34:42.193 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:42.193 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T10:34:42.193 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:34:42.193 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:34:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:42.212 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T10:34:42.236 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:34:42.243 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T10:34:42.248 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:34:42.323 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T10:34:42.345 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T10:34:42.393 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T10:34:43.017 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T10:34:43.131 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T10:34:43.316 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T10:34:43.347 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:34:43.371 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:34:43.371 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:43.371 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T10:34:43.371 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:34:43.371 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:34:43.371 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:43.414 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:34:43.437 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:34:43.437 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:43.438 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T10:34:43.438 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:43.455 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:34:43.455 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-09-10T10:34:43.456 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-09-10T10:34:43.456 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-09-10T10:34:43.456 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-09-10T10:34:43.456 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-09-10T10:34:43.456 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-09-10T10:34:43.456 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-09-10T10:34:43.456 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:43.482 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:34:43.505 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:34:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T10:34:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:34:43.505 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:34:43.505 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:43.636 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:34:43.652 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:34:43.652 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:43.652 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T10:34:43.652 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:43.807 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:34:43.831 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:34:43.831 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:43.831 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T10:34:43.831 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:34:43.831 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:34:43.832 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T10:34:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T10:34:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T10:34:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T10:34:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T10:34:44.993 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:34:45.094 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:34:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T10:34:45.147 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:34:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T10:34:45.174 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:34:45.174 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:45.174 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T10:34:45.174 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:34:45.175 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:34:45.175 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:45.407 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:34:45.430 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:34:45.430 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:45.430 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T10:34:45.430 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:34:45.430 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:34:45.430 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:45.666 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T10:34:45.696 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:34:45.769 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:34:45.798 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T10:34:45.866 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:34:45.889 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:34:45.889 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:45.889 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T10:34:45.889 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:34:45.890 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:34:45.890 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:45.996 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T10:34:46.053 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T10:34:46.108 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T10:34:46.292 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T10:34:46.342 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T10:34:46.412 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T10:34:46.500 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T10:34:46.640 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T10:34:46.710 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T10:34:46.741 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T10:34:46.777 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T10:34:47.240 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:34:47.268 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:34:47.300 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T10:34:47.362 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T10:34:47.391 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:34:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:34:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:34:48.387 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T10:34:48.428 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T10:34:48.437 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:34:48.468 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T10:34:48.476 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:34:48.500 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T10:34:48.527 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T10:34:48.706 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T10:34:48.757 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T10:34:48.788 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T10:34:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:34:48.817 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T10:34:48.855 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T10:34:48.856 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:34:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:34:48.949 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T10:34:48.949 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T10:34:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:34:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:34:49.369 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:34:49.411 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:34:49.432 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:34:49.442 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:34:49.477 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:34:49.578 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:34:49.740 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T10:34:49.771 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:34:49.795 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:34:49.795 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:49.795 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T10:34:49.795 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:34:49.795 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:34:49.796 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:49.847 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:34:49.859 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:34:49.956 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T10:34:50.004 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T10:34:50.260 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T10:34:50.358 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T10:34:50.398 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T10:34:50.479 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T10:34:50.521 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T10:34:50.544 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T10:34:50.579 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T10:34:50.579 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:34:50.662 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:34:50.663 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:34:50.746 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:34:50.771 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T10:34:50.777 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:34:50.777 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:34:50.933 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T10:34:50.956 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:34:50.956 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:34:50.967 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:34:50.990 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:34:50.990 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:50.990 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T10:34:50.990 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:34:50.990 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:34:50.991 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:51.023 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:34:51.042 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:34:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:34:51.065 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:51.065 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T10:34:51.065 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:51.086 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:34:51.118 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:34:51.141 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:34:51.141 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:51.141 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T10:34:51.141 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:34:51.141 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:34:51.142 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:51.444 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:34:51.466 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:34:51.466 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:34:51.466 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T10:34:51.466 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:34:51.467 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:34:51.467 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:34:56.032 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T10:34:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T10:34:56.114 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T10:34:56.147 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T10:34:56.174 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T10:34:56.353 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T10:34:56.405 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T10:34:56.434 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T10:34:56.464 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T10:34:56.503 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T10:34:56.546 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:34:56.546 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T10:34:56.546 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T10:34:56.546 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T10:34:56.546 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T10:34:56.546 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T10:34:56.546 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T10:34:56.547 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T10:34:56.547 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:34:56.547 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:34:56.547 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:34:56.547 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T10:34:56.547 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T10:34:56.547 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:34:56.547 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T10:34:56.560 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T10:34:56.560 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T10:34:56.561 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T10:34:56.562 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T10:34:56.563 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T10:34:56.564 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T10:34:56.565 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T10:34:56.566 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T10:34:56.567 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T10:34:56.568 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T10:34:56.569 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T10:34:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T10:34:56.571 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T10:34:56.572 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T10:34:56.572 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T10:34:56.572 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T10:34:56.572 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T10:34:56.572 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T10:34:56.572 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T10:34:57.154 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:34:57.201 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:34:57.221 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:34:58.054 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T10:34:58.148 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T10:34:58.189 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T10:34:58.271 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T10:34:58.302 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T10:34:58.327 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T10:34:58.361 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T10:34:58.361 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:34:58.455 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:34:58.455 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:34:58.538 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:34:58.563 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T10:34:58.568 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:34:58.568 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:34:58.740 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:34:58.740 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:34:58.816 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:34:58.888 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:34:59.280 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T10:34:59.281 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T10:34:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T10:34:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T10:34:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T10:34:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T10:34:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T10:34:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T10:34:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:34:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T10:34:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T10:34:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T10:34:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T10:34:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T10:34:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T10:34:59.294 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T10:34:59.294 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:34:59.294 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-09-10T10:34:59.686 DEBUG:teuthology.parallel:result is None 2024-09-10T10:35:01.557 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:35:01.557 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-09-10T10:35:01.557 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-09-10T10:35:01.557 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-09-10T10:35:01.557 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-09-10T10:35:01.557 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-09-10T10:35:01.557 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-09-10T10:35:01.557 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-09-10T10:35:01.557 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:01.738 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:35:01.756 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:35:01.756 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:35:01.756 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T10:35:01.756 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:03.291 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:35:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:35:03.320 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:35:03.320 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T10:35:03.320 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:35:03.320 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:35:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:03.558 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:35:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:35:03.585 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:35:03.585 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T10:35:03.585 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:35:03.585 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:35:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:35:04.076 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:35:04.076 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:35:04.076 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T10:35:04.077 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:35:04.077 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:35:04.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:04.201 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T10:35:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T10:35:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T10:35:04.528 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T10:35:04.578 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T10:35:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:35:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T10:35:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T10:35:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T10:35:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T10:35:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T10:35:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T10:35:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T10:35:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:35:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:35:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:35:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T10:35:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T10:35:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:35:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T10:35:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T10:35:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T10:35:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T10:35:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T10:35:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T10:35:04.658 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T10:35:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T10:35:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T10:35:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T10:35:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T10:35:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T10:35:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T10:35:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T10:35:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T10:35:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T10:35:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T10:35:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T10:35:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T10:35:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T10:35:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T10:35:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T10:35:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T10:35:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T10:35:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T10:35:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T10:35:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T10:35:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T10:35:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T10:35:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T10:35:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T10:35:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T10:35:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T10:35:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T10:35:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T10:35:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T10:35:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T10:35:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T10:35:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T10:35:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T10:35:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T10:35:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T10:35:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T10:35:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T10:35:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T10:35:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T10:35:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T10:35:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T10:35:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T10:35:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T10:35:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T10:35:04.961 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T10:35:05.001 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T10:35:05.029 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T10:35:05.490 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:35:05.519 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:35:05.551 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T10:35:05.617 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T10:35:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:35:06.747 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:35:06.777 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:35:07.330 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:35:07.330 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:35:07.330 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-09-10T10:35:07.330 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T10:35:07.330 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T10:35:07.330 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T10:35:07.330 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T10:35:07.330 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.330 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.330 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:07.331 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.332 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T10:35:07.333 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T10:35:07.334 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T10:35:07.335 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T10:35:07.336 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T10:35:07.337 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T10:35:07.337 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.337 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.337 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.337 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T10:35:07.337 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T10:35:07.337 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T10:35:07.337 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T10:35:07.337 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T10:35:07.337 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T10:35:07.338 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T10:35:07.339 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T10:35:07.340 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T10:35:07.340 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T10:35:07.340 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T10:35:07.340 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T10:35:07.340 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T10:35:07.340 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T10:35:07.340 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T10:35:07.340 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T10:35:07.340 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T10:35:07.341 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T10:35:07.342 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T10:35:07.343 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T10:35:07.343 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T10:35:07.343 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T10:35:07.343 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:35:07.343 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-10T10:35:07.751 DEBUG:teuthology.parallel:result is None 2024-09-10T10:35:07.792 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:35:07.819 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:35:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:35:08.105 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T10:35:08.138 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:35:08.164 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:35:08.165 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:35:08.165 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T10:35:08.165 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:35:08.165 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:35:08.165 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:35:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:35:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T10:35:08.384 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T10:35:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T10:35:09.400 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:35:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:35:09.425 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:35:09.425 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T10:35:09.425 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:35:09.425 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:35:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:09.467 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:35:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:35:09.492 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:35:09.492 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T10:35:09.492 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:09.543 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:35:09.568 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:35:09.568 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:35:09.568 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T10:35:09.568 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:35:09.568 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:35:09.568 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:09.885 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:35:09.911 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:35:09.911 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:35:09.911 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T10:35:09.911 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:35:09.911 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:35:09.912 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T10:35:14.823 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T10:35:14.865 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T10:35:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T10:35:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T10:35:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T10:35:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T10:35:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T10:35:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T10:35:15.284 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T10:35:15.910 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:35:15.949 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:35:15.970 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:35:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T10:35:16.996 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T10:35:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T10:35:17.135 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T10:35:17.166 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T10:35:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T10:35:17.217 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T10:35:17.217 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:35:17.310 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:35:17.310 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:35:17.410 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:35:17.435 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T10:35:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:35:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:35:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:35:17.637 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:35:17.722 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:35:17.790 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:35:23.476 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:35:23.476 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T10:35:23.476 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T10:35:23.476 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T10:35:23.477 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T10:35:23.478 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T10:35:23.479 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T10:35:23.480 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T10:35:23.481 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T10:35:23.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T10:35:23.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T10:35:23.484 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T10:35:23.485 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T10:35:23.486 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T10:35:23.487 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T10:35:23.488 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T10:35:23.489 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T10:35:26.292 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:35:26.292 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:26.292 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-09-10T10:35:26.292 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T10:35:26.292 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T10:35:26.292 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T10:35:26.292 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T10:35:26.292 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.292 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.292 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.293 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.294 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:35:26.295 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T10:35:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T10:35:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T10:35:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T10:35:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T10:35:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T10:35:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T10:35:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T10:35:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T10:35:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T10:35:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T10:35:26.297 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.297 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.297 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.297 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T10:35:26.297 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T10:35:26.297 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T10:35:26.297 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T10:35:26.297 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T10:35:26.297 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.297 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.298 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T10:35:26.298 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.298 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T10:35:26.298 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T10:35:26.298 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T10:35:26.298 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T10:35:26.298 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T10:35:26.298 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T10:35:26.298 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T10:35:26.298 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T10:35:26.299 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T10:35:26.299 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T10:35:26.299 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T10:35:26.299 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T10:35:26.299 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T10:35:26.299 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T10:35:26.299 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T10:35:26.299 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T10:35:26.299 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T10:35:26.299 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T10:35:26.300 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T10:35:26.300 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T10:35:26.300 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T10:35:26.300 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T10:35:26.300 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T10:35:26.301 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.301 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.301 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.301 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T10:35:26.301 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T10:35:26.301 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T10:35:26.301 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T10:35:26.301 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T10:35:26.301 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T10:35:26.301 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T10:35:26.302 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T10:35:26.302 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T10:35:26.302 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T10:35:26.302 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T10:35:26.302 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T10:35:26.302 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T10:35:26.302 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T10:35:26.302 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T10:35:26.302 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T10:35:26.302 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T10:35:26.303 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T10:35:26.303 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T10:35:26.303 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T10:35:26.303 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T10:35:26.303 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T10:35:26.303 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T10:35:26.304 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T10:35:26.305 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T10:35:26.306 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T10:35:26.307 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T10:35:26.308 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T10:35:26.308 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T10:35:26.308 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T10:35:26.308 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T10:35:26.308 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:35:26.308 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-10T10:35:26.883 DEBUG:teuthology.parallel:result is None 2024-09-10T10:35:26.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:35:27.049 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T10:35:27.079 INFO:teuthology.orchestra.run.smithi007.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T10:35:27.080 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T10:35:27.080 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T10:35:27.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:35:27.238 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T10:35:27.267 INFO:teuthology.orchestra.run.smithi103.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T10:35:27.267 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T10:35:27.267 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T10:35:27.270 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:35:27.411 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T10:35:27.439 INFO:teuthology.orchestra.run.smithi133.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T10:35:27.439 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T10:35:27.439 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T10:35:27.441 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T10:35:27.441 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:35:27.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T10:35:27.478 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:35:27.479 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T10:35:27.512 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:35:27.512 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T10:35:27.544 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T10:35:27.545 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:35:27.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T10:35:27.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T10:35:27.647 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:35:27.647 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T10:35:27.676 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T10:35:27.746 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:35:27.746 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T10:35:27.777 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T10:35:27.847 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T10:35:27.848 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:35:27.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T10:35:27.878 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T10:35:27.948 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:35:27.948 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T10:35:27.978 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T10:35:28.046 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:35:28.046 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T10:35:28.071 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T10:35:28.136 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T10:35:28.136 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:35:28.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T10:35:28.166 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T10:35:28.238 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:35:28.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T10:35:28.263 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T10:35:28.329 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:35:28.329 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T10:35:28.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T10:35:28.419 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T10:35:28.529 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'subvols': {'create': 2}} 2024-09-10T10:35:28.530 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:35:28.530 INFO:tasks.cephadm:Cluster fsid is 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:35:28.530 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T10:35:28.531 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.7', 'mon.b': '172.21.15.103', 'mon.c': '172.21.15.133'} 2024-09-10T10:35:28.531 INFO:tasks.cephadm:First mon is mon.a on smithi007 2024-09-10T10:35:28.531 INFO:tasks.cephadm:First mgr is x 2024-09-10T10:35:28.531 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T10:35:28.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname $(hostname -s) 2024-09-10T10:35:28.563 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-09-10T10:35:28.588 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2024-09-10T10:35:28.613 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T10:35:28.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:35:28.760 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82914/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:14:38.445020', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82912/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:05:03.996701', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-10T10:35:28.904 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240909.165402-squid, sha1 1cf819b7212a6048e92fc78c4eff963344f6d56b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:35:28.906 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T10:35:28.906 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T10:35:28.906 DEBUG:teuthology.orchestra.run.smithi007:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T10:35:29.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 10:35 /home/ubuntu/cephtest/cephadm 2024-09-10T10:35:29.303 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T10:35:29.678 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 10:35 /home/ubuntu/cephtest/cephadm 2024-09-10T10:35:29.679 DEBUG:teuthology.orchestra.run.smithi133:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T10:35:30.068 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 10:35 /home/ubuntu/cephtest/cephadm 2024-09-10T10:35:30.069 DEBUG:teuthology.orchestra.run.smithi007:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T10:35:30.089 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T10:35:30.106 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T10:35:30.141 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b on all hosts... 2024-09-10T10:35:30.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T10:35:30.144 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T10:35:30.149 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T10:35:31.623 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T10:35:31.658 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T10:35:31.896 INFO:teuthology.orchestra.run.smithi007.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T10:36:03.878 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T10:36:03.879 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T10:36:03.879 INFO:teuthology.orchestra.run.smithi007.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T10:36:03.879 INFO:teuthology.orchestra.run.smithi007.stdout: "repo_digests": [ 2024-09-10T10:36:03.879 INFO:teuthology.orchestra.run.smithi007.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T10:36:03.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T10:36:03.879 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T10:36:19.893 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-10T10:36:19.893 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T10:36:19.894 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T10:36:19.894 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2024-09-10T10:36:19.894 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T10:36:19.894 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-09-10T10:36:19.894 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-10T10:36:20.158 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T10:36:20.158 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T10:36:20.158 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T10:36:20.158 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-09-10T10:36:20.158 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T10:36:20.158 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-10T10:36:20.159 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T10:36:20.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph 2024-09-10T10:36:20.219 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-09-10T10:36:20.246 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2024-09-10T10:36:20.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /etc/ceph 2024-09-10T10:36:20.304 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-09-10T10:36:20.328 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2024-09-10T10:36:20.353 INFO:tasks.cephadm:Writing seed config... 2024-09-10T10:36:20.354 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-10T10:36:20.354 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-10T10:36:20.354 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T10:36:20.354 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-10T10:36:20.354 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-10T10:36:20.354 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T10:36:20.354 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-10T10:36:20.355 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-10T10:36:20.356 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T10:36:20.357 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-10T10:36:20.358 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T10:36:20.358 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-10T10:36:20.358 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:36:20.359 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T10:36:20.378 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 65af4d96-6f60-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-10T10:36:20.378 DEBUG:teuthology.orchestra.run.smithi007:mon.a> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T10:36:20.420 DEBUG:teuthology.orchestra.run.smithi007:mgr.x> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T10:36:20.464 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T10:36:20.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b -v bootstrap --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.7 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:20.684 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:36:20.684 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b', '-v', 'bootstrap', '--fsid', '65af4d96-6f60-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.7', '--skip-admin-label'] 2024-09-10T10:36:20.684 INFO:teuthology.orchestra.run.smithi007.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T10:36:20.684 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying podman|docker is present... 2024-09-10T10:36:20.709 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 5.2.2 2024-09-10T10:36:20.710 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying lvm2 is present... 2024-09-10T10:36:20.710 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying time synchronization is in place... 2024-09-10T10:36:20.718 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T10:36:20.718 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T10:36:20.724 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T10:36:20.724 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-10T10:36:20.731 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-09-10T10:36:20.737 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-09-10T10:36:20.737 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-09-10T10:36:20.737 INFO:teuthology.orchestra.run.smithi007.stdout:Repeating the final host check... 2024-09-10T10:36:20.760 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 5.2.2 2024-09-10T10:36:20.761 INFO:teuthology.orchestra.run.smithi007.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T10:36:20.761 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl is present 2024-09-10T10:36:20.761 INFO:teuthology.orchestra.run.smithi007.stdout:lvcreate is present 2024-09-10T10:36:20.767 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T10:36:20.767 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T10:36:20.774 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T10:36:20.774 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-10T10:36:20.781 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-09-10T10:36:20.787 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-09-10T10:36:20.787 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-09-10T10:36:20.788 INFO:teuthology.orchestra.run.smithi007.stdout:Host looks OK 2024-09-10T10:36:20.788 INFO:teuthology.orchestra.run.smithi007.stdout:Cluster fsid: 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:20.788 INFO:teuthology.orchestra.run.smithi007.stdout:Acquiring lock 140520614478848 on /run/cephadm/65af4d96-6f60-11ef-bcea-c7b262605968.lock 2024-09-10T10:36:20.788 INFO:teuthology.orchestra.run.smithi007.stdout:Lock 140520614478848 acquired on /run/cephadm/65af4d96-6f60-11ef-bcea-c7b262605968.lock 2024-09-10T10:36:20.788 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 3300 ... 2024-09-10T10:36:20.789 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 6789 ... 2024-09-10T10:36:20.789 INFO:teuthology.orchestra.run.smithi007.stdout:Base mon IP(s) is [172.21.15.7:3300, 172.21.15.7:6789], mon addrv is [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-09-10T10:36:20.791 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.7 metric 100 2024-09-10T10:36:20.792 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.7 metric 100 2024-09-10T10:36:20.794 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T10:36:20.794 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T10:36:20.794 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-09-10T10:36:20.797 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T10:36:20.797 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T10:36:20.797 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T10:36:20.797 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T10:36:20.797 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T10:36:20.797 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T10:36:20.798 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-09-10T10:36:20.798 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-09-10T10:36:20.798 INFO:teuthology.orchestra.run.smithi007.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T10:36:20.799 INFO:teuthology.orchestra.run.smithi007.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T10:36:20.799 INFO:teuthology.orchestra.run.smithi007.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T10:36:21.231 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T10:36:21.231 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T10:36:21.231 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T10:36:21.231 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:3965ca83761b6e221cd35b61f2ebac0e3b1f9d68a1ce39c1ada69153a1d73116 2024-09-10T10:36:21.231 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T10:36:21.232 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying config sha256:41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T10:36:21.232 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T10:36:25.884 INFO:teuthology.orchestra.run.smithi007.stdout:ceph: stdout ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:36:25.884 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph version: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:36:25.884 INFO:teuthology.orchestra.run.smithi007.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T10:36:26.772 INFO:teuthology.orchestra.run.smithi007.stdout:stat: stdout 167 167 2024-09-10T10:36:26.772 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial keys... 2024-09-10T10:36:27.627 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQArIeBmlPjHBhAAGV1bmR6FUddPq9hhRdYptw== 2024-09-10T10:36:28.667 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQArIeBmaaw+ORAA16sVFARxF2BZXIAKU0118w== 2024-09-10T10:36:29.530 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQAsIeBmrjaWOhAAxfes/E9TvHtEiqRTAt9TeQ== 2024-09-10T10:36:29.531 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial monmap... 2024-09-10T10:36:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T10:36:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T10:36:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T10:36:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool for a [v2:172.21.15.7:3300,v1:172.21.15.7:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T10:36:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-09-10T10:36:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: set fsid to 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:30.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T10:36:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:36:30.463 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mon... 2024-09-10T10:36:31.651 INFO:teuthology.orchestra.run.smithi007.stdout:create mon.a on 2024-09-10T10:36:31.867 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T10:36:32.012 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T10:36:32.196 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-65af4d96-6f60-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-65af4d96-6f60-11ef-bcea-c7b262605968.target. 2024-09-10T10:36:32.196 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-65af4d96-6f60-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-65af4d96-6f60-11ef-bcea-c7b262605968.target. 2024-09-10T10:36:32.452 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.a 2024-09-10T10:36:32.453 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T10:36:32.621 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-65af4d96-6f60-11ef-bcea-c7b262605968.target.wants/ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-65af4d96-6f60-11ef-bcea-c7b262605968@.service. 2024-09-10T10:36:33.121 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:33 smithi007 podman[30477]: 2024-09-10 10:36:33.079985204 +0000 UTC m=+0.330467109 container start 983fe6b97b86f4b535e574a41433d9bdd89e900654759a13c9002d8cada3d6d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T10:36:33.121 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:33 smithi007 ceph-mon[30496]: mkfs 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:33.184 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T10:36:33.184 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-09-10T10:36:33.192 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T10:36:33.192 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-10T10:36:33.192 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-09-10T10:36:33.192 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T10:36:33.192 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon to start... 2024-09-10T10:36:33.193 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon... 2024-09-10T10:36:33.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:33 smithi007 ceph-mon[30496]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T10:36:33.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:33 smithi007 bash[30477]: 983fe6b97b86f4b535e574a41433d9bdd89e900654759a13c9002d8cada3d6d4 2024-09-10T10:36:33.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:33 smithi007 systemd[1]: Started Ceph mon.a for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:36:34.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T10:36:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout id: 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T10:36:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout services: 2024-09-10T10:36:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.630109s) 2024-09-10T10:36:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T10:36:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T10:36:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout data: 2024-09-10T10:36:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T10:36:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T10:36:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T10:36:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T10:36:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.672+0000 7fdfa4830640 1 Processor -- start 2024-09-10T10:36:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.672+0000 7fdfa4830640 1 -- start start 2024-09-10T10:36:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.672+0000 7fdfa4830640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa010a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.672+0000 7fdfa4830640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa007be10 con 0x7fdfa0108220 2024-09-10T10:36:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.673+0000 7fdf9e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa010a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.673+0000 7fdf9e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa010a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59858/0 (socket says 172.21.15.7:59858) 2024-09-10T10:36:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.673+0000 7fdf9e59c640 1 -- 172.21.15.7:0/2182336211 learned_addr learned my addr 172.21.15.7:0/2182336211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.673+0000 7fdf9e59c640 1 -- 172.21.15.7:0/2182336211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa010ab50 con 0x7fdfa0108220 2024-09-10T10:36:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.676+0000 7fdf9e59c640 1 --2- 172.21.15.7:0/2182336211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa010a610 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009820 tx=0x7fdf9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c668cdf4fbcce8b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.676+0000 7fdf9d59a640 1 -- 172.21.15.7:0/2182336211 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf90037070 con 0x7fdfa0108220 2024-09-10T10:36:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.676+0000 7fdf9d59a640 1 -- 172.21.15.7:0/2182336211 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf90032440 con 0x7fdfa0108220 2024-09-10T10:36:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.676+0000 7fdf9d59a640 1 -- 172.21.15.7:0/2182336211 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf90032740 con 0x7fdfa0108220 2024-09-10T10:36:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.677+0000 7fdfa4830640 1 -- 172.21.15.7:0/2182336211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 msgr2=0x7fdfa010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.677+0000 7fdfa4830640 1 --2- 172.21.15.7:0/2182336211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa010a610 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009820 tx=0x7fdf9002a270 comp rx=0 tx=0).stop 2024-09-10T10:36:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.677+0000 7fdfa4830640 1 -- 172.21.15.7:0/2182336211 shutdown_connections 2024-09-10T10:36:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.677+0000 7fdfa4830640 1 --2- 172.21.15.7:0/2182336211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa010a610 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.677+0000 7fdfa4830640 1 -- 172.21.15.7:0/2182336211 >> 172.21.15.7:0/2182336211 conn(0x7fdfa0101e20 msgr2=0x7fdfa0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.677+0000 7fdfa4830640 1 -- 172.21.15.7:0/2182336211 shutdown_connections 2024-09-10T10:36:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.677+0000 7fdfa4830640 1 -- 172.21.15.7:0/2182336211 wait complete. 2024-09-10T10:36:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.678+0000 7fdfa4830640 1 Processor -- start 2024-09-10T10:36:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.678+0000 7fdfa4830640 1 -- start start 2024-09-10T10:36:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdfa4830640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa0107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdfa4830640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa010afc0 con 0x7fdfa0108220 2024-09-10T10:36:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdf9e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa0107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdf9e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa0107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59870/0 (socket says 172.21.15.7:59870) 2024-09-10T10:36:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdf9e59c640 1 -- 172.21.15.7:0/3352878605 learned_addr learned my addr 172.21.15.7:0/3352878605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdf9e59c640 1 -- 172.21.15.7:0/3352878605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa0107fe0 con 0x7fdfa0108220 2024-09-10T10:36:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdf9e59c640 1 --2- 172.21.15.7:0/3352878605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa0107aa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf900090b0 tx=0x7fdf900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdf8f7fe640 1 -- 172.21.15.7:0/3352878605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf90041070 con 0x7fdfa0108220 2024-09-10T10:36:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdf8f7fe640 1 -- 172.21.15.7:0/3352878605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9003c7a0 con 0x7fdfa0108220 2024-09-10T10:36:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdf8f7fe640 1 -- 172.21.15.7:0/3352878605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf90037040 con 0x7fdfa0108220 2024-09-10T10:36:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdfa4830640 1 -- 172.21.15.7:0/3352878605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa01061d0 con 0x7fdfa0108220 2024-09-10T10:36:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.679+0000 7fdfa4830640 1 -- 172.21.15.7:0/3352878605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa0106710 con 0x7fdfa0108220 2024-09-10T10:36:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.680+0000 7fdf8f7fe640 1 -- 172.21.15.7:0/3352878605 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf90044070 con 0x7fdfa0108220 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.680+0000 7fdf8f7fe640 1 -- 172.21.15.7:0/3352878605 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf9003d3f0 con 0x7fdfa0108220 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.680+0000 7fdfa4830640 1 -- 172.21.15.7:0/3352878605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa0108680 con 0x7fdfa0108220 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.683+0000 7fdf8f7fe640 1 -- 172.21.15.7:0/3352878605 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdf90046450 con 0x7fdfa0108220 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.745+0000 7fdfa4830640 1 -- 172.21.15.7:0/3352878605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fdfa0194860 con 0x7fdfa0108220 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.746+0000 7fdf8f7fe640 1 -- 172.21.15.7:0/3352878605 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fdf90046630 con 0x7fdfa0108220 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.747+0000 7fdfa4830640 1 -- 172.21.15.7:0/3352878605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 msgr2=0x7fdfa0107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.747+0000 7fdfa4830640 1 --2- 172.21.15.7:0/3352878605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa0107aa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf900090b0 tx=0x7fdf900046c0 comp rx=0 tx=0).stop 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.747+0000 7fdfa4830640 1 -- 172.21.15.7:0/3352878605 shutdown_connections 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.747+0000 7fdfa4830640 1 --2- 172.21.15.7:0/3352878605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfa0108220 0x7fdfa0107aa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.747+0000 7fdfa4830640 1 -- 172.21.15.7:0/3352878605 >> 172.21.15.7:0/3352878605 conn(0x7fdfa0101e20 msgr2=0x7fdfa0102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.747+0000 7fdfa4830640 1 -- 172.21.15.7:0/3352878605 shutdown_connections 2024-09-10T10:36:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:33.747+0000 7fdfa4830640 1 -- 172.21.15.7:0/3352878605 wait complete. 2024-09-10T10:36:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:mon is available 2024-09-10T10:36:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T10:36:34.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T10:36:34.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: monmap epoch 1 2024-09-10T10:36:34.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:34.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: last_changed 2024-09-10T10:36:29.861118+0000 2024-09-10T10:36:34.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: created 2024-09-10T10:36:29.861118+0000 2024-09-10T10:36:34.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: min_mon_release 19 (squid) 2024-09-10T10:36:34.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: election_strategy: 1 2024-09-10T10:36:34.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-10T10:36:34.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: fsmap 2024-09-10T10:36:34.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T10:36:34.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: mgrmap e1: no daemons active 2024-09-10T10:36:34.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:34 smithi007 ceph-mon[30496]: from='client.? 172.21.15.7:0/3352878605' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T10:36:35.333 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:35 smithi007 ceph-mon[30496]: from='client.? 172.21.15.7:0/2038700099' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T10:36:35.333 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:35 smithi007 ceph-mon[30496]: from='client.? 172.21.15.7:0/2038700099' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T10:36:35.336 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:35.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T10:36:35.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:35.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-09-10T10:36:35.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T10:36:35.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T10:36:35.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T10:36:35.342 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.716+0000 7fa776476640 1 Processor -- start 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.716+0000 7fa776476640 1 -- start start 2024-09-10T10:36:35.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.717+0000 7fa776476640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77010a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.717+0000 7fa776476640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa77007be50 con 0x7fa7701081e0 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.717+0000 7fa76ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77010a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.717+0000 7fa76ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77010a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59886/0 (socket says 172.21.15.7:59886) 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.717+0000 7fa76ffff640 1 -- 172.21.15.7:0/1271912606 learned_addr learned my addr 172.21.15.7:0/1271912606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.717+0000 7fa76ffff640 1 -- 172.21.15.7:0/1271912606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77010ab10 con 0x7fa7701081e0 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.718+0000 7fa76ffff640 1 --2- 172.21.15.7:0/1271912606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77010a5d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa760009820 tx=0x7fa76002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74dcdc5fcc9e93de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.718+0000 7fa76effd640 1 -- 172.21.15.7:0/1271912606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa760037070 con 0x7fa7701081e0 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.718+0000 7fa76effd640 1 -- 172.21.15.7:0/1271912606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa760032440 con 0x7fa7701081e0 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.718+0000 7fa76effd640 1 -- 172.21.15.7:0/1271912606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa760032740 con 0x7fa7701081e0 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.719+0000 7fa776476640 1 -- 172.21.15.7:0/1271912606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 msgr2=0x7fa77010a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.719+0000 7fa776476640 1 --2- 172.21.15.7:0/1271912606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77010a5d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa760009820 tx=0x7fa76002a270 comp rx=0 tx=0).stop 2024-09-10T10:36:35.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.719+0000 7fa776476640 1 -- 172.21.15.7:0/1271912606 shutdown_connections 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.719+0000 7fa776476640 1 --2- 172.21.15.7:0/1271912606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77010a5d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.719+0000 7fa776476640 1 -- 172.21.15.7:0/1271912606 >> 172.21.15.7:0/1271912606 conn(0x7fa770101e20 msgr2=0x7fa770104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.719+0000 7fa776476640 1 -- 172.21.15.7:0/1271912606 shutdown_connections 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.719+0000 7fa776476640 1 -- 172.21.15.7:0/1271912606 wait complete. 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.719+0000 7fa776476640 1 Processor -- start 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.719+0000 7fa776476640 1 -- start start 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.720+0000 7fa776476640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.720+0000 7fa776476640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa77010aff0 con 0x7fa7701081e0 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.720+0000 7fa76ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.720+0000 7fa76ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77019af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59894/0 (socket says 172.21.15.7:59894) 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.720+0000 7fa76ffff640 1 -- 172.21.15.7:0/2038700099 learned_addr learned my addr 172.21.15.7:0/2038700099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.720+0000 7fa76ffff640 1 -- 172.21.15.7:0/2038700099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77019b440 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.720+0000 7fa76ffff640 1 --2- 172.21.15.7:0/2038700099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77019af00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa7600090b0 tx=0x7fa7600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.720+0000 7fa76d7fa640 1 -- 172.21.15.7:0/2038700099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa760041070 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.721+0000 7fa76d7fa640 1 -- 172.21.15.7:0/2038700099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76003c7a0 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.721+0000 7fa776476640 1 -- 172.21.15.7:0/2038700099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77019b6d0 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.721+0000 7fa76d7fa640 1 -- 172.21.15.7:0/2038700099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa760037040 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.721+0000 7fa776476640 1 -- 172.21.15.7:0/2038700099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77019e3c0 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.721+0000 7fa76d7fa640 1 -- 172.21.15.7:0/2038700099 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa760044070 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.721+0000 7fa76d7fa640 1 -- 172.21.15.7:0/2038700099 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa76003d3f0 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.722+0000 7fa776476640 1 -- 172.21.15.7:0/2038700099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa770108640 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.724+0000 7fa76d7fa640 1 -- 172.21.15.7:0/2038700099 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa760046450 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.782+0000 7fa776476640 1 -- 172.21.15.7:0/2038700099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa77002d910 con 0x7fa7701081e0 2024-09-10T10:36:35.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.785+0000 7fa76d7fa640 1 -- 172.21.15.7:0/2038700099 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7fa760046630 con 0x7fa7701081e0 2024-09-10T10:36:35.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.786+0000 7fa76d7fa640 1 -- 172.21.15.7:0/2038700099 <== mon.0 v2:172.21.15.7:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa7600584b0 con 0x7fa7701081e0 2024-09-10T10:36:35.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.787+0000 7fa776476640 1 -- 172.21.15.7:0/2038700099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 msgr2=0x7fa77019af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:35.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.787+0000 7fa776476640 1 --2- 172.21.15.7:0/2038700099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77019af00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa7600090b0 tx=0x7fa7600046c0 comp rx=0 tx=0).stop 2024-09-10T10:36:35.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.787+0000 7fa776476640 1 -- 172.21.15.7:0/2038700099 shutdown_connections 2024-09-10T10:36:35.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.787+0000 7fa776476640 1 --2- 172.21.15.7:0/2038700099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701081e0 0x7fa77019af00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:35.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.787+0000 7fa776476640 1 -- 172.21.15.7:0/2038700099 >> 172.21.15.7:0/2038700099 conn(0x7fa770101e20 msgr2=0x7fa770102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:35.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.787+0000 7fa776476640 1 -- 172.21.15.7:0/2038700099 shutdown_connections 2024-09-10T10:36:35.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:34.787+0000 7fa776476640 1 -- 172.21.15.7:0/2038700099 wait complete. 2024-09-10T10:36:35.347 INFO:teuthology.orchestra.run.smithi007.stdout:Generating new minimal ceph.conf... 2024-09-10T10:36:36.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:36 smithi007 ceph-mon[30496]: from='client.? 172.21.15.7:0/3169626232' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:36:36.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.759+0000 7f7122bb9640 1 Processor -- start 2024-09-10T10:36:36.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.759+0000 7f7122bb9640 1 -- start start 2024-09-10T10:36:36.731 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.759+0000 7f7122bb9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.759+0000 7f7122bb9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f711c07c890 con 0x7f711c07de60 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.759+0000 7f7120955640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.759+0000 7f7120955640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59902/0 (socket says 172.21.15.7:59902) 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.759+0000 7f7120955640 1 -- 172.21.15.7:0/3660722445 learned_addr learned my addr 172.21.15.7:0/3660722445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.760+0000 7f7120955640 1 -- 172.21.15.7:0/3660722445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c07ca10 con 0x7f711c07de60 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.760+0000 7f7120955640 1 --2- 172.21.15.7:0/3660722445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c07c2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7110009820 tx=0x7f711002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7aad976e4d1e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.760+0000 7f711b7fe640 1 -- 172.21.15.7:0/3660722445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7110037070 con 0x7f711c07de60 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.760+0000 7f711b7fe640 1 -- 172.21.15.7:0/3660722445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7110032900 con 0x7f711c07de60 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.761+0000 7f711b7fe640 1 -- 172.21.15.7:0/3660722445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7110030850 con 0x7f711c07de60 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.761+0000 7f7122bb9640 1 -- 172.21.15.7:0/3660722445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 msgr2=0x7f711c07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.761+0000 7f7122bb9640 1 --2- 172.21.15.7:0/3660722445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c07c2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7110009820 tx=0x7f711002a270 comp rx=0 tx=0).stop 2024-09-10T10:36:36.732 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.761+0000 7f7122bb9640 1 -- 172.21.15.7:0/3660722445 shutdown_connections 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.761+0000 7f7122bb9640 1 --2- 172.21.15.7:0/3660722445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c07c2c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.761+0000 7f7122bb9640 1 -- 172.21.15.7:0/3660722445 >> 172.21.15.7:0/3660722445 conn(0x7f711c102e80 msgr2=0x7f711c1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.761+0000 7f7122bb9640 1 -- 172.21.15.7:0/3660722445 shutdown_connections 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.761+0000 7f7122bb9640 1 -- 172.21.15.7:0/3660722445 wait complete. 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.762+0000 7f7122bb9640 1 Processor -- start 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.762+0000 7f7122bb9640 1 -- start start 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.762+0000 7f7122bb9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c19d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.762+0000 7f7122bb9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f711c10dd20 con 0x7f711c07de60 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.762+0000 7f7120955640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c19d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.762+0000 7f7120955640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c19d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59904/0 (socket says 172.21.15.7:59904) 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.762+0000 7f7120955640 1 -- 172.21.15.7:0/3169626232 learned_addr learned my addr 172.21.15.7:0/3169626232 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.762+0000 7f7120955640 1 -- 172.21.15.7:0/3169626232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c19dea0 con 0x7f711c07de60 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.762+0000 7f7120955640 1 --2- 172.21.15.7:0/3169626232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c19d960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7110032ce0 tx=0x7f7110032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:36.733 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.763+0000 7f7119ffb640 1 -- 172.21.15.7:0/3169626232 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7110042070 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.763+0000 7f7119ffb640 1 -- 172.21.15.7:0/3169626232 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f71100308c0 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.763+0000 7f7122bb9640 1 -- 172.21.15.7:0/3169626232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c19e130 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.763+0000 7f7119ffb640 1 -- 172.21.15.7:0/3169626232 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7110037040 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.763+0000 7f7122bb9640 1 -- 172.21.15.7:0/3169626232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c1a0e20 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.763+0000 7f7119ffb640 1 -- 172.21.15.7:0/3169626232 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7110045070 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.763+0000 7f7119ffb640 1 -- 172.21.15.7:0/3169626232 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7110047b70 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.764+0000 7f7122bb9640 1 -- 172.21.15.7:0/3169626232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711c07c340 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.766+0000 7f7119ffb640 1 -- 172.21.15.7:0/3169626232 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f711003c720 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.826+0000 7f7122bb9640 1 -- 172.21.15.7:0/3169626232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f711c1a1320 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.827+0000 7f7119ffb640 1 -- 172.21.15.7:0/3169626232 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f711003c900 con 0x7f711c07de60 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.828+0000 7f7122bb9640 1 -- 172.21.15.7:0/3169626232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 msgr2=0x7f711c19d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:36.734 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.828+0000 7f7122bb9640 1 --2- 172.21.15.7:0/3169626232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c19d960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7110032ce0 tx=0x7f7110032d10 comp rx=0 tx=0).stop 2024-09-10T10:36:36.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.828+0000 7f7122bb9640 1 -- 172.21.15.7:0/3169626232 shutdown_connections 2024-09-10T10:36:36.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.828+0000 7f7122bb9640 1 --2- 172.21.15.7:0/3169626232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f711c07de60 0x7f711c19d960 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:36.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.828+0000 7f7122bb9640 1 -- 172.21.15.7:0/3169626232 >> 172.21.15.7:0/3169626232 conn(0x7f711c102e80 msgr2=0x7f711c103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:36.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.828+0000 7f7122bb9640 1 -- 172.21.15.7:0/3169626232 shutdown_connections 2024-09-10T10:36:36.735 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:35.828+0000 7f7122bb9640 1 -- 172.21.15.7:0/3169626232 wait complete. 2024-09-10T10:36:36.735 INFO:teuthology.orchestra.run.smithi007.stdout:Restarting the monitor... 2024-09-10T10:36:37.002 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:36 smithi007 systemd[1]: Stopping Ceph mon.a for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:36:37.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:37 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a[30492]: 2024-09-10T10:36:37.000+0000 7f27418c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:36:37.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:37 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a[30492]: 2024-09-10T10:36:37.000+0000 7f27418c1640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T10:36:37.352 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:37 smithi007 podman[30750]: 2024-09-10 10:36:37.092921356 +0000 UTC m=+0.190421017 container died 983fe6b97b86f4b535e574a41433d9bdd89e900654759a13c9002d8cada3d6d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T10:36:37.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:37 smithi007 podman[30750]: 2024-09-10 10:36:37.442147597 +0000 UTC m=+0.539647333 container cleanup 983fe6b97b86f4b535e574a41433d9bdd89e900654759a13c9002d8cada3d6d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:36:37.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:37 smithi007 bash[30750]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a 2024-09-10T10:36:38.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:37 smithi007 podman[30762]: 2024-09-10 10:36:37.701183257 +0000 UTC m=+0.608469427 container remove 983fe6b97b86f4b535e574a41433d9bdd89e900654759a13c9002d8cada3d6d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:36:38.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:37 smithi007 systemd[1]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T10:36:38.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:37 smithi007 systemd[1]: Stopped Ceph mon.a for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:36:38.090 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:37 smithi007 systemd[1]: Starting Ceph mon.a for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:36:38.363 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 podman[30854]: 2024-09-10 10:36:37.991622987 +0000 UTC m=+0.031565634 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 podman[30854]: 2024-09-10 10:36:38.102231021 +0000 UTC m=+0.142173710 container create bd02b0b05566704b5752f8296b1ec1caeb63c1b6cbd7fa6582570521d29ba87b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 podman[30854]: 2024-09-10 10:36:38.304485434 +0000 UTC m=+0.344428081 container init bd02b0b05566704b5752f8296b1ec1caeb63c1b6cbd7fa6582570521d29ba87b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 podman[30854]: 2024-09-10 10:36:38.30765825 +0000 UTC m=+0.347600897 container start bd02b0b05566704b5752f8296b1ec1caeb63c1b6cbd7fa6582570521d29ba87b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: pidfile_write: ignore empty --pid-file 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: load: jerasure load: lrc 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: RocksDB version: 7.9.2 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Git sha 0 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T10:36:38.364 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: DB SUMMARY 2024-09-10T10:36:38.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: DB Session ID: IL8J2HGKO91J07WL7FP2 2024-09-10T10:36:38.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: CURRENT file: CURRENT 2024-09-10T10:36:38.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T10:36:38.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T10:36:38.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T10:36:38.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106022 ; 2024-09-10T10:36:38.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.error_if_exists: 0 2024-09-10T10:36:38.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.create_if_missing: 0 2024-09-10T10:36:38.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.paranoid_checks: 1 2024-09-10T10:36:38.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T10:36:38.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T10:36:38.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T10:36:38.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.env: 0x5637799ce940 2024-09-10T10:36:38.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T10:36:38.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.info_log: 0x56377c10d6e0 2024-09-10T10:36:38.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.statistics: (nil) 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.use_fsync: 0 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_log_file_size: 0 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.allow_fallocate: 1 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T10:36:38.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.use_direct_reads: 0 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.db_log_dir: 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.wal_dir: 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T10:36:38.368 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.write_buffer_manager: 0x56377c11cdc0 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.unordered_write: 0 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T10:36:38.369 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.row_cache: None 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.wal_filter: None 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.two_write_queues: 0 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.wal_compression: 0 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.atomic_flush: 0 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T10:36:38.370 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.log_readahead_size: 0 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_background_jobs: 2 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_background_compactions: -1 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_subcompactions: 1 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T10:36:38.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_open_files: -1 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_background_flushes: -1 2024-09-10T10:36:38.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Compression algorithms supported: 2024-09-10T10:36:38.373 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: kZSTD supported: 0 2024-09-10T10:36:38.373 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: kXpressCompression supported: 0 2024-09-10T10:36:38.373 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: kBZip2Compression supported: 0 2024-09-10T10:36:38.373 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T10:36:38.373 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: kLZ4Compression supported: 1 2024-09-10T10:36:38.376 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: kZlibCompression supported: 1 2024-09-10T10:36:38.376 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T10:36:38.376 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: kSnappyCompression supported: 1 2024-09-10T10:36:38.376 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T10:36:38.376 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T10:36:38.376 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T10:36:38.376 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T10:36:38.376 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T10:36:38.376 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.merge_operator: 2024-09-10T10:36:38.376 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_filter: None 2024-09-10T10:36:38.377 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T10:36:38.377 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T10:36:38.377 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T10:36:38.377 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T10:36:38.377 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56377c10d320) 2024-09-10T10:36:38.377 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-09-10T10:36:38.377 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T10:36:38.377 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T10:36:38.377 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_type: 0 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_index_type: 0 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_shortening: 1 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: checksum: 4 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: no_block_cache: 0 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache: 0x56377c1011f0 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_options: 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: capacity : 536870912 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_shard_bits : 4 2024-09-10T10:36:38.378 INFO:journalctl@ceph.mon.a.smithi007.stdout: strict_capacity_limit : 0 2024-09-10T10:36:38.379 INFO:journalctl@ceph.mon.a.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-09-10T10:36:38.379 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_compressed: (nil) 2024-09-10T10:36:38.379 INFO:journalctl@ceph.mon.a.smithi007.stdout: persistent_cache: (nil) 2024-09-10T10:36:38.379 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size: 4096 2024-09-10T10:36:38.379 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size_deviation: 10 2024-09-10T10:36:38.380 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_restart_interval: 16 2024-09-10T10:36:38.380 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_block_restart_interval: 1 2024-09-10T10:36:38.380 INFO:journalctl@ceph.mon.a.smithi007.stdout: metadata_block_size: 4096 2024-09-10T10:36:38.380 INFO:journalctl@ceph.mon.a.smithi007.stdout: partition_filters: 0 2024-09-10T10:36:38.380 INFO:journalctl@ceph.mon.a.smithi007.stdout: use_delta_encoding: 1 2024-09-10T10:36:38.380 INFO:journalctl@ceph.mon.a.smithi007.stdout: filter_policy: bloomfilter 2024-09-10T10:36:38.380 INFO:journalctl@ceph.mon.a.smithi007.stdout: whole_key_filtering: 1 2024-09-10T10:36:38.380 INFO:journalctl@ceph.mon.a.smithi007.stdout: verify_compression: 0 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout: format_version: 5 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_index_compression: 1 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_align: 0 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout: max_auto_readahead_size: 262144 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout: prepopulate_block_cache: 0 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout: initial_auto_readahead_size: 8192 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compression: NoCompression 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T10:36:38.381 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.num_levels: 7 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T10:36:38.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T10:36:38.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T10:36:38.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T10:36:38.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T10:36:38.386 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T10:36:38.386 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T10:36:38.386 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T10:36:38.386 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.inplace_update_support: 0 2024-09-10T10:36:38.386 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T10:36:38.386 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T10:36:38.386 INFO:teuthology.orchestra.run.smithi007.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T10:36:38.387 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T10:36:38.387 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T10:36:38.387 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.bloom_locality: 0 2024-09-10T10:36:38.387 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.max_successive_merges: 0 2024-09-10T10:36:38.387 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T10:36:38.387 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.ttl: 2592000 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.enable_blob_files: false 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.min_blob_size: 0 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T10:36:38.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 694b82f8-81e5-4c6f-9589-14add37983fb 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725964598334740, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725964598335860, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100982, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 98867, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14508, "raw_average_key_size": 49, "raw_value_size": 91049, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725964598, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "694b82f8-81e5-4c6f-9589-14add37983fb", "db_session_id": "IL8J2HGKO91J07WL7FP2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725964598335936, "job": 1, "event": "recovery_finished"} 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T10:36:38.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56377c12ee00 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: DB pointer 0x56377c22a000 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** DB Stats ** 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T10:36:38.390 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: L0 2/0 100.48 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: Sum 2/0 100.48 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 137.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T10:36:38.391 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative compaction: 0.00 GB write, 29.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval compaction: 0.00 GB write, 29.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache BinnedLRUCache@0x56377c1011f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-10T10:36:38.392 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: starting mon.a rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: mon.a@-1(???) e1 preinit fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: mon.a@-1(???).mds e1 new map 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: mon.a@-1(???).mds e1 print_map 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout: e1 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout: btime 2024-09-10T10:36:33:117248+0000 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout: legacy client fscid: -1 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout: No filesystems configured 2024-09-10T10:36:38.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T10:36:38.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T10:36:38.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T10:36:38.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T10:36:38.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 ceph-mon[30873]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T10:36:38.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 bash[30854]: bd02b0b05566704b5752f8296b1ec1caeb63c1b6cbd7fa6582570521d29ba87b 2024-09-10T10:36:38.394 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:38 smithi007 systemd[1]: Started Ceph mon.a for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:36:39.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.748+0000 7f7d73eea640 1 Processor -- start 2024-09-10T10:36:39.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.749+0000 7f7d73eea640 1 -- start start 2024-09-10T10:36:39.343 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.749+0000 7f7d73eea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.749+0000 7f7d73eea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c1080c0 con 0x7f7d6c1076f0 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.749+0000 7f7d71c86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.749+0000 7f7d71c86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59920/0 (socket says 172.21.15.7:59920) 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.749+0000 7f7d71c86640 1 -- 172.21.15.7:0/619649355 learned_addr learned my addr 172.21.15.7:0/619649355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.750+0000 7f7d71c86640 1 -- 172.21.15.7:0/619649355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c1088f0 con 0x7f7d6c1076f0 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.751+0000 7f7d71c86640 1 --2- 172.21.15.7:0/619649355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009820 tx=0x7f7d6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e159201e3f365bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.752+0000 7f7d70c84640 1 -- 172.21.15.7:0/619649355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d68037070 con 0x7f7d6c1076f0 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.752+0000 7f7d70c84640 1 -- 172.21.15.7:0/619649355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7d68032900 con 0x7f7d6c1076f0 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.752+0000 7f7d70c84640 1 -- 172.21.15.7:0/619649355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d68030850 con 0x7f7d6c1076f0 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.752+0000 7f7d73eea640 1 -- 172.21.15.7:0/619649355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 msgr2=0x7f7d6c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.752+0000 7f7d73eea640 1 --2- 172.21.15.7:0/619649355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009820 tx=0x7f7d6802a270 comp rx=0 tx=0).stop 2024-09-10T10:36:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.752+0000 7f7d73eea640 1 -- 172.21.15.7:0/619649355 shutdown_connections 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.752+0000 7f7d73eea640 1 --2- 172.21.15.7:0/619649355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c107af0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.752+0000 7f7d73eea640 1 -- 172.21.15.7:0/619649355 >> 172.21.15.7:0/619649355 conn(0x7f7d6c102e80 msgr2=0x7f7d6c1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.752+0000 7f7d73eea640 1 -- 172.21.15.7:0/619649355 shutdown_connections 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.752+0000 7f7d73eea640 1 -- 172.21.15.7:0/619649355 wait complete. 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.753+0000 7f7d73eea640 1 Processor -- start 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.753+0000 7f7d73eea640 1 -- start start 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.753+0000 7f7d73eea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.753+0000 7f7d73eea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c10bca0 con 0x7f7d6c1076f0 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.753+0000 7f7d71c86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.753+0000 7f7d71c86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c19b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59928/0 (socket says 172.21.15.7:59928) 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.753+0000 7f7d71c86640 1 -- 172.21.15.7:0/2622208500 learned_addr learned my addr 172.21.15.7:0/2622208500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.754+0000 7f7d71c86640 1 -- 172.21.15.7:0/2622208500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c19bc80 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.754+0000 7f7d71c86640 1 --2- 172.21.15.7:0/2622208500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c19b740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009940 tx=0x7f7d68032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.754+0000 7f7d62ffd640 1 -- 172.21.15.7:0/2622208500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d68042070 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.754+0000 7f7d62ffd640 1 -- 172.21.15.7:0/2622208500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7d680308c0 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.754+0000 7f7d73eea640 1 -- 172.21.15.7:0/2622208500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c19bf10 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.754+0000 7f7d62ffd640 1 -- 172.21.15.7:0/2622208500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d68037040 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.754+0000 7f7d73eea640 1 -- 172.21.15.7:0/2622208500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c19ec00 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.754+0000 7f7d62ffd640 1 -- 172.21.15.7:0/2622208500 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d68045070 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.754+0000 7f7d62ffd640 1 -- 172.21.15.7:0/2622208500 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d68047b70 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.755+0000 7f7d73eea640 1 -- 172.21.15.7:0/2622208500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c10be20 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.757+0000 7f7d62ffd640 1 -- 172.21.15.7:0/2622208500 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7d680327b0 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.816+0000 7f7d73eea640 1 -- 172.21.15.7:0/2622208500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7d6c107b70 con 0x7f7d6c1076f0 2024-09-10T10:36:39.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.818+0000 7f7d62ffd640 1 -- 172.21.15.7:0/2622208500 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7d6803c760 con 0x7f7d6c1076f0 2024-09-10T10:36:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.819+0000 7f7d73eea640 1 -- 172.21.15.7:0/2622208500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 msgr2=0x7f7d6c19b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.819+0000 7f7d73eea640 1 --2- 172.21.15.7:0/2622208500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c19b740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009940 tx=0x7f7d68032ce0 comp rx=0 tx=0).stop 2024-09-10T10:36:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.819+0000 7f7d73eea640 1 -- 172.21.15.7:0/2622208500 shutdown_connections 2024-09-10T10:36:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.819+0000 7f7d73eea640 1 --2- 172.21.15.7:0/2622208500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c1076f0 0x7f7d6c19b740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.819+0000 7f7d73eea640 1 -- 172.21.15.7:0/2622208500 >> 172.21.15.7:0/2622208500 conn(0x7f7d6c102e80 msgr2=0x7f7d6c103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.819+0000 7f7d73eea640 1 -- 172.21.15.7:0/2622208500 shutdown_connections 2024-09-10T10:36:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:38.819+0000 7f7d73eea640 1 -- 172.21.15.7:0/2622208500 wait complete. 2024-09-10T10:36:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T10:36:39.369 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:36:39.369 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mgr... 2024-09-10T10:36:39.370 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T10:36:39.370 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T10:36:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T10:36:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: monmap epoch 1 2024-09-10T10:36:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: last_changed 2024-09-10T10:36:29.861118+0000 2024-09-10T10:36:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: created 2024-09-10T10:36:29.861118+0000 2024-09-10T10:36:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: min_mon_release 19 (squid) 2024-09-10T10:36:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: election_strategy: 1 2024-09-10T10:36:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-10T10:36:39.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: fsmap 2024-09-10T10:36:39.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T10:36:39.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: mgrmap e1: no daemons active 2024-09-10T10:36:39.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:39 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2622208500' entity='client.admin' 2024-09-10T10:36:39.638 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.x 2024-09-10T10:36:39.638 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T10:36:39.793 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-65af4d96-6f60-11ef-bcea-c7b262605968.target.wants/ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-65af4d96-6f60-11ef-bcea-c7b262605968@.service. 2024-09-10T10:36:39.815 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:39 smithi007 systemd[1]: Starting Ceph mgr.x for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:36:40.159 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 podman[31084]: 2024-09-10 10:36:40.046409976 +0000 UTC m=+0.112830613 container create 562a96b83dc9c9c174fedb37d9689f676e946a598b79afe4ca3bab29c04ff24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T10:36:40.160 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 podman[31084]: 2024-09-10 10:36:39.949967905 +0000 UTC m=+0.016388543 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:36:40.347 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T10:36:40.348 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-09-10T10:36:40.355 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T10:36:40.355 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-10T10:36:40.355 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-09-10T10:36:40.355 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T10:36:40.364 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T10:36:40.364 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-09-10T10:36:40.371 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T10:36:40.372 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-10T10:36:40.372 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-09-10T10:36:40.372 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T10:36:40.372 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr to start... 2024-09-10T10:36:40.372 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr... 2024-09-10T10:36:40.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 podman[31084]: 2024-09-10 10:36:40.257379317 +0000 UTC m=+0.323799955 container init 562a96b83dc9c9c174fedb37d9689f676e946a598b79afe4ca3bab29c04ff24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T10:36:40.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 podman[31084]: 2024-09-10 10:36:40.260198625 +0000 UTC m=+0.326619263 container start 562a96b83dc9c9c174fedb37d9689f676e946a598b79afe4ca3bab29c04ff24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T10:36:40.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 bash[31084]: 562a96b83dc9c9c174fedb37d9689f676e946a598b79afe4ca3bab29c04ff24e 2024-09-10T10:36:40.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 systemd[1]: Started Ceph mgr.x for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:36:40.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:40.387+0000 7fba2b890040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:36:40.736 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:40.451+0000 7fba2b890040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:36:40.736 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:40.608+0000 7fba2b890040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:36:40.736 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:40.681+0000 7fba2b890040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:36:40.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:40 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/338935979' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T10:36:40.993 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:40.841+0000 7fba2b890040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:36:41.328 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:40.991+0000 7fba2b890040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:36:41.328 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:41 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:41.327+0000 7fba2b890040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:36:41.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:41.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:36:41.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "65af4d96-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:36:41.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T10:36:41.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T10:36:41.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:41.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T10:36:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T10:36:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T10:36:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T10:36:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T10:36:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T10:36:41.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T10:36:41.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T10:36:41.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T10:36:41.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:41.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T10:36:41.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T10:36:41.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T10:36:41.505 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T10:36:33:117248+0000", 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:41.506 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T10:36:33.118081+0000", 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:36:41.507 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.816+0000 7f67a6eb7640 1 Processor -- start 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.816+0000 7f67a6eb7640 1 -- start start 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.816+0000 7f67a6eb7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a0107b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.816+0000 7f67a6eb7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0108140 con 0x7f67a0107770 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.816+0000 7f67a4c53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a0107b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.816+0000 7f67a4c53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a0107b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49744/0 (socket says 172.21.15.7:49744) 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.816+0000 7f67a4c53640 1 -- 172.21.15.7:0/4017244486 learned_addr learned my addr 172.21.15.7:0/4017244486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.817+0000 7f67a4c53640 1 -- 172.21.15.7:0/4017244486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a01088d0 con 0x7f67a0107770 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.818+0000 7f67a4c53640 1 --2- 172.21.15.7:0/4017244486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a0107b70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6794009820 tx=0x7f679402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d05f11d6e37666 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.818+0000 7f679f7fe640 1 -- 172.21.15.7:0/4017244486 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6794037070 con 0x7f67a0107770 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.818+0000 7f679f7fe640 1 -- 172.21.15.7:0/4017244486 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6794032900 con 0x7f67a0107770 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.818+0000 7f679f7fe640 1 -- 172.21.15.7:0/4017244486 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6794030850 con 0x7f67a0107770 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.819+0000 7f67a6eb7640 1 -- 172.21.15.7:0/4017244486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 msgr2=0x7f67a0107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.819+0000 7f67a6eb7640 1 --2- 172.21.15.7:0/4017244486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a0107b70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6794009820 tx=0x7f679402a270 comp rx=0 tx=0).stop 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.819+0000 7f67a6eb7640 1 -- 172.21.15.7:0/4017244486 shutdown_connections 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.819+0000 7f67a6eb7640 1 --2- 172.21.15.7:0/4017244486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a0107b70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.819+0000 7f67a6eb7640 1 -- 172.21.15.7:0/4017244486 >> 172.21.15.7:0/4017244486 conn(0x7f67a0102f00 msgr2=0x7f67a0105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.819+0000 7f67a6eb7640 1 -- 172.21.15.7:0/4017244486 shutdown_connections 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.819+0000 7f67a6eb7640 1 -- 172.21.15.7:0/4017244486 wait complete. 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.819+0000 7f67a6eb7640 1 Processor -- start 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.819+0000 7f67a6eb7640 1 -- start start 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.820+0000 7f67a6eb7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a019b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:41.509 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.820+0000 7f67a6eb7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67a010bc80 con 0x7f67a0107770 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.820+0000 7f67a4c53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a019b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.820+0000 7f67a4c53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a019b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49754/0 (socket says 172.21.15.7:49754) 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.820+0000 7f67a4c53640 1 -- 172.21.15.7:0/338935979 learned_addr learned my addr 172.21.15.7:0/338935979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.820+0000 7f67a4c53640 1 -- 172.21.15.7:0/338935979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a019bc40 con 0x7f67a0107770 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.820+0000 7f67a4c53640 1 --2- 172.21.15.7:0/338935979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a019b700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67940097f0 tx=0x7f6794030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.820+0000 7f679dffb640 1 -- 172.21.15.7:0/338935979 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6794037040 con 0x7f67a0107770 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.820+0000 7f679dffb640 1 -- 172.21.15.7:0/338935979 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6794040070 con 0x7f67a0107770 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.820+0000 7f67a6eb7640 1 -- 172.21.15.7:0/338935979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a019bed0 con 0x7f67a0107770 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.821+0000 7f679dffb640 1 -- 172.21.15.7:0/338935979 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f679403a390 con 0x7f67a0107770 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.821+0000 7f67a6eb7640 1 -- 172.21.15.7:0/338935979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a019ebc0 con 0x7f67a0107770 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.821+0000 7f679dffb640 1 -- 172.21.15.7:0/338935979 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f679403a630 con 0x7f67a0107770 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.821+0000 7f679dffb640 1 -- 172.21.15.7:0/338935979 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f67940459a0 con 0x7f67a0107770 2024-09-10T10:36:41.510 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.822+0000 7f67a6eb7640 1 -- 172.21.15.7:0/338935979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a010be00 con 0x7f67a0107770 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.824+0000 7f679dffb640 1 -- 172.21.15.7:0/338935979 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6794045460 con 0x7f67a0107770 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.886+0000 7f67a6eb7640 1 -- 172.21.15.7:0/338935979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f67a019ed90 con 0x7f67a0107770 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.886+0000 7f679dffb640 1 -- 172.21.15.7:0/338935979 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f679403b430 con 0x7f67a0107770 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.888+0000 7f67a6eb7640 1 -- 172.21.15.7:0/338935979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 msgr2=0x7f67a019b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.888+0000 7f67a6eb7640 1 --2- 172.21.15.7:0/338935979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a019b700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67940097f0 tx=0x7f6794030d80 comp rx=0 tx=0).stop 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.888+0000 7f67a6eb7640 1 -- 172.21.15.7:0/338935979 shutdown_connections 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.888+0000 7f67a6eb7640 1 --2- 172.21.15.7:0/338935979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a0107770 0x7f67a019b700 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.888+0000 7f67a6eb7640 1 -- 172.21.15.7:0/338935979 >> 172.21.15.7:0/338935979 conn(0x7f67a0102f00 msgr2=0x7f67a0103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.888+0000 7f67a6eb7640 1 -- 172.21.15.7:0/338935979 shutdown_connections 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:40.888+0000 7f67a6eb7640 1 -- 172.21.15.7:0/338935979 wait complete. 2024-09-10T10:36:41.511 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (1/15)... 2024-09-10T10:36:41.590 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:41 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:41.391+0000 7fba2b890040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:36:41.590 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:41 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:41.588+0000 7fba2b890040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:36:42.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:42 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:42.515+0000 7fba2b890040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:36:42.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:42 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:42.585+0000 7fba2b890040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:36:43.614 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:43.340+0000 7fba2b890040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:36:43.614 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:43.460+0000 7fba2b890040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:36:43.885 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:43.613+0000 7fba2b890040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:36:43.885 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:43.677+0000 7fba2b890040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:36:43.885 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:43.836+0000 7fba2b890040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:36:44.145 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:44 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/114037297' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T10:36:44.145 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:43.909+0000 7fba2b890040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:36:44.146 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:43.974+0000 7fba2b890040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:36:44.146 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:44.143+0000 7fba2b890040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:36:44.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:44.214+0000 7fba2b890040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:36:44.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:44.283+0000 7fba2b890040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:36:44.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:44.370+0000 7fba2b890040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "65af4d96-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T10:36:44.668 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T10:36:44.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T10:36:44.670 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T10:36:44.680 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T10:36:44.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T10:36:44.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T10:36:44.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:44.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T10:36:44.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T10:36:44.681 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T10:36:44.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T10:36:44.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T10:36:44.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T10:36:44.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T10:36:44.682 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T10:36:44.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T10:36:44.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:44.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T10:36:44.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:44.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T10:36:33:117248+0000", 2024-09-10T10:36:44.683 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T10:36:44.684 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T10:36:44.684 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:44.684 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T10:36:44.684 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T10:36:44.684 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T10:36:44.685 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T10:36:44.685 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T10:36:44.685 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T10:36:44.685 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T10:36:44.685 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:36:44.685 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:36:44.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:44.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T10:36:44.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:44.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T10:36:33.118081+0000", 2024-09-10T10:36:44.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:36:44.686 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:44.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T10:36:44.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:36:44.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.963+0000 7f1b33edc640 1 Processor -- start 2024-09-10T10:36:44.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.963+0000 7f1b33edc640 1 -- start start 2024-09-10T10:36:44.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.964+0000 7f1b33edc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:44.687 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.964+0000 7f1b33edc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c107000 con 0x7f1b2c106630 2024-09-10T10:36:44.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.964+0000 7f1b31c78640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:44.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.964+0000 7f1b31c78640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49768/0 (socket says 172.21.15.7:49768) 2024-09-10T10:36:44.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.964+0000 7f1b31c78640 1 -- 172.21.15.7:0/2601546593 learned_addr learned my addr 172.21.15.7:0/2601546593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:44.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.964+0000 7f1b31c78640 1 -- 172.21.15.7:0/2601546593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c107830 con 0x7f1b2c106630 2024-09-10T10:36:44.688 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.965+0000 7f1b31c78640 1 --2- 172.21.15.7:0/2601546593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c106a30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009820 tx=0x7f1b2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676f3d1fd2707bf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:44.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.965+0000 7f1b30c76640 1 -- 172.21.15.7:0/2601546593 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b28037070 con 0x7f1b2c106630 2024-09-10T10:36:44.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.965+0000 7f1b30c76640 1 -- 172.21.15.7:0/2601546593 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b28032900 con 0x7f1b2c106630 2024-09-10T10:36:44.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.965+0000 7f1b30c76640 1 -- 172.21.15.7:0/2601546593 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b28030850 con 0x7f1b2c106630 2024-09-10T10:36:44.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.966+0000 7f1b33edc640 1 -- 172.21.15.7:0/2601546593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 msgr2=0x7f1b2c106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:44.689 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.966+0000 7f1b33edc640 1 --2- 172.21.15.7:0/2601546593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c106a30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009820 tx=0x7f1b2802a270 comp rx=0 tx=0).stop 2024-09-10T10:36:44.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.966+0000 7f1b33edc640 1 -- 172.21.15.7:0/2601546593 shutdown_connections 2024-09-10T10:36:44.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.966+0000 7f1b33edc640 1 --2- 172.21.15.7:0/2601546593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c106a30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:44.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.966+0000 7f1b33edc640 1 -- 172.21.15.7:0/2601546593 >> 172.21.15.7:0/2601546593 conn(0x7f1b2c101e20 msgr2=0x7f1b2c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:44.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.966+0000 7f1b33edc640 1 -- 172.21.15.7:0/2601546593 shutdown_connections 2024-09-10T10:36:44.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.966+0000 7f1b33edc640 1 -- 172.21.15.7:0/2601546593 wait complete. 2024-09-10T10:36:44.690 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.966+0000 7f1b33edc640 1 Processor -- start 2024-09-10T10:36:44.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b33edc640 1 -- start start 2024-09-10T10:36:44.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b33edc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:44.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b33edc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c10abe0 con 0x7f1b2c106630 2024-09-10T10:36:44.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b31c78640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:44.691 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b31c78640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c19b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49776/0 (socket says 172.21.15.7:49776) 2024-09-10T10:36:44.692 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b31c78640 1 -- 172.21.15.7:0/114037297 learned_addr learned my addr 172.21.15.7:0/114037297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:44.692 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b31c78640 1 -- 172.21.15.7:0/114037297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c19bd40 con 0x7f1b2c106630 2024-09-10T10:36:44.692 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b31c78640 1 --2- 172.21.15.7:0/114037297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c19b800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009940 tx=0x7f1b280326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:44.692 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b22ffd640 1 -- 172.21.15.7:0/114037297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b28042070 con 0x7f1b2c106630 2024-09-10T10:36:44.692 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b22ffd640 1 -- 172.21.15.7:0/114037297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b28032be0 con 0x7f1b2c106630 2024-09-10T10:36:44.693 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b33edc640 1 -- 172.21.15.7:0/114037297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c19bfd0 con 0x7f1b2c106630 2024-09-10T10:36:44.693 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.967+0000 7f1b22ffd640 1 -- 172.21.15.7:0/114037297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b28037040 con 0x7f1b2c106630 2024-09-10T10:36:44.693 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.968+0000 7f1b33edc640 1 -- 172.21.15.7:0/114037297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c19ecc0 con 0x7f1b2c106630 2024-09-10T10:36:44.693 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.968+0000 7f1b22ffd640 1 -- 172.21.15.7:0/114037297 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1b28045070 con 0x7f1b2c106630 2024-09-10T10:36:44.693 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.968+0000 7f1b22ffd640 1 -- 172.21.15.7:0/114037297 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1b28047a30 con 0x7f1b2c106630 2024-09-10T10:36:44.694 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.968+0000 7f1b33edc640 1 -- 172.21.15.7:0/114037297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c10ad60 con 0x7f1b2c106630 2024-09-10T10:36:44.694 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:43.971+0000 7f1b22ffd640 1 -- 172.21.15.7:0/114037297 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1b28032d90 con 0x7f1b2c106630 2024-09-10T10:36:44.694 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:44.033+0000 7f1b33edc640 1 -- 172.21.15.7:0/114037297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1b2c195700 con 0x7f1b2c106630 2024-09-10T10:36:44.694 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:44.033+0000 7f1b22ffd640 1 -- 172.21.15.7:0/114037297 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f1b2803d430 con 0x7f1b2c106630 2024-09-10T10:36:44.694 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:44.034+0000 7f1b33edc640 1 -- 172.21.15.7:0/114037297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 msgr2=0x7f1b2c19b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:44.694 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:44.034+0000 7f1b33edc640 1 --2- 172.21.15.7:0/114037297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c19b800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009940 tx=0x7f1b280326e0 comp rx=0 tx=0).stop 2024-09-10T10:36:44.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:44.034+0000 7f1b33edc640 1 -- 172.21.15.7:0/114037297 shutdown_connections 2024-09-10T10:36:44.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:44.034+0000 7f1b33edc640 1 --2- 172.21.15.7:0/114037297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b2c106630 0x7f1b2c19b800 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:44.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:44.034+0000 7f1b33edc640 1 -- 172.21.15.7:0/114037297 >> 172.21.15.7:0/114037297 conn(0x7f1b2c101e20 msgr2=0x7f1b2c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:44.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:44.034+0000 7f1b33edc640 1 -- 172.21.15.7:0/114037297 shutdown_connections 2024-09-10T10:36:44.695 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:44.034+0000 7f1b33edc640 1 -- 172.21.15.7:0/114037297 wait complete. 2024-09-10T10:36:44.695 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (2/15)... 2024-09-10T10:36:44.936 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:44.433+0000 7fba2b890040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:36:44.937 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:44.494+0000 7fba2b890040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:36:45.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:45.342+0000 7fba2b890040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:36:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: Activating manager daemon x 2024-09-10T10:36:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: mgrmap e2: x(active, starting, since 0.00315663s) 2024-09-10T10:36:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: from='mgr.14100 172.21.15.7:0/401342056' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:36:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: from='mgr.14100 172.21.15.7:0/401342056' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:36:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: from='mgr.14100 172.21.15.7:0/401342056' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:36:45.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: from='mgr.14100 172.21.15.7:0/401342056' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:36:45.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: from='mgr.14100 172.21.15.7:0/401342056' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:36:45.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: Manager daemon x is now available 2024-09-10T10:36:45.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: from='mgr.14100 172.21.15.7:0/401342056' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:36:45.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: from='mgr.14100 172.21.15.7:0/401342056' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: from='mgr.14100 172.21.15.7:0/401342056' entity='mgr.x' 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: from='mgr.14100 172.21.15.7:0/401342056' entity='mgr.x' 2024-09-10T10:36:45.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:45 smithi007 ceph-mon[30873]: from='mgr.14100 172.21.15.7:0/401342056' entity='mgr.x' 2024-09-10T10:36:47.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:47 smithi007 ceph-mon[30873]: mgrmap e3: x(active, since 1.00749s) 2024-09-10T10:36:47.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:47 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1701757314' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T10:36:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:36:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "65af4d96-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:36:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T10:36:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T10:36:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T10:36:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T10:36:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T10:36:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:47.959 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T10:36:47.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:47.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T10:36:47.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T10:36:47.960 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T10:36:47.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T10:36:47.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T10:36:47.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T10:36:47.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:47.971 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T10:36:47.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T10:36:47.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T10:36:47.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T10:36:47.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T10:36:47.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T10:36:47.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T10:36:47.972 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T10:36:47.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T10:36:47.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:47.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T10:36:47.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:47.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T10:36:33:117248+0000", 2024-09-10T10:36:47.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T10:36:47.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T10:36:47.973 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:47.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T10:36:47.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T10:36:47.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T10:36:47.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T10:36:47.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T10:36:47.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T10:36:47.974 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T10:36:47.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-09-10T10:36:47.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:36:47.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:47.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T10:36:47.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T10:36:47.975 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T10:36:33.118081+0000", 2024-09-10T10:36:47.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T10:36:47.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-09-10T10:36:47.976 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T10:36:47.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:36:47.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.079+0000 7fd6363cd640 1 Processor -- start 2024-09-10T10:36:47.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.079+0000 7fd6363cd640 1 -- start start 2024-09-10T10:36:47.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.079+0000 7fd6363cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd630108080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:47.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.079+0000 7fd6363cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd630108650 con 0x7fd630107c80 2024-09-10T10:36:47.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.079+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd630108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:47.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.079+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd630108080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49856/0 (socket says 172.21.15.7:49856) 2024-09-10T10:36:47.977 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.079+0000 7fd62ffff640 1 -- 172.21.15.7:0/2350463304 learned_addr learned my addr 172.21.15.7:0/2350463304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:47.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.080+0000 7fd62ffff640 1 -- 172.21.15.7:0/2350463304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6301087d0 con 0x7fd630107c80 2024-09-10T10:36:47.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.081+0000 7fd62ffff640 1 --2- 172.21.15.7:0/2350463304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd630108080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd620009820 tx=0x7fd62002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3833beff39b2b14e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:47.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.081+0000 7fd62effd640 1 -- 172.21.15.7:0/2350463304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd620037070 con 0x7fd630107c80 2024-09-10T10:36:47.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.081+0000 7fd62effd640 1 -- 172.21.15.7:0/2350463304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd620032900 con 0x7fd630107c80 2024-09-10T10:36:47.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.081+0000 7fd62effd640 1 -- 172.21.15.7:0/2350463304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd620030850 con 0x7fd630107c80 2024-09-10T10:36:47.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.081+0000 7fd6363cd640 1 -- 172.21.15.7:0/2350463304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 msgr2=0x7fd630108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:47.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.081+0000 7fd6363cd640 1 --2- 172.21.15.7:0/2350463304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd630108080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd620009820 tx=0x7fd62002a270 comp rx=0 tx=0).stop 2024-09-10T10:36:47.978 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.081+0000 7fd6363cd640 1 -- 172.21.15.7:0/2350463304 shutdown_connections 2024-09-10T10:36:47.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.081+0000 7fd6363cd640 1 --2- 172.21.15.7:0/2350463304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd630108080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.081+0000 7fd6363cd640 1 -- 172.21.15.7:0/2350463304 >> 172.21.15.7:0/2350463304 conn(0x7fd630101530 msgr2=0x7fd630103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:47.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.081+0000 7fd6363cd640 1 -- 172.21.15.7:0/2350463304 shutdown_connections 2024-09-10T10:36:47.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.082+0000 7fd6363cd640 1 -- 172.21.15.7:0/2350463304 wait complete. 2024-09-10T10:36:47.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.082+0000 7fd6363cd640 1 Processor -- start 2024-09-10T10:36:47.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.082+0000 7fd6363cd640 1 -- start start 2024-09-10T10:36:47.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd6363cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd63019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:47.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd6363cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6301066a0 con 0x7fd630107c80 2024-09-10T10:36:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd63019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd62ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd63019d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49872/0 (socket says 172.21.15.7:49872) 2024-09-10T10:36:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd62ffff640 1 -- 172.21.15.7:0/1701757314 learned_addr learned my addr 172.21.15.7:0/1701757314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd62ffff640 1 -- 172.21.15.7:0/1701757314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63019dde0 con 0x7fd630107c80 2024-09-10T10:36:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd62ffff640 1 --2- 172.21.15.7:0/1701757314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd63019d8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd620032ce0 tx=0x7fd620032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd62d7fa640 1 -- 172.21.15.7:0/1701757314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd620042070 con 0x7fd630107c80 2024-09-10T10:36:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd62d7fa640 1 -- 172.21.15.7:0/1701757314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd6200308c0 con 0x7fd630107c80 2024-09-10T10:36:47.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd6363cd640 1 -- 172.21.15.7:0/1701757314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63019e070 con 0x7fd630107c80 2024-09-10T10:36:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.083+0000 7fd62d7fa640 1 -- 172.21.15.7:0/1701757314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd620037040 con 0x7fd630107c80 2024-09-10T10:36:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.084+0000 7fd6363cd640 1 -- 172.21.15.7:0/1701757314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6301a0d60 con 0x7fd630107c80 2024-09-10T10:36:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.084+0000 7fd62d7fa640 1 -- 172.21.15.7:0/1701757314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fd620045070 con 0x7fd630107c80 2024-09-10T10:36:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.084+0000 7fd6363cd640 1 -- 172.21.15.7:0/1701757314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd630108100 con 0x7fd630107c80 2024-09-10T10:36:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.085+0000 7fd62d7fa640 1 --2- 172.21.15.7:0/1701757314 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fd618038e60 0x7fd61803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.085+0000 7fd62d7fa640 1 -- 172.21.15.7:0/1701757314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd62006bdb0 con 0x7fd630107c80 2024-09-10T10:36:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.085+0000 7fd62f7fe640 1 --2- 172.21.15.7:0/1701757314 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fd618038e60 0x7fd61803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:47.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.085+0000 7fd62f7fe640 1 --2- 172.21.15.7:0/1701757314 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fd618038e60 0x7fd61803b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd6240098c0 tx=0x7fd624006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.090+0000 7fd62d7fa640 1 -- 172.21.15.7:0/1701757314 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd620045360 con 0x7fd630107c80 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.332+0000 7fd6363cd640 1 -- 172.21.15.7:0/1701757314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd630196de0 con 0x7fd630107c80 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.333+0000 7fd62d7fa640 1 -- 172.21.15.7:0/1701757314 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fd62002ed90 con 0x7fd630107c80 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.336+0000 7fd6363cd640 1 -- 172.21.15.7:0/1701757314 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fd618038e60 msgr2=0x7fd61803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.336+0000 7fd6363cd640 1 --2- 172.21.15.7:0/1701757314 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fd618038e60 0x7fd61803b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd6240098c0 tx=0x7fd624006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.336+0000 7fd6363cd640 1 -- 172.21.15.7:0/1701757314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 msgr2=0x7fd63019d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.336+0000 7fd6363cd640 1 --2- 172.21.15.7:0/1701757314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd63019d8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd620032ce0 tx=0x7fd620032d10 comp rx=0 tx=0).stop 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.336+0000 7fd6363cd640 1 -- 172.21.15.7:0/1701757314 shutdown_connections 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.336+0000 7fd6363cd640 1 --2- 172.21.15.7:0/1701757314 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fd618038e60 0x7fd61803b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.336+0000 7fd6363cd640 1 --2- 172.21.15.7:0/1701757314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd630107c80 0x7fd63019d8a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.336+0000 7fd6363cd640 1 -- 172.21.15.7:0/1701757314 >> 172.21.15.7:0/1701757314 conn(0x7fd630101530 msgr2=0x7fd630193f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:47.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.336+0000 7fd6363cd640 1 -- 172.21.15.7:0/1701757314 shutdown_connections 2024-09-10T10:36:47.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:47.336+0000 7fd6363cd640 1 -- 172.21.15.7:0/1701757314 wait complete. 2024-09-10T10:36:47.983 INFO:teuthology.orchestra.run.smithi007.stdout:mgr is available 2024-09-10T10:36:48.567 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:48 smithi007 ceph-mon[30873]: mgrmap e4: x(active, since 2s) 2024-09-10T10:36:49.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:49.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T10:36:49.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:36:49.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T10:36:49.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T10:36:49.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T10:36:49.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.380+0000 7ff2c5d87640 1 Processor -- start 2024-09-10T10:36:49.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.381+0000 7ff2c5d87640 1 -- start start 2024-09-10T10:36:49.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.381+0000 7ff2c5d87640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c0106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:49.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.381+0000 7ff2c5d87640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0107000 con 0x7ff2c0106630 2024-09-10T10:36:49.116 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.381+0000 7ff2bf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c0106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:49.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.381+0000 7ff2bf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c0106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49878/0 (socket says 172.21.15.7:49878) 2024-09-10T10:36:49.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.381+0000 7ff2bf7fe640 1 -- 172.21.15.7:0/3010444516 learned_addr learned my addr 172.21.15.7:0/3010444516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:49.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.381+0000 7ff2bf7fe640 1 -- 172.21.15.7:0/3010444516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c0107830 con 0x7ff2c0106630 2024-09-10T10:36:49.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.382+0000 7ff2bf7fe640 1 --2- 172.21.15.7:0/3010444516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c0106a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009820 tx=0x7ff2b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28187bde9e77563e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:49.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.382+0000 7ff2be7fc640 1 -- 172.21.15.7:0/3010444516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0037070 con 0x7ff2c0106630 2024-09-10T10:36:49.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.382+0000 7ff2be7fc640 1 -- 172.21.15.7:0/3010444516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2b0032900 con 0x7ff2c0106630 2024-09-10T10:36:49.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.382+0000 7ff2be7fc640 1 -- 172.21.15.7:0/3010444516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0030850 con 0x7ff2c0106630 2024-09-10T10:36:49.117 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.383+0000 7ff2c5d87640 1 -- 172.21.15.7:0/3010444516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 msgr2=0x7ff2c0106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:49.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.383+0000 7ff2c5d87640 1 --2- 172.21.15.7:0/3010444516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c0106a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009820 tx=0x7ff2b002a270 comp rx=0 tx=0).stop 2024-09-10T10:36:49.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.383+0000 7ff2c5d87640 1 -- 172.21.15.7:0/3010444516 shutdown_connections 2024-09-10T10:36:49.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.383+0000 7ff2c5d87640 1 --2- 172.21.15.7:0/3010444516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c0106a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.383+0000 7ff2c5d87640 1 -- 172.21.15.7:0/3010444516 >> 172.21.15.7:0/3010444516 conn(0x7ff2c0101e20 msgr2=0x7ff2c0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:49.118 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.383+0000 7ff2c5d87640 1 -- 172.21.15.7:0/3010444516 shutdown_connections 2024-09-10T10:36:49.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.383+0000 7ff2c5d87640 1 -- 172.21.15.7:0/3010444516 wait complete. 2024-09-10T10:36:49.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.384+0000 7ff2c5d87640 1 Processor -- start 2024-09-10T10:36:49.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.384+0000 7ff2c5d87640 1 -- start start 2024-09-10T10:36:49.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.384+0000 7ff2c5d87640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:49.119 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.384+0000 7ff2c5d87640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c010abe0 con 0x7ff2c0106630 2024-09-10T10:36:49.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.384+0000 7ff2bf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:49.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.384+0000 7ff2bf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c019b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49888/0 (socket says 172.21.15.7:49888) 2024-09-10T10:36:49.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.384+0000 7ff2bf7fe640 1 -- 172.21.15.7:0/2947991912 learned_addr learned my addr 172.21.15.7:0/2947991912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:49.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.384+0000 7ff2bf7fe640 1 -- 172.21.15.7:0/2947991912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c019bd40 con 0x7ff2c0106630 2024-09-10T10:36:49.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.385+0000 7ff2bf7fe640 1 --2- 172.21.15.7:0/2947991912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c019b800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00097f0 tx=0x7ff2b0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:49.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.385+0000 7ff2bcff9640 1 -- 172.21.15.7:0/2947991912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0042070 con 0x7ff2c0106630 2024-09-10T10:36:49.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.385+0000 7ff2bcff9640 1 -- 172.21.15.7:0/2947991912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2b0032ca0 con 0x7ff2c0106630 2024-09-10T10:36:49.120 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.385+0000 7ff2c5d87640 1 -- 172.21.15.7:0/2947991912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c019bfd0 con 0x7ff2c0106630 2024-09-10T10:36:49.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.385+0000 7ff2bcff9640 1 -- 172.21.15.7:0/2947991912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0037040 con 0x7ff2c0106630 2024-09-10T10:36:49.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.385+0000 7ff2c5d87640 1 -- 172.21.15.7:0/2947991912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c019ecc0 con 0x7ff2c0106630 2024-09-10T10:36:49.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.386+0000 7ff2bcff9640 1 -- 172.21.15.7:0/2947991912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff2b0045070 con 0x7ff2c0106630 2024-09-10T10:36:49.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.386+0000 7ff2bcff9640 1 --2- 172.21.15.7:0/2947991912 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7ff2a8038f20 0x7ff2a803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:49.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.386+0000 7ff2c5d87640 1 -- 172.21.15.7:0/2947991912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c010ad60 con 0x7ff2c0106630 2024-09-10T10:36:49.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.386+0000 7ff2bcff9640 1 -- 172.21.15.7:0/2947991912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff2b006bfd0 con 0x7ff2c0106630 2024-09-10T10:36:49.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.386+0000 7ff2beffd640 1 --2- 172.21.15.7:0/2947991912 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7ff2a8038f20 0x7ff2a803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:49.121 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.387+0000 7ff2beffd640 1 --2- 172.21.15.7:0/2947991912 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7ff2a8038f20 0x7ff2a803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40098c0 tx=0x7ff2b4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:49.122 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.391+0000 7ff2bcff9640 1 -- 172.21.15.7:0/2947991912 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2b00043c0 con 0x7ff2c0106630 2024-09-10T10:36:49.122 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.566+0000 7ff2c5d87640 1 -- 172.21.15.7:0/2947991912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff2c0106ab0 con 0x7ff2c0106630 2024-09-10T10:36:49.122 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.567+0000 7ff2bcff9640 1 -- 172.21.15.7:0/2947991912 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+427 (secure 0 0 0) 0x7ff2b003ccd0 con 0x7ff2c0106630 2024-09-10T10:36:49.122 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.569+0000 7ff2c5d87640 1 -- 172.21.15.7:0/2947991912 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7ff2a8038f20 msgr2=0x7ff2a803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:49.122 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.569+0000 7ff2c5d87640 1 --2- 172.21.15.7:0/2947991912 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7ff2a8038f20 0x7ff2a803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40098c0 tx=0x7ff2b4006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.123 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.569+0000 7ff2c5d87640 1 -- 172.21.15.7:0/2947991912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 msgr2=0x7ff2c019b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:49.123 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.569+0000 7ff2c5d87640 1 --2- 172.21.15.7:0/2947991912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c019b800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00097f0 tx=0x7ff2b0030ec0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.123 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.570+0000 7ff2c5d87640 1 -- 172.21.15.7:0/2947991912 shutdown_connections 2024-09-10T10:36:49.123 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.570+0000 7ff2c5d87640 1 --2- 172.21.15.7:0/2947991912 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7ff2a8038f20 0x7ff2a803b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.123 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.570+0000 7ff2c5d87640 1 --2- 172.21.15.7:0/2947991912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2c0106630 0x7ff2c019b800 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:49.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.570+0000 7ff2c5d87640 1 -- 172.21.15.7:0/2947991912 >> 172.21.15.7:0/2947991912 conn(0x7ff2c0101e20 msgr2=0x7ff2c0102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:49.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.570+0000 7ff2c5d87640 1 -- 172.21.15.7:0/2947991912 shutdown_connections 2024-09-10T10:36:49.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:48.570+0000 7ff2c5d87640 1 -- 172.21.15.7:0/2947991912 wait complete. 2024-09-10T10:36:49.124 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling cephadm module... 2024-09-10T10:36:49.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:49 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2947991912' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T10:36:50.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:50 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4251285163' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T10:36:50.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: ignoring --setuser ceph since I am not root 2024-09-10T10:36:50.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: ignoring --setgroup ceph since I am not root 2024-09-10T10:36:50.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:50.503+0000 7ffbbbad5040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:36:50.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:50.562+0000 7ffbbbad5040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:36:51.006 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:50.717+0000 7ffbbbad5040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:36:51.006 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:50.787+0000 7ffbbbad5040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:36:51.007 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:50.940+0000 7ffbbbad5040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:36:51.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.475+0000 7f98d696b640 1 Processor -- start 2024-09-10T10:36:51.014 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.475+0000 7f98d696b640 1 -- start start 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.475+0000 7f98d696b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d0107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.475+0000 7f98d696b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98d01080c0 con 0x7f98d01076f0 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.475+0000 7f98cffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d0107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.476+0000 7f98cffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d0107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49892/0 (socket says 172.21.15.7:49892) 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.476+0000 7f98cffff640 1 -- 172.21.15.7:0/379797978 learned_addr learned my addr 172.21.15.7:0/379797978 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.476+0000 7f98cffff640 1 -- 172.21.15.7:0/379797978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d01088f0 con 0x7f98d01076f0 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.476+0000 7f98cffff640 1 --2- 172.21.15.7:0/379797978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d0107af0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009820 tx=0x7f98c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd23b9a29481909 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.477+0000 7f98ceffd640 1 -- 172.21.15.7:0/379797978 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c0037070 con 0x7f98d01076f0 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.477+0000 7f98ceffd640 1 -- 172.21.15.7:0/379797978 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98c0032900 con 0x7f98d01076f0 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.477+0000 7f98ceffd640 1 -- 172.21.15.7:0/379797978 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c0030850 con 0x7f98d01076f0 2024-09-10T10:36:51.015 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.477+0000 7f98d696b640 1 -- 172.21.15.7:0/379797978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 msgr2=0x7f98d0107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.477+0000 7f98d696b640 1 --2- 172.21.15.7:0/379797978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d0107af0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009820 tx=0x7f98c002a270 comp rx=0 tx=0).stop 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.477+0000 7f98d696b640 1 -- 172.21.15.7:0/379797978 shutdown_connections 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.477+0000 7f98d696b640 1 --2- 172.21.15.7:0/379797978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d0107af0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.477+0000 7f98d696b640 1 -- 172.21.15.7:0/379797978 >> 172.21.15.7:0/379797978 conn(0x7f98d0102e80 msgr2=0x7f98d01052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.477+0000 7f98d696b640 1 -- 172.21.15.7:0/379797978 shutdown_connections 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.477+0000 7f98d696b640 1 -- 172.21.15.7:0/379797978 wait complete. 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.478+0000 7f98d696b640 1 Processor -- start 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.478+0000 7f98d696b640 1 -- start start 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.478+0000 7f98d696b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d007db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.478+0000 7f98d696b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98d010bca0 con 0x7f98d01076f0 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.478+0000 7f98cffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.478+0000 7f98cffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49906/0 (socket says 172.21.15.7:49906) 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.478+0000 7f98cffff640 1 -- 172.21.15.7:0/4251285163 learned_addr learned my addr 172.21.15.7:0/4251285163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.479+0000 7f98cffff640 1 -- 172.21.15.7:0/4251285163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d007e090 con 0x7f98d01076f0 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.479+0000 7f98cffff640 1 --2- 172.21.15.7:0/4251285163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d007db50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009940 tx=0x7f98c0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.479+0000 7f98cd7fa640 1 -- 172.21.15.7:0/4251285163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c0042070 con 0x7f98d01076f0 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.479+0000 7f98cd7fa640 1 -- 172.21.15.7:0/4251285163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98c00308c0 con 0x7f98d01076f0 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.479+0000 7f98d696b640 1 -- 172.21.15.7:0/4251285163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d007c280 con 0x7f98d01076f0 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.479+0000 7f98cd7fa640 1 -- 172.21.15.7:0/4251285163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98c0037040 con 0x7f98d01076f0 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.479+0000 7f98d696b640 1 -- 172.21.15.7:0/4251285163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d007c7c0 con 0x7f98d01076f0 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.480+0000 7f98cd7fa640 1 -- 172.21.15.7:0/4251285163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f98c0045070 con 0x7f98d01076f0 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.480+0000 7f98d696b640 1 -- 172.21.15.7:0/4251285163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d010be20 con 0x7f98d01076f0 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.480+0000 7f98cd7fa640 1 --2- 172.21.15.7:0/4251285163 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7f98b8038f20 0x7f98b803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.480+0000 7f98cd7fa640 1 -- 172.21.15.7:0/4251285163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f98c006bfe0 con 0x7f98d01076f0 2024-09-10T10:36:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.480+0000 7f98cf7fe640 1 --2- 172.21.15.7:0/4251285163 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7f98b8038f20 0x7f98b803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.481+0000 7f98cf7fe640 1 --2- 172.21.15.7:0/4251285163 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7f98b8038f20 0x7f98b803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98c40098c0 tx=0x7f98c4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.485+0000 7f98cd7fa640 1 -- 172.21.15.7:0/4251285163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98c003cd90 con 0x7f98d01076f0 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:49.708+0000 7f98d696b640 1 -- 172.21.15.7:0/4251285163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f98d007cfe0 con 0x7f98d01076f0 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.354+0000 7f98cd7fa640 1 -- 172.21.15.7:0/4251285163 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f98c006ba00 con 0x7f98d01076f0 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.356+0000 7f98cd7fa640 1 -- 172.21.15.7:0/4251285163 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f98c003c870 con 0x7f98d01076f0 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.357+0000 7f98d696b640 1 -- 172.21.15.7:0/4251285163 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7f98b8038f20 msgr2=0x7f98b803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.357+0000 7f98d696b640 1 --2- 172.21.15.7:0/4251285163 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7f98b8038f20 0x7f98b803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98c40098c0 tx=0x7f98c4006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.357+0000 7f98d696b640 1 -- 172.21.15.7:0/4251285163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 msgr2=0x7f98d007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.357+0000 7f98d696b640 1 --2- 172.21.15.7:0/4251285163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d007db50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009940 tx=0x7f98c0032ce0 comp rx=0 tx=0).stop 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.357+0000 7f98d696b640 1 -- 172.21.15.7:0/4251285163 shutdown_connections 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.357+0000 7f98d696b640 1 --2- 172.21.15.7:0/4251285163 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7f98b8038f20 0x7f98b803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:51.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.357+0000 7f98d696b640 1 --2- 172.21.15.7:0/4251285163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98d01076f0 0x7f98d007db50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:51.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.357+0000 7f98d696b640 1 -- 172.21.15.7:0/4251285163 >> 172.21.15.7:0/4251285163 conn(0x7f98d0102e80 msgr2=0x7f98d0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:51.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.357+0000 7f98d696b640 1 -- 172.21.15.7:0/4251285163 shutdown_connections 2024-09-10T10:36:51.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:50.357+0000 7f98d696b640 1 -- 172.21.15.7:0/4251285163 wait complete. 2024-09-10T10:36:51.356 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:51 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:51.095+0000 7ffbbbad5040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:36:51.653 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:51 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4251285163' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T10:36:51.653 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:51 smithi007 ceph-mon[30873]: mgrmap e5: x(active, since 5s) 2024-09-10T10:36:51.654 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:51 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:51.434+0000 7ffbbbad5040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:36:51.654 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:51 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:51.499+0000 7ffbbbad5040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:36:51.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:51 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:51.696+0000 7ffbbbad5040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:36:52.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:36:52.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T10:36:52.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T10:36:52.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T10:36:52.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T10:36:52.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:36:52.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.445+0000 7efdae826640 1 Processor -- start 2024-09-10T10:36:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.445+0000 7efdae826640 1 -- start start 2024-09-10T10:36:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.446+0000 7efdae826640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda8106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.446+0000 7efdae826640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efda8107000 con 0x7efda8106630 2024-09-10T10:36:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.446+0000 7efda7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda8106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.446+0000 7efda7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda8106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39036/0 (socket says 172.21.15.7:39036) 2024-09-10T10:36:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.446+0000 7efda7fff640 1 -- 172.21.15.7:0/493185093 learned_addr learned my addr 172.21.15.7:0/493185093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.446+0000 7efda7fff640 1 -- 172.21.15.7:0/493185093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda8107830 con 0x7efda8106630 2024-09-10T10:36:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.447+0000 7efda7fff640 1 --2- 172.21.15.7:0/493185093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda8106a30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efd98009820 tx=0x7efd9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58113e7a482927f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:52.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.447+0000 7efda6ffd640 1 -- 172.21.15.7:0/493185093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd98037070 con 0x7efda8106630 2024-09-10T10:36:52.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.447+0000 7efda6ffd640 1 -- 172.21.15.7:0/493185093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd98032900 con 0x7efda8106630 2024-09-10T10:36:52.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.447+0000 7efda6ffd640 1 -- 172.21.15.7:0/493185093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd98030850 con 0x7efda8106630 2024-09-10T10:36:52.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.448+0000 7efdae826640 1 -- 172.21.15.7:0/493185093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 msgr2=0x7efda8106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:52.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.448+0000 7efdae826640 1 --2- 172.21.15.7:0/493185093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda8106a30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efd98009820 tx=0x7efd9802a270 comp rx=0 tx=0).stop 2024-09-10T10:36:52.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.448+0000 7efdae826640 1 -- 172.21.15.7:0/493185093 shutdown_connections 2024-09-10T10:36:52.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.448+0000 7efdae826640 1 --2- 172.21.15.7:0/493185093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda8106a30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:52.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.448+0000 7efdae826640 1 -- 172.21.15.7:0/493185093 >> 172.21.15.7:0/493185093 conn(0x7efda8101e20 msgr2=0x7efda8104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:52.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.448+0000 7efdae826640 1 -- 172.21.15.7:0/493185093 shutdown_connections 2024-09-10T10:36:52.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.448+0000 7efdae826640 1 -- 172.21.15.7:0/493185093 wait complete. 2024-09-10T10:36:52.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.449+0000 7efdae826640 1 Processor -- start 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.449+0000 7efdae826640 1 -- start start 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.449+0000 7efdae826640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda807ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.449+0000 7efdae826640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efda810abe0 con 0x7efda8106630 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.449+0000 7efda7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda807ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.449+0000 7efda7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda807ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39040/0 (socket says 172.21.15.7:39040) 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.449+0000 7efda7fff640 1 -- 172.21.15.7:0/3359567437 learned_addr learned my addr 172.21.15.7:0/3359567437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.450+0000 7efda7fff640 1 -- 172.21.15.7:0/3359567437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda819e520 con 0x7efda8106630 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.450+0000 7efda7fff640 1 --2- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda807ba30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efd98009940 tx=0x7efd980326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.450+0000 7efda57fa640 1 -- 172.21.15.7:0/3359567437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd98042070 con 0x7efda8106630 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.450+0000 7efda57fa640 1 -- 172.21.15.7:0/3359567437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd98032be0 con 0x7efda8106630 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.450+0000 7efdae826640 1 -- 172.21.15.7:0/3359567437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda819e6f0 con 0x7efda8106630 2024-09-10T10:36:52.358 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.450+0000 7efda57fa640 1 -- 172.21.15.7:0/3359567437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd98037040 con 0x7efda8106630 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.450+0000 7efdae826640 1 -- 172.21.15.7:0/3359567437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda819ea90 con 0x7efda8106630 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.451+0000 7efda57fa640 1 -- 172.21.15.7:0/3359567437 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7efd98045070 con 0x7efda8106630 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.451+0000 7efdae826640 1 -- 172.21.15.7:0/3359567437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda810ad60 con 0x7efda8106630 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.451+0000 7efda57fa640 1 --2- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7efd90038fc0 0x7efd9003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.451+0000 7efda57fa640 1 -- 172.21.15.7:0/3359567437 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd9806cf20 con 0x7efda8106630 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.451+0000 7efda77fe640 1 -- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7efd90038fc0 msgr2=0x7efd9003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4050686422 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.451+0000 7efda77fe640 1 --2- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7efd90038fc0 0x7efd9003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.456+0000 7efda57fa640 1 -- 172.21.15.7:0/3359567437 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd98045360 con 0x7efda8106630 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.652+0000 7efda77fe640 1 -- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7efd90038fc0 msgr2=0x7efd9003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4050686422 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.652+0000 7efda77fe640 1 --2- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7efd90038fc0 0x7efd9003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.680+0000 7efdae826640 1 -- 172.21.15.7:0/3359567437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7efda8106ab0 con 0x7efda8106630 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.680+0000 7efda57fa640 1 -- 172.21.15.7:0/3359567437 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7efd9803abf0 con 0x7efda8106630 2024-09-10T10:36:52.359 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.682+0000 7efdae826640 1 -- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7efd90038fc0 msgr2=0x7efd9003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.682+0000 7efdae826640 1 --2- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7efd90038fc0 0x7efd9003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.682+0000 7efdae826640 1 -- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 msgr2=0x7efda807ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.682+0000 7efdae826640 1 --2- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda807ba30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efd98009940 tx=0x7efd980326e0 comp rx=0 tx=0).stop 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.682+0000 7efdae826640 1 -- 172.21.15.7:0/3359567437 shutdown_connections 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.682+0000 7efdae826640 1 --2- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7efd90038fc0 0x7efd9003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.682+0000 7efdae826640 1 --2- 172.21.15.7:0/3359567437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efda8106630 0x7efda807ba30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.682+0000 7efdae826640 1 -- 172.21.15.7:0/3359567437 >> 172.21.15.7:0/3359567437 conn(0x7efda8101e20 msgr2=0x7efda8102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.683+0000 7efdae826640 1 -- 172.21.15.7:0/3359567437 shutdown_connections 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:51.683+0000 7efdae826640 1 -- 172.21.15.7:0/3359567437 wait complete. 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-09-10T10:36:52.360 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 5... 2024-09-10T10:36:52.632 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:52 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3359567437' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T10:36:52.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:52.630+0000 7ffbbbad5040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:36:52.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:52.702+0000 7ffbbbad5040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:36:53.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:53 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:53.448+0000 7ffbbbad5040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:36:53.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:53 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:53.569+0000 7ffbbbad5040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:36:54.008 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:53 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:53.718+0000 7ffbbbad5040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:36:54.009 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:53 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:53.781+0000 7ffbbbad5040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:36:54.009 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:53 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:53.937+0000 7ffbbbad5040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:36:54.009 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:54.007+0000 7ffbbbad5040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:36:54.304 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:54.070+0000 7ffbbbad5040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:36:54.304 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:54.234+0000 7ffbbbad5040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:36:54.304 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:54.302+0000 7ffbbbad5040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:36:54.582 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:54.371+0000 7ffbbbad5040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:36:54.582 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:54.457+0000 7ffbbbad5040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:36:54.582 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:54.519+0000 7ffbbbad5040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:36:54.582 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:54.580+0000 7ffbbbad5040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:36:55.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:55.405+0000 7ffbbbad5040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:36:55.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: Active manager daemon x restarted 2024-09-10T10:36:55.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: Activating manager daemon x 2024-09-10T10:36:55.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T10:36:55.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: mgrmap e6: x(active, starting, since 0.00337135s) 2024-09-10T10:36:55.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:36:55.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:36:55.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:36:55.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:36:55.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:36:55.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: Manager daemon x is now available 2024-09-10T10:36:55.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:36:55.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:36:55.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:55.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:55.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:36:55.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:55 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:36:56.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:56 smithi007 ceph-mon[30873]: Found migration_current of "None". Setting to last migration. 2024-09-10T10:36:56.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:56 smithi007 ceph-mon[30873]: mgrmap e7: x(active, since 1.00524s) 2024-09-10T10:36:57.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:36:57.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T10:36:57.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T10:36:57.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:36:57.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.761+0000 7fb18a466640 1 Processor -- start 2024-09-10T10:36:57.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.761+0000 7fb18a466640 1 -- start start 2024-09-10T10:36:57.029 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.761+0000 7fb18a466640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb18410a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:57.029 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.761+0000 7fb18a466640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb18407be90 con 0x7fb1841081a0 2024-09-10T10:36:57.029 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.761+0000 7fb183fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb18410a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:57.029 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.761+0000 7fb183fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb18410a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39048/0 (socket says 172.21.15.7:39048) 2024-09-10T10:36:57.029 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.761+0000 7fb183fff640 1 -- 172.21.15.7:0/2576581622 learned_addr learned my addr 172.21.15.7:0/2576581622 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:57.029 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.762+0000 7fb183fff640 1 -- 172.21.15.7:0/2576581622 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18410aad0 con 0x7fb1841081a0 2024-09-10T10:36:57.029 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.762+0000 7fb183fff640 1 --2- 172.21.15.7:0/2576581622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb18410a590 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb174009820 tx=0x7fb17402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce02aa98ea3bf9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:57.032 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.762+0000 7fb182ffd640 1 -- 172.21.15.7:0/2576581622 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb174037070 con 0x7fb1841081a0 2024-09-10T10:36:57.032 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.763+0000 7fb182ffd640 1 -- 172.21.15.7:0/2576581622 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb174032900 con 0x7fb1841081a0 2024-09-10T10:36:57.032 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.763+0000 7fb182ffd640 1 -- 172.21.15.7:0/2576581622 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb174030850 con 0x7fb1841081a0 2024-09-10T10:36:57.032 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.763+0000 7fb18a466640 1 -- 172.21.15.7:0/2576581622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 msgr2=0x7fb18410a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:57.032 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.763+0000 7fb18a466640 1 --2- 172.21.15.7:0/2576581622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb18410a590 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb174009820 tx=0x7fb17402a270 comp rx=0 tx=0).stop 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.763+0000 7fb18a466640 1 -- 172.21.15.7:0/2576581622 shutdown_connections 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.763+0000 7fb18a466640 1 --2- 172.21.15.7:0/2576581622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb18410a590 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.763+0000 7fb18a466640 1 -- 172.21.15.7:0/2576581622 >> 172.21.15.7:0/2576581622 conn(0x7fb184101e20 msgr2=0x7fb184104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.763+0000 7fb18a466640 1 -- 172.21.15.7:0/2576581622 shutdown_connections 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.763+0000 7fb18a466640 1 -- 172.21.15.7:0/2576581622 wait complete. 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.764+0000 7fb18a466640 1 Processor -- start 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.764+0000 7fb18a466640 1 -- start start 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.764+0000 7fb18a466640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb184107a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.764+0000 7fb18a466640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb18410bb70 con 0x7fb1841081a0 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.764+0000 7fb183fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb184107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:57.033 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.764+0000 7fb183fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb184107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39060/0 (socket says 172.21.15.7:39060) 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.764+0000 7fb183fff640 1 -- 172.21.15.7:0/1209948470 learned_addr learned my addr 172.21.15.7:0/1209948470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.765+0000 7fb183fff640 1 -- 172.21.15.7:0/1209948470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb184107f60 con 0x7fb1841081a0 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.765+0000 7fb183fff640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb184107a20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb1740097f0 tx=0x7fb174030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.765+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb174042070 con 0x7fb1841081a0 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.765+0000 7fb18a466640 1 -- 172.21.15.7:0/1209948470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb184106150 con 0x7fb1841081a0 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.765+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb174032ca0 con 0x7fb1841081a0 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.765+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb174037040 con 0x7fb1841081a0 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.765+0000 7fb18a466640 1 -- 172.21.15.7:0/1209948470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb184106690 con 0x7fb1841081a0 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.766+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb174045070 con 0x7fb1841081a0 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.766+0000 7fb1817fa640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 0x7fb16c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.766+0000 7fb18a466640 1 -- 172.21.15.7:0/1209948470 --> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb184108600 con 0x7fb16c038f70 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.766+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb17406bfc0 con 0x7fb1841081a0 2024-09-10T10:36:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.766+0000 7fb1837fe640 1 -- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 msgr2=0x7fb16c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4050686422 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.766+0000 7fb1837fe640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 0x7fb16c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.967+0000 7fb1837fe640 1 -- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 msgr2=0x7fb16c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4050686422 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:52.967+0000 7fb1837fe640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 0x7fb16c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:53.368+0000 7fb1837fe640 1 -- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 msgr2=0x7fb16c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4050686422 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:53.368+0000 7fb1837fe640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 0x7fb16c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:54.169+0000 7fb1837fe640 1 -- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 msgr2=0x7fb16c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/4050686422 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:54.169+0000 7fb1837fe640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 0x7fb16c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:55.409+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fb17406ac30 con 0x7fb1841081a0 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:55.409+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 msgr2=0x7fb16c03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:55.409+0000 7fb1817fa640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 0x7fb16c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.410+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fb17403cd10 con 0x7fb1841081a0 2024-09-10T10:36:57.035 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.410+0000 7fb1817fa640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fb16c03c6b0 0x7fb16c03eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.410+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 --> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb184108600 con 0x7fb16c03c6b0 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.410+0000 7fb1837fe640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fb16c03c6b0 0x7fb16c03eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.411+0000 7fb1837fe640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fb16c03c6b0 0x7fb16c03eaa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb178003980 tx=0x7fb178006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.412+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 <== mgr.14118 v2:172.21.15.7:6800/2266895831 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb184108600 con 0x7fb16c03c6b0 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.424+0000 7fb18a466640 1 -- 172.21.15.7:0/1209948470 --> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb1841030e0 con 0x7fb16c03c6b0 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb1817fa640 1 -- 172.21.15.7:0/1209948470 <== mgr.14118 v2:172.21.15.7:6800/2266895831 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb1841030e0 con 0x7fb16c03c6b0 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 -- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fb16c03c6b0 msgr2=0x7fb16c03eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fb16c03c6b0 0x7fb16c03eaa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb178003980 tx=0x7fb178006e10 comp rx=0 tx=0).stop 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 -- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 msgr2=0x7fb184107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb184107a20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb1740097f0 tx=0x7fb174030ec0 comp rx=0 tx=0).stop 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 -- 172.21.15.7:0/1209948470 shutdown_connections 2024-09-10T10:36:57.036 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fb16c03c6b0 0x7fb16c03eaa0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:57.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:6800/4050686422,v1:172.21.15.7:6801/4050686422] conn(0x7fb16c038f70 0x7fb16c03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:57.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 --2- 172.21.15.7:0/1209948470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1841081a0 0x7fb184107a20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:57.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 -- 172.21.15.7:0/1209948470 >> 172.21.15.7:0/1209948470 conn(0x7fb184101e20 msgr2=0x7fb1841029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:57.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 -- 172.21.15.7:0/1209948470 shutdown_connections 2024-09-10T10:36:57.037 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:56.425+0000 7fb18a466640 1 -- 172.21.15.7:0/1209948470 wait complete. 2024-09-10T10:36:57.037 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 5 is available 2024-09-10T10:36:57.037 INFO:teuthology.orchestra.run.smithi007.stdout:Setting orchestrator backend to cephadm... 2024-09-10T10:36:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-mon[30873]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T10:36:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-mon[30873]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T10:36:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:36:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:36:57.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:36:57.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:57.935 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:57.869+0000 7ffb285f3640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:10:36:57] ENGINE Error in HTTPServer.serve 2024-09-10T10:36:57.935 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: Traceback (most recent call last): 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self._connections.run(self.expiration_interval) 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self._run(expiration_interval) 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:36:57.936 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:36:57.937 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: s = self.context.wrap_socket( 2024-09-10T10:36:57.937 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:36:57.937 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: return self.sslsocket_class._create( 2024-09-10T10:36:57.937 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:36:57.937 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self.do_handshake() 2024-09-10T10:36:57.937 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:36:57.937 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self._sslobj.do_handshake() 2024-09-10T10:36:57.937 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:36:57.937 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:36:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:36:58.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.434+0000 7f5c7958c640 1 Processor -- start 2024-09-10T10:36:58.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.434+0000 7f5c7958c640 1 -- start start 2024-09-10T10:36:58.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.434+0000 7f5c7958c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c7410a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:58.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.434+0000 7f5c7958c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7407be90 con 0x7f5c741081a0 2024-09-10T10:36:58.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.435+0000 7f5c72ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c7410a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:58.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.435+0000 7f5c72ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c7410a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39104/0 (socket says 172.21.15.7:39104) 2024-09-10T10:36:58.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.435+0000 7f5c72ffd640 1 -- 172.21.15.7:0/105700341 learned_addr learned my addr 172.21.15.7:0/105700341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:58.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.435+0000 7f5c72ffd640 1 -- 172.21.15.7:0/105700341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7410aad0 con 0x7f5c741081a0 2024-09-10T10:36:58.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.435+0000 7f5c72ffd640 1 --2- 172.21.15.7:0/105700341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c7410a590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009820 tx=0x7f5c6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d3211484f7d312 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:58.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.436+0000 7f5c71ffb640 1 -- 172.21.15.7:0/105700341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c64037070 con 0x7f5c741081a0 2024-09-10T10:36:58.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.436+0000 7f5c71ffb640 1 -- 172.21.15.7:0/105700341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5c64032900 con 0x7f5c741081a0 2024-09-10T10:36:58.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.436+0000 7f5c71ffb640 1 -- 172.21.15.7:0/105700341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c64030850 con 0x7f5c741081a0 2024-09-10T10:36:58.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.436+0000 7f5c7958c640 1 -- 172.21.15.7:0/105700341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 msgr2=0x7f5c7410a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:58.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.436+0000 7f5c7958c640 1 --2- 172.21.15.7:0/105700341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c7410a590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009820 tx=0x7f5c6402a270 comp rx=0 tx=0).stop 2024-09-10T10:36:58.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.436+0000 7f5c7958c640 1 -- 172.21.15.7:0/105700341 shutdown_connections 2024-09-10T10:36:58.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.436+0000 7f5c7958c640 1 --2- 172.21.15.7:0/105700341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c7410a590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:58.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.436+0000 7f5c7958c640 1 -- 172.21.15.7:0/105700341 >> 172.21.15.7:0/105700341 conn(0x7f5c74101e20 msgr2=0x7f5c74104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:58.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.436+0000 7f5c7958c640 1 -- 172.21.15.7:0/105700341 shutdown_connections 2024-09-10T10:36:58.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.436+0000 7f5c7958c640 1 -- 172.21.15.7:0/105700341 wait complete. 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.437+0000 7f5c7958c640 1 Processor -- start 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.437+0000 7f5c7958c640 1 -- start start 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.437+0000 7f5c7958c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c74107a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.437+0000 7f5c7958c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7410bb70 con 0x7f5c741081a0 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.438+0000 7f5c72ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c74107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.438+0000 7f5c72ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c74107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39110/0 (socket says 172.21.15.7:39110) 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.438+0000 7f5c72ffd640 1 -- 172.21.15.7:0/3664077923 learned_addr learned my addr 172.21.15.7:0/3664077923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.438+0000 7f5c72ffd640 1 -- 172.21.15.7:0/3664077923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c74107f60 con 0x7f5c741081a0 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.438+0000 7f5c72ffd640 1 --2- 172.21.15.7:0/3664077923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c74107a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c64031000 tx=0x7f5c640326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.438+0000 7f5c5bfff640 1 -- 172.21.15.7:0/3664077923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c64042070 con 0x7f5c741081a0 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.438+0000 7f5c5bfff640 1 -- 172.21.15.7:0/3664077923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5c64032be0 con 0x7f5c741081a0 2024-09-10T10:36:58.269 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.438+0000 7f5c7958c640 1 -- 172.21.15.7:0/3664077923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c74106150 con 0x7f5c741081a0 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.438+0000 7f5c5bfff640 1 -- 172.21.15.7:0/3664077923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c64037040 con 0x7f5c741081a0 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.438+0000 7f5c7958c640 1 -- 172.21.15.7:0/3664077923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c74106690 con 0x7f5c741081a0 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.439+0000 7f5c5bfff640 1 -- 172.21.15.7:0/3664077923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f5c64045070 con 0x7f5c741081a0 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.439+0000 7f5c7958c640 1 -- 172.21.15.7:0/3664077923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c74108600 con 0x7f5c741081a0 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.440+0000 7f5c5bfff640 1 --2- 172.21.15.7:0/3664077923 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5c5c038e60 0x7f5c5c03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.440+0000 7f5c5bfff640 1 -- 172.21.15.7:0/3664077923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5c6406bf50 con 0x7f5c741081a0 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.440+0000 7f5c727fc640 1 --2- 172.21.15.7:0/3664077923 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5c5c038e60 0x7f5c5c03b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.440+0000 7f5c727fc640 1 --2- 172.21.15.7:0/3664077923 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5c5c038e60 0x7f5c5c03b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5c680098c0 tx=0x7f5c68006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.445+0000 7f5c5bfff640 1 -- 172.21.15.7:0/3664077923 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c6403d8f0 con 0x7f5c741081a0 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.635+0000 7f5c7958c640 1 -- 172.21.15.7:0/3664077923 --> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f5c74106fe0 con 0x7f5c5c038e60 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.640+0000 7f5c5bfff640 1 -- 172.21.15.7:0/3664077923 <== mgr.14118 v2:172.21.15.7:6800/2266895831 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5c74106fe0 con 0x7f5c5c038e60 2024-09-10T10:36:58.270 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.642+0000 7f5c7958c640 1 -- 172.21.15.7:0/3664077923 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5c5c038e60 msgr2=0x7f5c5c03b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:58.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.642+0000 7f5c7958c640 1 --2- 172.21.15.7:0/3664077923 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5c5c038e60 0x7f5c5c03b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5c680098c0 tx=0x7f5c68006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:58.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.642+0000 7f5c7958c640 1 -- 172.21.15.7:0/3664077923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 msgr2=0x7f5c74107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:58.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.642+0000 7f5c7958c640 1 --2- 172.21.15.7:0/3664077923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c74107a20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c64031000 tx=0x7f5c640326e0 comp rx=0 tx=0).stop 2024-09-10T10:36:58.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.642+0000 7f5c7958c640 1 -- 172.21.15.7:0/3664077923 shutdown_connections 2024-09-10T10:36:58.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.642+0000 7f5c7958c640 1 --2- 172.21.15.7:0/3664077923 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5c5c038e60 0x7f5c5c03b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:58.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.642+0000 7f5c7958c640 1 --2- 172.21.15.7:0/3664077923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c741081a0 0x7f5c74107a20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:58.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.642+0000 7f5c7958c640 1 -- 172.21.15.7:0/3664077923 >> 172.21.15.7:0/3664077923 conn(0x7f5c74101e20 msgr2=0x7f5c74102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:58.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.642+0000 7f5c7958c640 1 -- 172.21.15.7:0/3664077923 shutdown_connections 2024-09-10T10:36:58.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:57.643+0000 7f5c7958c640 1 -- 172.21.15.7:0/3664077923 wait complete. 2024-09-10T10:36:58.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:58 smithi007 ceph-mon[30873]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:36:58.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:58 smithi007 ceph-mon[30873]: [10/Sep/2024:10:36:57] ENGINE Bus STARTING 2024-09-10T10:36:58.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:58 smithi007 ceph-mon[30873]: mgrmap e8: x(active, since 2s) 2024-09-10T10:36:58.865 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:58 smithi007 ceph-mon[30873]: [10/Sep/2024:10:36:57] ENGINE Serving on https://172.21.15.7:7150 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:58 smithi007 ceph-mon[30873]: [10/Sep/2024:10:36:57] ENGINE Error in HTTPServer.serve 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout: Traceback (most recent call last): 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout: self._connections.run(self.expiration_interval) 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout: self._run(expiration_interval) 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:36:58.866 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout: s = self.context.wrap_socket( 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout: return self.sslsocket_class._create( 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout: self.do_handshake() 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout: self._sslobj.do_handshake() 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:58 smithi007 ceph-mon[30873]: [10/Sep/2024:10:36:57] ENGINE Serving on http://172.21.15.7:8765 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:58 smithi007 ceph-mon[30873]: [10/Sep/2024:10:36:57] ENGINE Bus STARTED 2024-09-10T10:36:58.867 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:58 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:36:59.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T10:36:59.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.675+0000 7f7938135640 1 Processor -- start 2024-09-10T10:36:59.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.676+0000 7f7938135640 1 -- start start 2024-09-10T10:36:59.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.676+0000 7f7938135640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f793010a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:59.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.676+0000 7f7938135640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f793007be90 con 0x7f79301081a0 2024-09-10T10:36:59.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.676+0000 7f7935ed1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f793010a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:59.447 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.676+0000 7f7935ed1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f793010a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39112/0 (socket says 172.21.15.7:39112) 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.676+0000 7f7935ed1640 1 -- 172.21.15.7:0/1009297348 learned_addr learned my addr 172.21.15.7:0/1009297348 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.676+0000 7f7935ed1640 1 -- 172.21.15.7:0/1009297348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793010aad0 con 0x7f79301081a0 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.677+0000 7f7935ed1640 1 --2- 172.21.15.7:0/1009297348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f793010a590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f792c009e80 tx=0x7f792c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71d693d3b5e516c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.677+0000 7f7934ecf640 1 -- 172.21.15.7:0/1009297348 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f792c036070 con 0x7f79301081a0 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.677+0000 7f7934ecf640 1 -- 172.21.15.7:0/1009297348 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f792c0318c0 con 0x7f79301081a0 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.677+0000 7f7934ecf640 1 -- 172.21.15.7:0/1009297348 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f792c007ab0 con 0x7f79301081a0 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.678+0000 7f7938135640 1 -- 172.21.15.7:0/1009297348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 msgr2=0x7f793010a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.678+0000 7f7938135640 1 --2- 172.21.15.7:0/1009297348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f793010a590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f792c009e80 tx=0x7f792c02a7a0 comp rx=0 tx=0).stop 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.678+0000 7f7938135640 1 -- 172.21.15.7:0/1009297348 shutdown_connections 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.678+0000 7f7938135640 1 --2- 172.21.15.7:0/1009297348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f793010a590 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.678+0000 7f7938135640 1 -- 172.21.15.7:0/1009297348 >> 172.21.15.7:0/1009297348 conn(0x7f7930101e20 msgr2=0x7f7930104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.678+0000 7f7938135640 1 -- 172.21.15.7:0/1009297348 shutdown_connections 2024-09-10T10:36:59.448 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.678+0000 7f7938135640 1 -- 172.21.15.7:0/1009297348 wait complete. 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.678+0000 7f7938135640 1 Processor -- start 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.678+0000 7f7938135640 1 -- start start 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7938135640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f7930107a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7938135640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f793010bb70 con 0x7f79301081a0 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7935ed1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f7930107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7935ed1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f7930107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39124/0 (socket says 172.21.15.7:39124) 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7935ed1640 1 -- 172.21.15.7:0/1555495472 learned_addr learned my addr 172.21.15.7:0/1555495472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7935ed1640 1 -- 172.21.15.7:0/1555495472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7930107f60 con 0x7f79301081a0 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7935ed1640 1 --2- 172.21.15.7:0/1555495472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f7930107a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f792c02acc0 tx=0x7f792c031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7926ffd640 1 -- 172.21.15.7:0/1555495472 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f792c036050 con 0x7f79301081a0 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7926ffd640 1 -- 172.21.15.7:0/1555495472 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f792c037040 con 0x7f79301081a0 2024-09-10T10:36:59.449 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7938135640 1 -- 172.21.15.7:0/1555495472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7930106150 con 0x7f79301081a0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.679+0000 7f7926ffd640 1 -- 172.21.15.7:0/1555495472 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f792c03d920 con 0x7f79301081a0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.680+0000 7f7938135640 1 -- 172.21.15.7:0/1555495472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7930106690 con 0x7f79301081a0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.680+0000 7f7926ffd640 1 -- 172.21.15.7:0/1555495472 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f792c044020 con 0x7f79301081a0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.681+0000 7f7938135640 1 -- 172.21.15.7:0/1555495472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7930108600 con 0x7f79301081a0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.681+0000 7f7926ffd640 1 --2- 172.21.15.7:0/1555495472 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f791c038fc0 0x7f791c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.681+0000 7f7926ffd640 1 -- 172.21.15.7:0/1555495472 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f792c042020 con 0x7f79301081a0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.681+0000 7f79356d0640 1 --2- 172.21.15.7:0/1555495472 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f791c038fc0 0x7f791c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.681+0000 7f79356d0640 1 --2- 172.21.15.7:0/1555495472 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f791c038fc0 0x7f791c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79200098c0 tx=0x7f7920006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.686+0000 7f7926ffd640 1 -- 172.21.15.7:0/1555495472 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f792c04b020 con 0x7f79301081a0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.863+0000 7f7938135640 1 -- 172.21.15.7:0/1555495472 --> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f7930106fe0 con 0x7f791c038fc0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.864+0000 7f7926ffd640 1 -- 172.21.15.7:0/1555495472 <== mgr.14118 v2:172.21.15.7:6800/2266895831 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f7930106fe0 con 0x7f791c038fc0 2024-09-10T10:36:59.450 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.866+0000 7f7938135640 1 -- 172.21.15.7:0/1555495472 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f791c038fc0 msgr2=0x7f791c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.866+0000 7f7938135640 1 --2- 172.21.15.7:0/1555495472 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f791c038fc0 0x7f791c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79200098c0 tx=0x7f7920006db0 comp rx=0 tx=0).stop 2024-09-10T10:36:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.866+0000 7f7938135640 1 -- 172.21.15.7:0/1555495472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 msgr2=0x7f7930107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:36:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.866+0000 7f7938135640 1 --2- 172.21.15.7:0/1555495472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f7930107a20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f792c02acc0 tx=0x7f792c031ca0 comp rx=0 tx=0).stop 2024-09-10T10:36:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.866+0000 7f7938135640 1 -- 172.21.15.7:0/1555495472 shutdown_connections 2024-09-10T10:36:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.866+0000 7f7938135640 1 --2- 172.21.15.7:0/1555495472 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f791c038fc0 0x7f791c03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.866+0000 7f7938135640 1 --2- 172.21.15.7:0/1555495472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301081a0 0x7f7930107a20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:36:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.866+0000 7f7938135640 1 -- 172.21.15.7:0/1555495472 >> 172.21.15.7:0/1555495472 conn(0x7f7930101e20 msgr2=0x7f79301029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:36:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.866+0000 7f7938135640 1 -- 172.21.15.7:0/1555495472 shutdown_connections 2024-09-10T10:36:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:58.866+0000 7f7938135640 1 -- 172.21.15.7:0/1555495472 wait complete. 2024-09-10T10:36:59.451 INFO:teuthology.orchestra.run.smithi007.stdout:Generating ssh key... 2024-09-10T10:37:00.060 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:36:59 smithi007 ceph-mon[30873]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:00.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: Generating public/private rsa key pair. 2024-09-10T10:37:00.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: Your identification has been saved in /tmp/tmpumvta_j8/key 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: Your public key has been saved in /tmp/tmpumvta_j8/key.pub 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: The key fingerprint is: 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: SHA256:zCjp52wpc6Qc6FzylQqEIFwLdnJZSBhTV4Q3eXQkueE ceph-65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: The key's randomart image is: 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: +---[RSA 3072]----+ 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: |.***+o+ooooo | 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: |+o*oo. + +o | 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: |o. . . + o | 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: |. . . + E | 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: | . .o ..S | 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: | +.o.+ | 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: | o *.*.. | 2024-09-10T10:37:00.435 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: | o B++ | 2024-09-10T10:37:00.436 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: | =o | 2024-09-10T10:37:00.436 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: +----[SHA256]-----+ 2024-09-10T10:37:00.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.873+0000 7fa883d5a640 1 Processor -- start 2024-09-10T10:37:00.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.873+0000 7fa883d5a640 1 -- start start 2024-09-10T10:37:00.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.873+0000 7fa883d5a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c107b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:00.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.873+0000 7fa883d5a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c108140 con 0x7fa87c107770 2024-09-10T10:37:00.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.874+0000 7fa881af6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c107b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.874+0000 7fa881af6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c107b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39128/0 (socket says 172.21.15.7:39128) 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.874+0000 7fa881af6640 1 -- 172.21.15.7:0/950743508 learned_addr learned my addr 172.21.15.7:0/950743508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.874+0000 7fa881af6640 1 -- 172.21.15.7:0/950743508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c1088d0 con 0x7fa87c107770 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.875+0000 7fa881af6640 1 --2- 172.21.15.7:0/950743508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c107b70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa874009820 tx=0x7fa87402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f016e8361f768a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.875+0000 7fa880af4640 1 -- 172.21.15.7:0/950743508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa874037070 con 0x7fa87c107770 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.875+0000 7fa880af4640 1 -- 172.21.15.7:0/950743508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa874032900 con 0x7fa87c107770 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.875+0000 7fa880af4640 1 -- 172.21.15.7:0/950743508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa874030850 con 0x7fa87c107770 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.875+0000 7fa883d5a640 1 -- 172.21.15.7:0/950743508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 msgr2=0x7fa87c107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.875+0000 7fa883d5a640 1 --2- 172.21.15.7:0/950743508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c107b70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa874009820 tx=0x7fa87402a270 comp rx=0 tx=0).stop 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.876+0000 7fa883d5a640 1 -- 172.21.15.7:0/950743508 shutdown_connections 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.876+0000 7fa883d5a640 1 --2- 172.21.15.7:0/950743508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c107b70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:00.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.876+0000 7fa883d5a640 1 -- 172.21.15.7:0/950743508 >> 172.21.15.7:0/950743508 conn(0x7fa87c102f00 msgr2=0x7fa87c105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.876+0000 7fa883d5a640 1 -- 172.21.15.7:0/950743508 shutdown_connections 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.876+0000 7fa883d5a640 1 -- 172.21.15.7:0/950743508 wait complete. 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.876+0000 7fa883d5a640 1 Processor -- start 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.876+0000 7fa883d5a640 1 -- start start 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.876+0000 7fa883d5a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c07db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.876+0000 7fa883d5a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa87c10bc80 con 0x7fa87c107770 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.877+0000 7fa881af6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c07db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.877+0000 7fa881af6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c07db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39136/0 (socket says 172.21.15.7:39136) 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.877+0000 7fa881af6640 1 -- 172.21.15.7:0/3113442138 learned_addr learned my addr 172.21.15.7:0/3113442138 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.877+0000 7fa881af6640 1 -- 172.21.15.7:0/3113442138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87c07e090 con 0x7fa87c107770 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.877+0000 7fa881af6640 1 --2- 172.21.15.7:0/3113442138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c07db50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa874009940 tx=0x7fa874032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:00.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.877+0000 7fa87affd640 1 -- 172.21.15.7:0/3113442138 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa874042070 con 0x7fa87c107770 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.877+0000 7fa87affd640 1 -- 172.21.15.7:0/3113442138 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa8740308c0 con 0x7fa87c107770 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.877+0000 7fa883d5a640 1 -- 172.21.15.7:0/3113442138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87c07c280 con 0x7fa87c107770 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.877+0000 7fa87affd640 1 -- 172.21.15.7:0/3113442138 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa874037040 con 0x7fa87c107770 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.877+0000 7fa883d5a640 1 -- 172.21.15.7:0/3113442138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87c07c7c0 con 0x7fa87c107770 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.878+0000 7fa87affd640 1 -- 172.21.15.7:0/3113442138 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa874045070 con 0x7fa87c107770 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.878+0000 7fa883d5a640 1 -- 172.21.15.7:0/3113442138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa87c10be00 con 0x7fa87c107770 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.879+0000 7fa87affd640 1 --2- 172.21.15.7:0/3113442138 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fa864038fc0 0x7fa86403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.879+0000 7fa87affd640 1 -- 172.21.15.7:0/3113442138 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa87406bff0 con 0x7fa87c107770 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.879+0000 7fa8812f5640 1 --2- 172.21.15.7:0/3113442138 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fa864038fc0 0x7fa86403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.879+0000 7fa8812f5640 1 --2- 172.21.15.7:0/3113442138 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fa864038fc0 0x7fa86403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa8700098c0 tx=0x7fa870006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:00.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:36:59.884+0000 7fa87affd640 1 -- 172.21.15.7:0/3113442138 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa87404b170 con 0x7fa87c107770 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.059+0000 7fa883d5a640 1 -- 172.21.15.7:0/3113442138 --> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa87c07d050 con 0x7fa864038fc0 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.297+0000 7fa87affd640 1 -- 172.21.15.7:0/3113442138 <== mgr.14118 v2:172.21.15.7:6800/2266895831 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa87c07d050 con 0x7fa864038fc0 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.299+0000 7fa883d5a640 1 -- 172.21.15.7:0/3113442138 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fa864038fc0 msgr2=0x7fa86403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.299+0000 7fa883d5a640 1 --2- 172.21.15.7:0/3113442138 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fa864038fc0 0x7fa86403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa8700098c0 tx=0x7fa870006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.299+0000 7fa883d5a640 1 -- 172.21.15.7:0/3113442138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 msgr2=0x7fa87c07db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.299+0000 7fa883d5a640 1 --2- 172.21.15.7:0/3113442138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c07db50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa874009940 tx=0x7fa874032ce0 comp rx=0 tx=0).stop 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.299+0000 7fa883d5a640 1 -- 172.21.15.7:0/3113442138 shutdown_connections 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.299+0000 7fa883d5a640 1 --2- 172.21.15.7:0/3113442138 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fa864038fc0 0x7fa86403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.299+0000 7fa883d5a640 1 --2- 172.21.15.7:0/3113442138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa87c107770 0x7fa87c07db50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.299+0000 7fa883d5a640 1 -- 172.21.15.7:0/3113442138 >> 172.21.15.7:0/3113442138 conn(0x7fa87c102f00 msgr2=0x7fa87c103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.299+0000 7fa883d5a640 1 -- 172.21.15.7:0/3113442138 shutdown_connections 2024-09-10T10:37:00.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:00.299+0000 7fa883d5a640 1 -- 172.21.15.7:0/3113442138 wait complete. 2024-09-10T10:37:01.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:01 smithi007 ceph-mon[30873]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:01.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:01 smithi007 ceph-mon[30873]: Generating ssh key... 2024-09-10T10:37:01.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:01 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:37:01.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:01 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBAC63hyxwWCbZDFBBncXvIvACXJ9bZwRTNm7ZJo53XA+IUttcctyl1oBI8hFRQnV8USmOE/ZYIV2GeZlYm7Ayyjo3Nssd0u0EUOjrhYFcSijbkvaUcl+pPCEVTD7p1Co7UxFPldEls5URR9FBMLfDEx0Xkd6w3ozI+jURIQJIEdIjQiKpinq4Db1K3QONW4Oipcmoa/QKS0Q9txQ5P/yWu2Tx52xglLm6Nd/T7mcvuT1boec7M7Y5YOduyrWmZ2Nn1C6VEXpH/kttwOOR6aMQ7eTIlMn+Eed8NwCwMvB6VRnLVGHGnoMoqAb0Sf6MRvQsO9CQOpQ0u9vgLYZRJQj7zIc/xu5uMyU6cg9p4S3w9tRadOVpL7AN9H/3xcZf2x+MfJ0X0I/E0Lrz2pHq/vheNma4aO5fLH5S2sUMWBWBP8Sqfktio1fmhHWyFj/m0lw+bfvrGQTLvSfnr1Qsn90r9o4UlC1rrxFBcte5xOg6MBLkxtev99arvBQqgBAaWN8= ceph-65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.230+0000 7f470edb1640 1 Processor -- start 2024-09-10T10:37:02.038 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.230+0000 7f470edb1640 1 -- start start 2024-09-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.230+0000 7f470edb1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470810a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.230+0000 7f470edb1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f470807be10 con 0x7f4708108220 2024-09-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.231+0000 7f470cb4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470810a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.231+0000 7f470cb4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470810a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36798/0 (socket says 172.21.15.7:36798) 2024-09-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.231+0000 7f470cb4d640 1 -- 172.21.15.7:0/292859051 learned_addr learned my addr 172.21.15.7:0/292859051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.231+0000 7f470cb4d640 1 -- 172.21.15.7:0/292859051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470810ab50 con 0x7f4708108220 2024-09-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.231+0000 7f470cb4d640 1 --2- 172.21.15.7:0/292859051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470810a610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009820 tx=0x7f46f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fa6a6532d62318 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.232+0000 7f47077fe640 1 -- 172.21.15.7:0/292859051 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46f8037070 con 0x7f4708108220 2024-09-10T10:37:02.039 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.232+0000 7f47077fe640 1 -- 172.21.15.7:0/292859051 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f46f8032900 con 0x7f4708108220 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.232+0000 7f47077fe640 1 -- 172.21.15.7:0/292859051 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46f8030850 con 0x7f4708108220 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.232+0000 7f470edb1640 1 -- 172.21.15.7:0/292859051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 msgr2=0x7f470810a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.232+0000 7f470edb1640 1 --2- 172.21.15.7:0/292859051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470810a610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009820 tx=0x7f46f802a270 comp rx=0 tx=0).stop 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.232+0000 7f470edb1640 1 -- 172.21.15.7:0/292859051 shutdown_connections 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.232+0000 7f470edb1640 1 --2- 172.21.15.7:0/292859051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470810a610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.232+0000 7f470edb1640 1 -- 172.21.15.7:0/292859051 >> 172.21.15.7:0/292859051 conn(0x7f4708101e20 msgr2=0x7f4708104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.232+0000 7f470edb1640 1 -- 172.21.15.7:0/292859051 shutdown_connections 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.232+0000 7f470edb1640 1 -- 172.21.15.7:0/292859051 wait complete. 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.233+0000 7f470edb1640 1 Processor -- start 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.233+0000 7f470edb1640 1 -- start start 2024-09-10T10:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.233+0000 7f470edb1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.233+0000 7f470edb1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f470810bb30 con 0x7f4708108220 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.233+0000 7f470cb4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.233+0000 7f470cb4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470819b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36800/0 (socket says 172.21.15.7:36800) 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.233+0000 7f470cb4d640 1 -- 172.21.15.7:0/1644365774 learned_addr learned my addr 172.21.15.7:0/1644365774 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.234+0000 7f470cb4d640 1 -- 172.21.15.7:0/1644365774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470819bca0 con 0x7f4708108220 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.234+0000 7f470cb4d640 1 --2- 172.21.15.7:0/1644365774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470819b760 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f46f8031000 tx=0x7f46f80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.234+0000 7f4705ffb640 1 -- 172.21.15.7:0/1644365774 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46f8042070 con 0x7f4708108220 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.234+0000 7f4705ffb640 1 -- 172.21.15.7:0/1644365774 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f46f8032be0 con 0x7f4708108220 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.234+0000 7f470edb1640 1 -- 172.21.15.7:0/1644365774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470819bf30 con 0x7f4708108220 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.234+0000 7f4705ffb640 1 -- 172.21.15.7:0/1644365774 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46f8037040 con 0x7f4708108220 2024-09-10T10:37:02.041 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.234+0000 7f470edb1640 1 -- 172.21.15.7:0/1644365774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470819ec20 con 0x7f4708108220 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.235+0000 7f4705ffb640 1 -- 172.21.15.7:0/1644365774 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f46f8045070 con 0x7f4708108220 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.235+0000 7f470edb1640 1 -- 172.21.15.7:0/1644365774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4708108680 con 0x7f4708108220 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.235+0000 7f4705ffb640 1 --2- 172.21.15.7:0/1644365774 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f46f0038f70 0x7f46f003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.235+0000 7f4705ffb640 1 -- 172.21.15.7:0/1644365774 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f46f806bff0 con 0x7f4708108220 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.235+0000 7f4707fff640 1 --2- 172.21.15.7:0/1644365774 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f46f0038f70 0x7f46f003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.236+0000 7f4707fff640 1 --2- 172.21.15.7:0/1644365774 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f46f0038f70 0x7f46f003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0098c0 tx=0x7f46fc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.240+0000 7f4705ffb640 1 -- 172.21.15.7:0/1644365774 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46f8042210 con 0x7f4708108220 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.419+0000 7f470edb1640 1 -- 172.21.15.7:0/1644365774 --> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4708103170 con 0x7f46f0038f70 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.419+0000 7f4705ffb640 1 -- 172.21.15.7:0/1644365774 <== mgr.14118 v2:172.21.15.7:6800/2266895831 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4708103170 con 0x7f46f0038f70 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.421+0000 7f470edb1640 1 -- 172.21.15.7:0/1644365774 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f46f0038f70 msgr2=0x7f46f003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.421+0000 7f470edb1640 1 --2- 172.21.15.7:0/1644365774 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f46f0038f70 0x7f46f003b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0098c0 tx=0x7f46fc006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:02.042 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.421+0000 7f470edb1640 1 -- 172.21.15.7:0/1644365774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 msgr2=0x7f470819b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.421+0000 7f470edb1640 1 --2- 172.21.15.7:0/1644365774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470819b760 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f46f8031000 tx=0x7f46f80326e0 comp rx=0 tx=0).stop 2024-09-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.421+0000 7f470edb1640 1 -- 172.21.15.7:0/1644365774 shutdown_connections 2024-09-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.421+0000 7f470edb1640 1 --2- 172.21.15.7:0/1644365774 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f46f0038f70 0x7f46f003b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.421+0000 7f470edb1640 1 --2- 172.21.15.7:0/1644365774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4708108220 0x7f470819b760 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.421+0000 7f470edb1640 1 -- 172.21.15.7:0/1644365774 >> 172.21.15.7:0/1644365774 conn(0x7f4708101e20 msgr2=0x7f4708102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.421+0000 7f470edb1640 1 -- 172.21.15.7:0/1644365774 shutdown_connections 2024-09-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:01.421+0000 7f470edb1640 1 -- 172.21.15.7:0/1644365774 wait complete. 2024-09-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:37:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T10:37:02.046 INFO:teuthology.orchestra.run.smithi007.stdout:Adding host smithi007... 2024-09-10T10:37:02.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:02 smithi007 ceph-mon[30873]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:03.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:03 smithi007 ceph-mon[30873]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:04.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:04 smithi007 ceph-mon[30873]: Deploying cephadm binary to smithi007 2024-09-10T10:37:07.281 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Added host 'smithi007' with addr '172.21.15.7' 2024-09-10T10:37:07.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.509+0000 7f59f4cbf640 1 Processor -- start 2024-09-10T10:37:07.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.509+0000 7f59f4cbf640 1 -- start start 2024-09-10T10:37:07.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.509+0000 7f59f4cbf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f0107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:07.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.509+0000 7f59f4cbf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01080c0 con 0x7f59f01076f0 2024-09-10T10:37:07.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.510+0000 7f59ee59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f0107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:07.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.510+0000 7f59ee59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f0107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36806/0 (socket says 172.21.15.7:36806) 2024-09-10T10:37:07.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.510+0000 7f59ee59c640 1 -- 172.21.15.7:0/2496961692 learned_addr learned my addr 172.21.15.7:0/2496961692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:07.282 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.510+0000 7f59ee59c640 1 -- 172.21.15.7:0/2496961692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f01088f0 con 0x7f59f01076f0 2024-09-10T10:37:07.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.511+0000 7f59ee59c640 1 --2- 172.21.15.7:0/2496961692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f0107af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009820 tx=0x7f59e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7090392ed6f5504b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:07.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.511+0000 7f59ed59a640 1 -- 172.21.15.7:0/2496961692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e0037070 con 0x7f59f01076f0 2024-09-10T10:37:07.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.511+0000 7f59ed59a640 1 -- 172.21.15.7:0/2496961692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59e0032900 con 0x7f59f01076f0 2024-09-10T10:37:07.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.511+0000 7f59ed59a640 1 -- 172.21.15.7:0/2496961692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e0030850 con 0x7f59f01076f0 2024-09-10T10:37:07.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.511+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2496961692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 msgr2=0x7f59f0107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:07.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.511+0000 7f59f4cbf640 1 --2- 172.21.15.7:0/2496961692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f0107af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009820 tx=0x7f59e002a270 comp rx=0 tx=0).stop 2024-09-10T10:37:07.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.511+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2496961692 shutdown_connections 2024-09-10T10:37:07.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.511+0000 7f59f4cbf640 1 --2- 172.21.15.7:0/2496961692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f0107af0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:07.283 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.512+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2496961692 >> 172.21.15.7:0/2496961692 conn(0x7f59f0102e80 msgr2=0x7f59f01052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.512+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2496961692 shutdown_connections 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.512+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2496961692 wait complete. 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.512+0000 7f59f4cbf640 1 Processor -- start 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.512+0000 7f59f4cbf640 1 -- start start 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59f4cbf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59f4cbf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59f010bca0 con 0x7f59f01076f0 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59ee59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59ee59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36812/0 (socket says 172.21.15.7:36812) 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59ee59c640 1 -- 172.21.15.7:0/2703002002 learned_addr learned my addr 172.21.15.7:0/2703002002 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59ee59c640 1 -- 172.21.15.7:0/2703002002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f019bc80 con 0x7f59f01076f0 2024-09-10T10:37:07.284 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59ee59c640 1 --2- 172.21.15.7:0/2703002002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f019b740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009940 tx=0x7f59e0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59df7fe640 1 -- 172.21.15.7:0/2703002002 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e0042070 con 0x7f59f01076f0 2024-09-10T10:37:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2703002002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f019bf10 con 0x7f59f01076f0 2024-09-10T10:37:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59df7fe640 1 -- 172.21.15.7:0/2703002002 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59e00308c0 con 0x7f59f01076f0 2024-09-10T10:37:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.513+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2703002002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f019ec00 con 0x7f59f01076f0 2024-09-10T10:37:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.514+0000 7f59df7fe640 1 -- 172.21.15.7:0/2703002002 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e0037040 con 0x7f59f01076f0 2024-09-10T10:37:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.514+0000 7f59df7fe640 1 -- 172.21.15.7:0/2703002002 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f59e0045070 con 0x7f59f01076f0 2024-09-10T10:37:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.514+0000 7f59df7fe640 1 --2- 172.21.15.7:0/2703002002 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f59d8038f70 0x7f59d803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.514+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2703002002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f010be20 con 0x7f59f01076f0 2024-09-10T10:37:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.514+0000 7f59df7fe640 1 -- 172.21.15.7:0/2703002002 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f59e006c290 con 0x7f59f01076f0 2024-09-10T10:37:07.285 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.515+0000 7f59edd9b640 1 --2- 172.21.15.7:0/2703002002 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f59d8038f70 0x7f59d803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:07.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.515+0000 7f59edd9b640 1 --2- 172.21.15.7:0/2703002002 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f59d8038f70 0x7f59d803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59e40098c0 tx=0x7f59e4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:07.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.519+0000 7f59df7fe640 1 -- 172.21.15.7:0/2703002002 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59e00692e0 con 0x7f59f01076f0 2024-09-10T10:37:07.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:02.697+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2703002002 --> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}) -- 0x7f59f0104130 con 0x7f59d8038f70 2024-09-10T10:37:07.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.653+0000 7f59df7fe640 1 -- 172.21.15.7:0/2703002002 <== mgr.14118 v2:172.21.15.7:6800/2266895831 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f59f0104130 con 0x7f59d8038f70 2024-09-10T10:37:07.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.655+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2703002002 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f59d8038f70 msgr2=0x7f59d803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:07.286 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.655+0000 7f59f4cbf640 1 --2- 172.21.15.7:0/2703002002 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f59d8038f70 0x7f59d803b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59e40098c0 tx=0x7f59e4006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.655+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2703002002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 msgr2=0x7f59f019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.655+0000 7f59f4cbf640 1 --2- 172.21.15.7:0/2703002002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f019b740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009940 tx=0x7f59e0032ce0 comp rx=0 tx=0).stop 2024-09-10T10:37:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.655+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2703002002 shutdown_connections 2024-09-10T10:37:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.655+0000 7f59f4cbf640 1 --2- 172.21.15.7:0/2703002002 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f59d8038f70 0x7f59d803b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.655+0000 7f59f4cbf640 1 --2- 172.21.15.7:0/2703002002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59f01076f0 0x7f59f019b740 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:07.287 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.655+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2703002002 >> 172.21.15.7:0/2703002002 conn(0x7f59f0102e80 msgr2=0x7f59f0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:07.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.656+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2703002002 shutdown_connections 2024-09-10T10:37:07.288 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:06.656+0000 7f59f4cbf640 1 -- 172.21.15.7:0/2703002002 wait complete. 2024-09-10T10:37:07.288 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mon service... 2024-09-10T10:37:07.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:07 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:37:07.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:07 smithi007 ceph-mon[30873]: Added host smithi007 2024-09-10T10:37:07.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:07 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:09.178 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:08 smithi007 ceph-mon[30873]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:09.178 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:08 smithi007 ceph-mon[30873]: Saving service mon spec with placement count:5 2024-09-10T10:37:09.178 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:08 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:37:09.671 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.713+0000 7f4321929640 1 Processor -- start 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.713+0000 7f4321929640 1 -- start start 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.713+0000 7f4321929640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c10a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.713+0000 7f4321929640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f431c07bdb0 con 0x7f431c108280 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.714+0000 7f431affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c10a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.714+0000 7f431affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c10a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36824/0 (socket says 172.21.15.7:36824) 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.714+0000 7f431affd640 1 -- 172.21.15.7:0/449148855 learned_addr learned my addr 172.21.15.7:0/449148855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.714+0000 7f431affd640 1 -- 172.21.15.7:0/449148855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c10abb0 con 0x7f431c108280 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.715+0000 7f431affd640 1 --2- 172.21.15.7:0/449148855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c10a670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f430c009820 tx=0x7f430c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d639ca2b7f37263 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.715+0000 7f4319ffb640 1 -- 172.21.15.7:0/449148855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f430c037070 con 0x7f431c108280 2024-09-10T10:37:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.715+0000 7f4319ffb640 1 -- 172.21.15.7:0/449148855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f430c032900 con 0x7f431c108280 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.715+0000 7f4319ffb640 1 -- 172.21.15.7:0/449148855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f430c030850 con 0x7f431c108280 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.715+0000 7f4321929640 1 -- 172.21.15.7:0/449148855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 msgr2=0x7f431c10a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.715+0000 7f4321929640 1 --2- 172.21.15.7:0/449148855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c10a670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f430c009820 tx=0x7f430c02a270 comp rx=0 tx=0).stop 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.716+0000 7f4321929640 1 -- 172.21.15.7:0/449148855 shutdown_connections 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.716+0000 7f4321929640 1 --2- 172.21.15.7:0/449148855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c10a670 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.716+0000 7f4321929640 1 -- 172.21.15.7:0/449148855 >> 172.21.15.7:0/449148855 conn(0x7f431c101e20 msgr2=0x7f431c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.716+0000 7f4321929640 1 -- 172.21.15.7:0/449148855 shutdown_connections 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.716+0000 7f4321929640 1 -- 172.21.15.7:0/449148855 wait complete. 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.716+0000 7f4321929640 1 Processor -- start 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.716+0000 7f4321929640 1 -- start start 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f4321929640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c107b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:09.673 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f4321929640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f431c10bb30 con 0x7f431c108280 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f431affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f431affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36840/0 (socket says 172.21.15.7:36840) 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f431affd640 1 -- 172.21.15.7:0/3832387757 learned_addr learned my addr 172.21.15.7:0/3832387757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f431affd640 1 -- 172.21.15.7:0/3832387757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c108040 con 0x7f431c108280 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f431affd640 1 --2- 172.21.15.7:0/3832387757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c107b00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f430c0097f0 tx=0x7f430c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f4320927640 1 -- 172.21.15.7:0/3832387757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f430c042070 con 0x7f431c108280 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f4320927640 1 -- 172.21.15.7:0/3832387757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f430c032ca0 con 0x7f431c108280 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f4321929640 1 -- 172.21.15.7:0/3832387757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c106230 con 0x7f431c108280 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.717+0000 7f4321929640 1 -- 172.21.15.7:0/3832387757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c106770 con 0x7f431c108280 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.718+0000 7f4320927640 1 -- 172.21.15.7:0/3832387757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f430c037040 con 0x7f431c108280 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.718+0000 7f4320927640 1 -- 172.21.15.7:0/3832387757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f430c045070 con 0x7f431c108280 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.718+0000 7f4321929640 1 -- 172.21.15.7:0/3832387757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431c1086e0 con 0x7f431c108280 2024-09-10T10:37:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.719+0000 7f4320927640 1 --2- 172.21.15.7:0/3832387757 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4304038f70 0x7f430403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.719+0000 7f4320927640 1 -- 172.21.15.7:0/3832387757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f430c06c010 con 0x7f431c108280 2024-09-10T10:37:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.719+0000 7f431a7fc640 1 --2- 172.21.15.7:0/3832387757 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4304038f70 0x7f430403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.719+0000 7f431a7fc640 1 --2- 172.21.15.7:0/3832387757 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4304038f70 0x7f430403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43100098c0 tx=0x7f4310006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.723+0000 7f4320927640 1 -- 172.21.15.7:0/3832387757 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f430c045520 con 0x7f431c108280 2024-09-10T10:37:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.910+0000 7f4321929640 1 -- 172.21.15.7:0/3832387757 --> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f431c107100 con 0x7f4304038f70 2024-09-10T10:37:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.915+0000 7f4320927640 1 -- 172.21.15.7:0/3832387757 <== mgr.14118 v2:172.21.15.7:6800/2266895831 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f431c107100 con 0x7f4304038f70 2024-09-10T10:37:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.917+0000 7f4321929640 1 -- 172.21.15.7:0/3832387757 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4304038f70 msgr2=0x7f430403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.917+0000 7f4321929640 1 --2- 172.21.15.7:0/3832387757 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4304038f70 0x7f430403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43100098c0 tx=0x7f4310006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.917+0000 7f4321929640 1 -- 172.21.15.7:0/3832387757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 msgr2=0x7f431c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:09.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.917+0000 7f4321929640 1 --2- 172.21.15.7:0/3832387757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c107b00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f430c0097f0 tx=0x7f430c030ec0 comp rx=0 tx=0).stop 2024-09-10T10:37:09.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.917+0000 7f4321929640 1 -- 172.21.15.7:0/3832387757 shutdown_connections 2024-09-10T10:37:09.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.917+0000 7f4321929640 1 --2- 172.21.15.7:0/3832387757 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4304038f70 0x7f430403b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:09.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.917+0000 7f4321929640 1 --2- 172.21.15.7:0/3832387757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431c108280 0x7f431c107b00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:09.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.917+0000 7f4321929640 1 -- 172.21.15.7:0/3832387757 >> 172.21.15.7:0/3832387757 conn(0x7f431c101e20 msgr2=0x7f431c102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:09.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.917+0000 7f4321929640 1 -- 172.21.15.7:0/3832387757 shutdown_connections 2024-09-10T10:37:09.676 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:07.917+0000 7f4321929640 1 -- 172.21.15.7:0/3832387757 wait complete. 2024-09-10T10:37:09.676 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mgr service... 2024-09-10T10:37:10.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:10 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:37:10.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:10 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:37:11.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:11 smithi007 ceph-mon[30873]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:11.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:11 smithi007 ceph-mon[30873]: Saving service mgr spec with placement count:2 2024-09-10T10:37:11.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:11 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:37:11.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:11 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:37:11.599 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T10:37:11.599 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.106+0000 7fe9c528e640 1 Processor -- start 2024-09-10T10:37:11.599 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.106+0000 7fe9c528e640 1 -- start start 2024-09-10T10:37:11.599 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.106+0000 7fe9c528e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c0107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:11.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.106+0000 7fe9c528e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01080c0 con 0x7fe9c01076f0 2024-09-10T10:37:11.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.106+0000 7fe9beffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c0107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:11.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.106+0000 7fe9beffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c0107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36842/0 (socket says 172.21.15.7:36842) 2024-09-10T10:37:11.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.106+0000 7fe9beffd640 1 -- 172.21.15.7:0/1418909765 learned_addr learned my addr 172.21.15.7:0/1418909765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:11.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.106+0000 7fe9beffd640 1 -- 172.21.15.7:0/1418909765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c01088f0 con 0x7fe9c01076f0 2024-09-10T10:37:11.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.107+0000 7fe9beffd640 1 --2- 172.21.15.7:0/1418909765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c0107af0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009820 tx=0x7fe9b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b9bf7c88b1cf51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:11.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.107+0000 7fe9be7fc640 1 -- 172.21.15.7:0/1418909765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9b0037070 con 0x7fe9c01076f0 2024-09-10T10:37:11.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.107+0000 7fe9be7fc640 1 -- 172.21.15.7:0/1418909765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b0032920 con 0x7fe9c01076f0 2024-09-10T10:37:11.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.107+0000 7fe9be7fc640 1 -- 172.21.15.7:0/1418909765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9b0030810 con 0x7fe9c01076f0 2024-09-10T10:37:11.600 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.108+0000 7fe9c528e640 1 -- 172.21.15.7:0/1418909765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 msgr2=0x7fe9c0107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.108+0000 7fe9c528e640 1 --2- 172.21.15.7:0/1418909765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c0107af0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009820 tx=0x7fe9b002a270 comp rx=0 tx=0).stop 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.108+0000 7fe9c528e640 1 -- 172.21.15.7:0/1418909765 shutdown_connections 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.108+0000 7fe9c528e640 1 --2- 172.21.15.7:0/1418909765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c0107af0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.108+0000 7fe9c528e640 1 -- 172.21.15.7:0/1418909765 >> 172.21.15.7:0/1418909765 conn(0x7fe9c0102e80 msgr2=0x7fe9c01052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.108+0000 7fe9c528e640 1 -- 172.21.15.7:0/1418909765 shutdown_connections 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.108+0000 7fe9c528e640 1 -- 172.21.15.7:0/1418909765 wait complete. 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.109+0000 7fe9c528e640 1 Processor -- start 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.109+0000 7fe9c528e640 1 -- start start 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.109+0000 7fe9c528e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c007db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.109+0000 7fe9c528e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c010bca0 con 0x7fe9c01076f0 2024-09-10T10:37:11.601 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.109+0000 7fe9beffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.109+0000 7fe9beffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36850/0 (socket says 172.21.15.7:36850) 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.109+0000 7fe9beffd640 1 -- 172.21.15.7:0/3516060464 learned_addr learned my addr 172.21.15.7:0/3516060464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.109+0000 7fe9beffd640 1 -- 172.21.15.7:0/3516060464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c007e090 con 0x7fe9c01076f0 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.110+0000 7fe9beffd640 1 --2- 172.21.15.7:0/3516060464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c007db50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0030560 tx=0x7fe9b0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.110+0000 7fe9bcff9640 1 -- 172.21.15.7:0/3516060464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9b0037040 con 0x7fe9c01076f0 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.110+0000 7fe9bcff9640 1 -- 172.21.15.7:0/3516060464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b0040070 con 0x7fe9c01076f0 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.110+0000 7fe9c528e640 1 -- 172.21.15.7:0/3516060464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c007c280 con 0x7fe9c01076f0 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.110+0000 7fe9bcff9640 1 -- 172.21.15.7:0/3516060464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9b003a740 con 0x7fe9c01076f0 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.110+0000 7fe9c528e640 1 -- 172.21.15.7:0/3516060464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c007c7c0 con 0x7fe9c01076f0 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.111+0000 7fe9bcff9640 1 -- 172.21.15.7:0/3516060464 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe9b003a8e0 con 0x7fe9c01076f0 2024-09-10T10:37:11.602 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.111+0000 7fe9c528e640 1 -- 172.21.15.7:0/3516060464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c010be20 con 0x7fe9c01076f0 2024-09-10T10:37:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.111+0000 7fe9bcff9640 1 --2- 172.21.15.7:0/3516060464 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fe9ac038fc0 0x7fe9ac03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.111+0000 7fe9bcff9640 1 -- 172.21.15.7:0/3516060464 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe9b006c660 con 0x7fe9c01076f0 2024-09-10T10:37:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.111+0000 7fe9b7fff640 1 --2- 172.21.15.7:0/3516060464 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fe9ac038fc0 0x7fe9ac03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.112+0000 7fe9b7fff640 1 --2- 172.21.15.7:0/3516060464 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fe9ac038fc0 0x7fe9ac03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80098c0 tx=0x7fe9a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.116+0000 7fe9bcff9640 1 -- 172.21.15.7:0/3516060464 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9b0044c70 con 0x7fe9c01076f0 2024-09-10T10:37:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.316+0000 7fe9c528e640 1 -- 172.21.15.7:0/3516060464 --> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe9c007cfc0 con 0x7fe9ac038fc0 2024-09-10T10:37:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.320+0000 7fe9bcff9640 1 -- 172.21.15.7:0/3516060464 <== mgr.14118 v2:172.21.15.7:6800/2266895831 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe9c007cfc0 con 0x7fe9ac038fc0 2024-09-10T10:37:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.323+0000 7fe9c528e640 1 -- 172.21.15.7:0/3516060464 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fe9ac038fc0 msgr2=0x7fe9ac03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.323+0000 7fe9c528e640 1 --2- 172.21.15.7:0/3516060464 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fe9ac038fc0 0x7fe9ac03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80098c0 tx=0x7fe9a8006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:11.603 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.323+0000 7fe9c528e640 1 -- 172.21.15.7:0/3516060464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 msgr2=0x7fe9c007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.323+0000 7fe9c528e640 1 --2- 172.21.15.7:0/3516060464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c007db50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0030560 tx=0x7fe9b0032670 comp rx=0 tx=0).stop 2024-09-10T10:37:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.323+0000 7fe9c528e640 1 -- 172.21.15.7:0/3516060464 shutdown_connections 2024-09-10T10:37:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.323+0000 7fe9c528e640 1 --2- 172.21.15.7:0/3516060464 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7fe9ac038fc0 0x7fe9ac03b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.323+0000 7fe9c528e640 1 --2- 172.21.15.7:0/3516060464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9c01076f0 0x7fe9c007db50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.323+0000 7fe9c528e640 1 -- 172.21.15.7:0/3516060464 >> 172.21.15.7:0/3516060464 conn(0x7fe9c0102e80 msgr2=0x7fe9c0103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.323+0000 7fe9c528e640 1 -- 172.21.15.7:0/3516060464 shutdown_connections 2024-09-10T10:37:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:10.323+0000 7fe9c528e640 1 -- 172.21.15.7:0/3516060464 wait complete. 2024-09-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.008+0000 7f444e18b640 1 Processor -- start 2024-09-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.008+0000 7f444e18b640 1 -- start start 2024-09-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.009+0000 7f444e18b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.009+0000 7f444e18b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f444807c8b0 con 0x7f444807de80 2024-09-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.009+0000 7f44477fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.009+0000 7f44477fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59586/0 (socket says 172.21.15.7:59586) 2024-09-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.009+0000 7f44477fe640 1 -- 172.21.15.7:0/3065544985 learned_addr learned my addr 172.21.15.7:0/3065544985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.009+0000 7f44477fe640 1 -- 172.21.15.7:0/3065544985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444807ca30 con 0x7f444807de80 2024-09-10T10:37:12.839 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.010+0000 7f44477fe640 1 --2- 172.21.15.7:0/3065544985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444807c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4438009e80 tx=0x7f443802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8818a8eff857f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.010+0000 7f44467fc640 1 -- 172.21.15.7:0/3065544985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4438036070 con 0x7f444807de80 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.010+0000 7f44467fc640 1 -- 172.21.15.7:0/3065544985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44380318e0 con 0x7f444807de80 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.010+0000 7f44467fc640 1 -- 172.21.15.7:0/3065544985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4438007a00 con 0x7f444807de80 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.011+0000 7f444e18b640 1 -- 172.21.15.7:0/3065544985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 msgr2=0x7f444807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.011+0000 7f444e18b640 1 --2- 172.21.15.7:0/3065544985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444807c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4438009e80 tx=0x7f443802a7a0 comp rx=0 tx=0).stop 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.011+0000 7f444e18b640 1 -- 172.21.15.7:0/3065544985 shutdown_connections 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.011+0000 7f444e18b640 1 --2- 172.21.15.7:0/3065544985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444807c2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.011+0000 7f444e18b640 1 -- 172.21.15.7:0/3065544985 >> 172.21.15.7:0/3065544985 conn(0x7f4448102e80 msgr2=0x7f44481052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.011+0000 7f444e18b640 1 -- 172.21.15.7:0/3065544985 shutdown_connections 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.011+0000 7f444e18b640 1 -- 172.21.15.7:0/3065544985 wait complete. 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.011+0000 7f444e18b640 1 Processor -- start 2024-09-10T10:37:12.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.011+0000 7f444e18b640 1 -- start start 2024-09-10T10:37:12.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f444e18b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:12.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f444e18b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f444810ddc0 con 0x7f444807de80 2024-09-10T10:37:12.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f44477fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:12.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f44477fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444819d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59598/0 (socket says 172.21.15.7:59598) 2024-09-10T10:37:12.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f44477fe640 1 -- 172.21.15.7:0/3876328353 learned_addr learned my addr 172.21.15.7:0/3876328353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:12.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f44477fe640 1 -- 172.21.15.7:0/3876328353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444819de50 con 0x7f444807de80 2024-09-10T10:37:12.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f44477fe640 1 --2- 172.21.15.7:0/3876328353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444819d910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f443802acc0 tx=0x7f4438031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:12.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f4444ff9640 1 -- 172.21.15.7:0/3876328353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4438036050 con 0x7f444807de80 2024-09-10T10:37:12.841 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f444e18b640 1 -- 172.21.15.7:0/3876328353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444819e0e0 con 0x7f444807de80 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f4444ff9640 1 -- 172.21.15.7:0/3876328353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4438037040 con 0x7f444807de80 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f4444ff9640 1 -- 172.21.15.7:0/3876328353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f443803d8e0 con 0x7f444807de80 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.012+0000 7f444e18b640 1 -- 172.21.15.7:0/3876328353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44481a0dd0 con 0x7f444807de80 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.013+0000 7f4444ff9640 1 -- 172.21.15.7:0/3876328353 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4438044020 con 0x7f444807de80 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.014+0000 7f4444ff9640 1 --2- 172.21.15.7:0/3876328353 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4430038fc0 0x7f443003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.014+0000 7f444e18b640 1 -- 172.21.15.7:0/3876328353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444807c360 con 0x7f444807de80 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.014+0000 7f4444ff9640 1 -- 172.21.15.7:0/3876328353 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4438042020 con 0x7f444807de80 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.014+0000 7f4446ffd640 1 --2- 172.21.15.7:0/3876328353 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4430038fc0 0x7f443003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.015+0000 7f4446ffd640 1 --2- 172.21.15.7:0/3876328353 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4430038fc0 0x7f443003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f443c0098c0 tx=0x7f443c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.019+0000 7f4444ff9640 1 -- 172.21.15.7:0/3876328353 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f443804b020 con 0x7f444807de80 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.198+0000 7f444e18b640 1 -- 172.21.15.7:0/3876328353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f44481a13c0 con 0x7f444807de80 2024-09-10T10:37:12.842 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.200+0000 7f4444ff9640 1 -- 172.21.15.7:0/3876328353 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f4438031e00 con 0x7f444807de80 2024-09-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.203+0000 7f444e18b640 1 -- 172.21.15.7:0/3876328353 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4430038fc0 msgr2=0x7f443003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.203+0000 7f444e18b640 1 --2- 172.21.15.7:0/3876328353 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4430038fc0 0x7f443003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f443c0098c0 tx=0x7f443c006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.203+0000 7f444e18b640 1 -- 172.21.15.7:0/3876328353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 msgr2=0x7f444819d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.203+0000 7f444e18b640 1 --2- 172.21.15.7:0/3876328353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444819d910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f443802acc0 tx=0x7f4438031cc0 comp rx=0 tx=0).stop 2024-09-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.203+0000 7f444e18b640 1 -- 172.21.15.7:0/3876328353 shutdown_connections 2024-09-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.203+0000 7f444e18b640 1 --2- 172.21.15.7:0/3876328353 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f4430038fc0 0x7f443003b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.203+0000 7f444e18b640 1 --2- 172.21.15.7:0/3876328353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444807de80 0x7f444819d910 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.203+0000 7f444e18b640 1 -- 172.21.15.7:0/3876328353 >> 172.21.15.7:0/3876328353 conn(0x7f4448102e80 msgr2=0x7f4448103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.203+0000 7f444e18b640 1 -- 172.21.15.7:0/3876328353 shutdown_connections 2024-09-10T10:37:12.843 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:12.203+0000 7f444e18b640 1 -- 172.21.15.7:0/3876328353 wait complete. 2024-09-10T10:37:13.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:13 smithi007 ceph-mon[30873]: from='mgr.14118 172.21.15.7:0/702265513' entity='mgr.x' 2024-09-10T10:37:13.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:13 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3876328353' entity='client.admin' 2024-09-10T10:37:14.123 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.251+0000 7f5bdebfb640 1 Processor -- start 2024-09-10T10:37:14.123 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.251+0000 7f5bdebfb640 1 -- start start 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.251+0000 7f5bdebfb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd8106410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.251+0000 7f5bdebfb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd807be10 con 0x7f5bd8106010 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.251+0000 7f5bdc997640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd8106410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.251+0000 7f5bdc997640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd8106410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59612/0 (socket says 172.21.15.7:59612) 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.251+0000 7f5bdc997640 1 -- 172.21.15.7:0/3386022799 learned_addr learned my addr 172.21.15.7:0/3386022799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.252+0000 7f5bdc997640 1 -- 172.21.15.7:0/3386022799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd8106950 con 0x7f5bd8106010 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.253+0000 7f5bdc997640 1 --2- 172.21.15.7:0/3386022799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd8106410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009820 tx=0x7f5bcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb398696ae834bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.253+0000 7f5bd77fe640 1 -- 172.21.15.7:0/3386022799 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bcc037070 con 0x7f5bd8106010 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.253+0000 7f5bd77fe640 1 -- 172.21.15.7:0/3386022799 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bcc032920 con 0x7f5bd8106010 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.253+0000 7f5bd77fe640 1 -- 172.21.15.7:0/3386022799 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bcc030810 con 0x7f5bd8106010 2024-09-10T10:37:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.253+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3386022799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 msgr2=0x7f5bd8106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.254+0000 7f5bdebfb640 1 --2- 172.21.15.7:0/3386022799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd8106410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009820 tx=0x7f5bcc02a270 comp rx=0 tx=0).stop 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.254+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3386022799 shutdown_connections 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.254+0000 7f5bdebfb640 1 --2- 172.21.15.7:0/3386022799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd8106410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.254+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3386022799 >> 172.21.15.7:0/3386022799 conn(0x7f5bd8101e20 msgr2=0x7f5bd8104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.254+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3386022799 shutdown_connections 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.254+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3386022799 wait complete. 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.254+0000 7f5bdebfb640 1 Processor -- start 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.254+0000 7f5bdebfb640 1 -- start start 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bdebfb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bdebfb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd8107a90 con 0x7f5bd8106010 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bdc997640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bdc997640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd819d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59624/0 (socket says 172.21.15.7:59624) 2024-09-10T10:37:14.125 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bdc997640 1 -- 172.21.15.7:0/3621655346 learned_addr learned my addr 172.21.15.7:0/3621655346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bdc997640 1 -- 172.21.15.7:0/3621655346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd819dee0 con 0x7f5bd8106010 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bdc997640 1 --2- 172.21.15.7:0/3621655346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd819d9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc030420 tx=0x7f5bcc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bd5ffb640 1 -- 172.21.15.7:0/3621655346 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bcc037040 con 0x7f5bd8106010 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3621655346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd819e170 con 0x7f5bd8106010 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bd5ffb640 1 -- 172.21.15.7:0/3621655346 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bcc040070 con 0x7f5bd8106010 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.255+0000 7f5bd5ffb640 1 -- 172.21.15.7:0/3621655346 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bcc03a6e0 con 0x7f5bd8106010 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.256+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3621655346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd81a0e60 con 0x7f5bd8106010 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.257+0000 7f5bd5ffb640 1 -- 172.21.15.7:0/3621655346 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5bcc03a990 con 0x7f5bd8106010 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.257+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3621655346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd8106410 con 0x7f5bd8106010 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.257+0000 7f5bd5ffb640 1 --2- 172.21.15.7:0/3621655346 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5bc8038fc0 0x7f5bc803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.257+0000 7f5bd5ffb640 1 -- 172.21.15.7:0/3621655346 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5bcc06c630 con 0x7f5bd8106010 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.257+0000 7f5bd7fff640 1 --2- 172.21.15.7:0/3621655346 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5bc8038fc0 0x7f5bc803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.258+0000 7f5bd7fff640 1 --2- 172.21.15.7:0/3621655346 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5bc8038fc0 0x7f5bc803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40098c0 tx=0x7f5bc4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:14.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.262+0000 7f5bd5ffb640 1 -- 172.21.15.7:0/3621655346 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5bcc044c70 con 0x7f5bd8106010 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.440+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3621655346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f5bd802da10 con 0x7f5bd8106010 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.443+0000 7f5bd5ffb640 1 -- 172.21.15.7:0/3621655346 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f5bcc042e30 con 0x7f5bd8106010 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.445+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3621655346 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5bc8038fc0 msgr2=0x7f5bc803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.445+0000 7f5bdebfb640 1 --2- 172.21.15.7:0/3621655346 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5bc8038fc0 0x7f5bc803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40098c0 tx=0x7f5bc4006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.445+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3621655346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 msgr2=0x7f5bd819d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.445+0000 7f5bdebfb640 1 --2- 172.21.15.7:0/3621655346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd819d9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc030420 tx=0x7f5bcc032670 comp rx=0 tx=0).stop 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.445+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3621655346 shutdown_connections 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.445+0000 7f5bdebfb640 1 --2- 172.21.15.7:0/3621655346 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f5bc8038fc0 0x7f5bc803b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.445+0000 7f5bdebfb640 1 --2- 172.21.15.7:0/3621655346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bd8106010 0x7f5bd819d9a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.445+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3621655346 >> 172.21.15.7:0/3621655346 conn(0x7f5bd8101e20 msgr2=0x7f5bd8194600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.445+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3621655346 shutdown_connections 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:13.445+0000 7f5bdebfb640 1 -- 172.21.15.7:0/3621655346 wait complete. 2024-09-10T10:37:14.127 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling the dashboard module... 2024-09-10T10:37:14.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:14 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3621655346' entity='client.admin' 2024-09-10T10:37:15.449 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:15 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1807314876' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T10:37:15.785 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:15 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: ignoring --setuser ceph since I am not root 2024-09-10T10:37:15.785 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:15 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: ignoring --setgroup ceph since I am not root 2024-09-10T10:37:15.786 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:15 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:15.610+0000 7fe09fb27040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:37:15.786 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:15 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:15.671+0000 7fe09fb27040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:37:16.056 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:15 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:15.829+0000 7fe09fb27040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:37:16.056 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:15 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:15.900+0000 7fe09fb27040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:37:16.057 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:16 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:16.055+0000 7fe09fb27040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:37:16.105 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.508+0000 7f1ebf0a2640 1 Processor -- start 2024-09-10T10:37:16.105 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.508+0000 7f1ebf0a2640 1 -- start start 2024-09-10T10:37:16.105 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.509+0000 7f1ebf0a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8105dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:16.105 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.509+0000 7f1ebf0a2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb8106390 con 0x7f1eb81059c0 2024-09-10T10:37:16.105 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.509+0000 7f1ebce3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8105dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.509+0000 7f1ebce3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8105dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59628/0 (socket says 172.21.15.7:59628) 2024-09-10T10:37:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.509+0000 7f1ebce3e640 1 -- 172.21.15.7:0/2947735950 learned_addr learned my addr 172.21.15.7:0/2947735950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.509+0000 7f1ebce3e640 1 -- 172.21.15.7:0/2947735950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb81089a0 con 0x7f1eb81059c0 2024-09-10T10:37:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.510+0000 7f1ebce3e640 1 --2- 172.21.15.7:0/2947735950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8105dc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009820 tx=0x7f1eac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8c518478470fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.510+0000 7f1eb77fe640 1 -- 172.21.15.7:0/2947735950 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eac037070 con 0x7f1eb81059c0 2024-09-10T10:37:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.510+0000 7f1eb77fe640 1 -- 172.21.15.7:0/2947735950 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac032920 con 0x7f1eb81059c0 2024-09-10T10:37:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.510+0000 7f1eb77fe640 1 -- 172.21.15.7:0/2947735950 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eac030810 con 0x7f1eb81059c0 2024-09-10T10:37:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.511+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/2947735950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 msgr2=0x7f1eb8105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:16.106 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.511+0000 7f1ebf0a2640 1 --2- 172.21.15.7:0/2947735950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8105dc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009820 tx=0x7f1eac02a270 comp rx=0 tx=0).stop 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.511+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/2947735950 shutdown_connections 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.511+0000 7f1ebf0a2640 1 --2- 172.21.15.7:0/2947735950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8105dc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.511+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/2947735950 >> 172.21.15.7:0/2947735950 conn(0x7f1eb8101530 msgr2=0x7f1eb8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.511+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/2947735950 shutdown_connections 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.511+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/2947735950 wait complete. 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.511+0000 7f1ebf0a2640 1 Processor -- start 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.511+0000 7f1ebf0a2640 1 -- start start 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1ebf0a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8108220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1ebf0a2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb810bbd0 con 0x7f1eb81059c0 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1ebce3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1ebce3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8108220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59642/0 (socket says 172.21.15.7:59642) 2024-09-10T10:37:16.107 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1ebce3e640 1 -- 172.21.15.7:0/1807314876 learned_addr learned my addr 172.21.15.7:0/1807314876 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1ebce3e640 1 -- 172.21.15.7:0/1807314876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb8108760 con 0x7f1eb81059c0 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1ebce3e640 1 --2- 172.21.15.7:0/1807314876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8108220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1eac030560 tx=0x7f1eac032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1eb5ffb640 1 -- 172.21.15.7:0/1807314876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eac037040 con 0x7f1eb81059c0 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1eb5ffb640 1 -- 172.21.15.7:0/1807314876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac040070 con 0x7f1eb81059c0 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/1807314876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb8106950 con 0x7f1eb81059c0 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1eb5ffb640 1 -- 172.21.15.7:0/1807314876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1eac03a740 con 0x7f1eb81059c0 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.512+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/1807314876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb8106e90 con 0x7f1eb81059c0 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.513+0000 7f1eb5ffb640 1 -- 172.21.15.7:0/1807314876 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1eac03bb90 con 0x7f1eb81059c0 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.513+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/1807314876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb8105e40 con 0x7f1eb81059c0 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.513+0000 7f1eb5ffb640 1 --2- 172.21.15.7:0/1807314876 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f1ea8038fc0 0x7f1ea803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.514+0000 7f1eb5ffb640 1 -- 172.21.15.7:0/1807314876 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1eac06c600 con 0x7f1eb81059c0 2024-09-10T10:37:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.514+0000 7f1eb7fff640 1 --2- 172.21.15.7:0/1807314876 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f1ea8038fc0 0x7f1ea803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.514+0000 7f1eb7fff640 1 --2- 172.21.15.7:0/1807314876 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f1ea8038fc0 0x7f1ea803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40098c0 tx=0x7f1ea4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.518+0000 7f1eb5ffb640 1 -- 172.21.15.7:0/1807314876 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1eac04a200 con 0x7f1eb81059c0 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:14.748+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/1807314876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f1eb8107770 con 0x7f1eb81059c0 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.445+0000 7f1eb5ffb640 1 -- 172.21.15.7:0/1807314876 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f1eac031e80 con 0x7f1eb81059c0 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.446+0000 7f1eb5ffb640 1 -- 172.21.15.7:0/1807314876 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f1eac03a8e0 con 0x7f1eb81059c0 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.449+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/1807314876 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f1ea8038fc0 msgr2=0x7f1ea803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.449+0000 7f1ebf0a2640 1 --2- 172.21.15.7:0/1807314876 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f1ea8038fc0 0x7f1ea803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40098c0 tx=0x7f1ea4006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.449+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/1807314876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 msgr2=0x7f1eb8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.449+0000 7f1ebf0a2640 1 --2- 172.21.15.7:0/1807314876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8108220 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1eac030560 tx=0x7f1eac032670 comp rx=0 tx=0).stop 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.449+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/1807314876 shutdown_connections 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.449+0000 7f1ebf0a2640 1 --2- 172.21.15.7:0/1807314876 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f1ea8038fc0 0x7f1ea803b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:16.109 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.449+0000 7f1ebf0a2640 1 --2- 172.21.15.7:0/1807314876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb81059c0 0x7f1eb8108220 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:16.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.449+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/1807314876 >> 172.21.15.7:0/1807314876 conn(0x7f1eb8101530 msgr2=0x7f1eb81020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:16.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.449+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/1807314876 shutdown_connections 2024-09-10T10:37:16.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:15.449+0000 7f1ebf0a2640 1 -- 172.21.15.7:0/1807314876 wait complete. 2024-09-10T10:37:16.315 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:16 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:16.207+0000 7fe09fb27040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:37:16.615 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:16 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1807314876' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T10:37:16.615 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:16 smithi007 ceph-mon[30873]: mgrmap e9: x(active, since 20s) 2024-09-10T10:37:16.616 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:16 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:16.549+0000 7fe09fb27040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:37:16.616 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:16 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:16.614+0000 7fe09fb27040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:37:16.933 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:16 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:16.814+0000 7fe09fb27040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:37:17.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:37:17.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T10:37:17.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T10:37:17.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T10:37:17.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T10:37:17.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:37:17.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.485+0000 7f3674134640 1 Processor -- start 2024-09-10T10:37:17.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.485+0000 7f3674134640 1 -- start start 2024-09-10T10:37:17.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.486+0000 7f3674134640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c106410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:17.293 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.486+0000 7f3674134640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f366c07be10 con 0x7f366c106010 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.486+0000 7f3671ed0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c106410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.486+0000 7f3671ed0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c106410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59674/0 (socket says 172.21.15.7:59674) 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.486+0000 7f3671ed0640 1 -- 172.21.15.7:0/3464478647 learned_addr learned my addr 172.21.15.7:0/3464478647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.486+0000 7f3671ed0640 1 -- 172.21.15.7:0/3464478647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c106950 con 0x7f366c106010 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.487+0000 7f3671ed0640 1 --2- 172.21.15.7:0/3464478647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c106410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3668009820 tx=0x7f366802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38ce54c6013e783 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.487+0000 7f3670ece640 1 -- 172.21.15.7:0/3464478647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668037070 con 0x7f366c106010 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.487+0000 7f3670ece640 1 -- 172.21.15.7:0/3464478647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3668032920 con 0x7f366c106010 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.487+0000 7f3670ece640 1 -- 172.21.15.7:0/3464478647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668030810 con 0x7f366c106010 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.488+0000 7f3674134640 1 -- 172.21.15.7:0/3464478647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 msgr2=0x7f366c106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.488+0000 7f3674134640 1 --2- 172.21.15.7:0/3464478647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c106410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3668009820 tx=0x7f366802a270 comp rx=0 tx=0).stop 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.488+0000 7f3674134640 1 -- 172.21.15.7:0/3464478647 shutdown_connections 2024-09-10T10:37:17.294 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.488+0000 7f3674134640 1 --2- 172.21.15.7:0/3464478647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c106410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.488+0000 7f3674134640 1 -- 172.21.15.7:0/3464478647 >> 172.21.15.7:0/3464478647 conn(0x7f366c101e20 msgr2=0x7f366c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.488+0000 7f3674134640 1 -- 172.21.15.7:0/3464478647 shutdown_connections 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.488+0000 7f3674134640 1 -- 172.21.15.7:0/3464478647 wait complete. 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.489+0000 7f3674134640 1 Processor -- start 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.489+0000 7f3674134640 1 -- start start 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.489+0000 7f3674134640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.489+0000 7f3674134640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f366c107a90 con 0x7f366c106010 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.489+0000 7f3671ed0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.489+0000 7f3671ed0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59680/0 (socket says 172.21.15.7:59680) 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.489+0000 7f3671ed0640 1 -- 172.21.15.7:0/4230903467 learned_addr learned my addr 172.21.15.7:0/4230903467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.489+0000 7f3671ed0640 1 -- 172.21.15.7:0/4230903467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c19dee0 con 0x7f366c106010 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.489+0000 7f3671ed0640 1 --2- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c19d9a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3668030420 tx=0x7f3668032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:17.295 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.490+0000 7f3662ffd640 1 -- 172.21.15.7:0/4230903467 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668037040 con 0x7f366c106010 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.490+0000 7f3662ffd640 1 -- 172.21.15.7:0/4230903467 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3668040070 con 0x7f366c106010 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.490+0000 7f3662ffd640 1 -- 172.21.15.7:0/4230903467 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f366803a6e0 con 0x7f366c106010 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.490+0000 7f3674134640 1 -- 172.21.15.7:0/4230903467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c19e170 con 0x7f366c106010 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.490+0000 7f3674134640 1 -- 172.21.15.7:0/4230903467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1a0e60 con 0x7f366c106010 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.491+0000 7f3662ffd640 1 -- 172.21.15.7:0/4230903467 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f366803a990 con 0x7f366c106010 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.491+0000 7f3662ffd640 1 --2- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f3658039020 0x7f365803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.491+0000 7f3662ffd640 1 -- 172.21.15.7:0/4230903467 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f366806c5e0 con 0x7f366c106010 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.491+0000 7f36716cf640 1 -- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f3658039020 msgr2=0x7f365803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2266895831 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.491+0000 7f36716cf640 1 --2- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f3658039020 0x7f365803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.491+0000 7f3674134640 1 -- 172.21.15.7:0/4230903467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c106410 con 0x7f366c106010 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.496+0000 7f3662ffd640 1 -- 172.21.15.7:0/4230903467 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f366803b420 con 0x7f366c106010 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.692+0000 7f36716cf640 1 -- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f3658039020 msgr2=0x7f365803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2266895831 2024-09-10T10:37:17.296 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.692+0000 7f36716cf640 1 --2- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f3658039020 0x7f365803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.719+0000 7f3674134640 1 -- 172.21.15.7:0/4230903467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f366c02d940 con 0x7f366c106010 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.720+0000 7f3662ffd640 1 -- 172.21.15.7:0/4230903467 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f3668041270 con 0x7f366c106010 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.722+0000 7f3674134640 1 -- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f3658039020 msgr2=0x7f365803b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.722+0000 7f3674134640 1 --2- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f3658039020 0x7f365803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.722+0000 7f3674134640 1 -- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 msgr2=0x7f366c19d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.722+0000 7f3674134640 1 --2- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c19d9a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3668030420 tx=0x7f3668032670 comp rx=0 tx=0).stop 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.722+0000 7f3674134640 1 -- 172.21.15.7:0/4230903467 shutdown_connections 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.722+0000 7f3674134640 1 --2- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f3658039020 0x7f365803b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.722+0000 7f3674134640 1 --2- 172.21.15.7:0/4230903467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f366c106010 0x7f366c19d9a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.722+0000 7f3674134640 1 -- 172.21.15.7:0/4230903467 >> 172.21.15.7:0/4230903467 conn(0x7f366c101e20 msgr2=0x7f366c194600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.722+0000 7f3674134640 1 -- 172.21.15.7:0/4230903467 shutdown_connections 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:16.722+0000 7f3674134640 1 -- 172.21.15.7:0/4230903467 wait complete. 2024-09-10T10:37:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-09-10T10:37:17.298 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 9... 2024-09-10T10:37:17.528 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:17 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4230903467' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T10:37:17.836 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:17 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:17.763+0000 7fe09fb27040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:37:17.836 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:17 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:17.834+0000 7fe09fb27040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:37:18.874 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:18 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:18.600+0000 7fe09fb27040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:37:18.874 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:18 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:18.722+0000 7fe09fb27040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:37:19.128 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:18 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:18.872+0000 7fe09fb27040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:37:19.128 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:18 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:18.937+0000 7fe09fb27040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:37:19.128 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:19 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:19.094+0000 7fe09fb27040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:37:19.399 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:19 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:19.166+0000 7fe09fb27040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:37:19.399 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:19 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:19.230+0000 7fe09fb27040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:37:19.399 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:19 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:19.397+0000 7fe09fb27040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:37:19.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:19 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:19.466+0000 7fe09fb27040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:37:19.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:19 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:19.537+0000 7fe09fb27040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:37:19.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:19 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:19.624+0000 7fe09fb27040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:37:20.183 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:19 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:19.687+0000 7fe09fb27040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:37:20.184 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:19 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:19.749+0000 7fe09fb27040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:37:20.842 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:20.586+0000 7fe09fb27040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:37:20.842 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: Active manager daemon x restarted 2024-09-10T10:37:20.842 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: Activating manager daemon x 2024-09-10T10:37:20.842 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T10:37:20.842 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: mgrmap e10: x(active, starting, since 0.00310384s) 2024-09-10T10:37:20.842 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:37:20.842 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:37:20.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:37:20.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:37:20.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:37:20.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: Manager daemon x is now available 2024-09-10T10:37:20.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:20.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:37:20.843 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:37:22.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-09-10T10:37:22.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T10:37:22.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T10:37:22.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-09-10T10:37:22.567 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.718+0000 7f890f644640 1 Processor -- start 2024-09-10T10:37:22.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.718+0000 7f890f644640 1 -- start start 2024-09-10T10:37:22.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.718+0000 7f890f644640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f8908106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:22.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.718+0000 7f890f644640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8908107000 con 0x7f8908106630 2024-09-10T10:37:22.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.718+0000 7f890d3e0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f8908106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:22.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.718+0000 7f890d3e0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f8908106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59682/0 (socket says 172.21.15.7:59682) 2024-09-10T10:37:22.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.719+0000 7f890d3e0640 1 -- 172.21.15.7:0/1570382705 learned_addr learned my addr 172.21.15.7:0/1570382705 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:22.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.719+0000 7f890d3e0640 1 -- 172.21.15.7:0/1570382705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8908107830 con 0x7f8908106630 2024-09-10T10:37:22.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.719+0000 7f890d3e0640 1 --2- 172.21.15.7:0/1570382705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f8908106a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8904009820 tx=0x7f890402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8feeac33bd1e8091 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:22.568 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.720+0000 7f88fffff640 1 -- 172.21.15.7:0/1570382705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8904037070 con 0x7f8908106630 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.720+0000 7f88fffff640 1 -- 172.21.15.7:0/1570382705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8904032920 con 0x7f8908106630 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.720+0000 7f88fffff640 1 -- 172.21.15.7:0/1570382705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8904030810 con 0x7f8908106630 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.720+0000 7f890f644640 1 -- 172.21.15.7:0/1570382705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 msgr2=0x7f8908106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.720+0000 7f890f644640 1 --2- 172.21.15.7:0/1570382705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f8908106a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8904009820 tx=0x7f890402a270 comp rx=0 tx=0).stop 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.720+0000 7f890f644640 1 -- 172.21.15.7:0/1570382705 shutdown_connections 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.720+0000 7f890f644640 1 --2- 172.21.15.7:0/1570382705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f8908106a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.720+0000 7f890f644640 1 -- 172.21.15.7:0/1570382705 >> 172.21.15.7:0/1570382705 conn(0x7f8908101e20 msgr2=0x7f8908104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.721+0000 7f890f644640 1 -- 172.21.15.7:0/1570382705 shutdown_connections 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.721+0000 7f890f644640 1 -- 172.21.15.7:0/1570382705 wait complete. 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.721+0000 7f890f644640 1 Processor -- start 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.721+0000 7f890f644640 1 -- start start 2024-09-10T10:37:22.569 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.721+0000 7f890f644640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f890819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.721+0000 7f890f644640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f890810abe0 con 0x7f8908106630 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.722+0000 7f890d3e0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f890819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.722+0000 7f890d3e0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f890819b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59684/0 (socket says 172.21.15.7:59684) 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.722+0000 7f890d3e0640 1 -- 172.21.15.7:0/2633021275 learned_addr learned my addr 172.21.15.7:0/2633021275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.722+0000 7f890d3e0640 1 -- 172.21.15.7:0/2633021275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890819bdd0 con 0x7f8908106630 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.722+0000 7f890d3e0640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f890819b890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89040097f0 tx=0x7f8904032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.722+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8904037040 con 0x7f8908106630 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.722+0000 7f890f644640 1 -- 172.21.15.7:0/2633021275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890819c060 con 0x7f8908106630 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.722+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8904030c40 con 0x7f8908106630 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.722+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f890403a520 con 0x7f8908106630 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.722+0000 7f890f644640 1 -- 172.21.15.7:0/2633021275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890819ed50 con 0x7f8908106630 2024-09-10T10:37:22.570 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.723+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f890403a7e0 con 0x7f8908106630 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.724+0000 7f88fe7fc640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 0x7f88f403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.724+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 --> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f88f403bb90 con 0x7f88f4038fc0 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.724+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f890406c6c0 con 0x7f8908106630 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.724+0000 7f890cbdf640 1 -- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 msgr2=0x7f88f403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2266895831 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.724+0000 7f890cbdf640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 0x7f88f403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.925+0000 7f890cbdf640 1 -- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 msgr2=0x7f88f403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2266895831 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:17.925+0000 7f890cbdf640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 0x7f88f403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:18.326+0000 7f890cbdf640 1 -- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 msgr2=0x7f88f403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2266895831 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:18.326+0000 7f890cbdf640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 0x7f88f403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:19.127+0000 7f890cbdf640 1 -- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 msgr2=0x7f88f403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2266895831 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:19.127+0000 7f890cbdf640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 0x7f88f403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:20.589+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f890402fd60 con 0x7f8908106630 2024-09-10T10:37:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:20.589+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 msgr2=0x7f88f403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:20.589+0000 7f88fe7fc640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 0x7f88f403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.591+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f8904044400 con 0x7f8908106630 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.591+0000 7f88fe7fc640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f88f403ca80 0x7f88f403ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.591+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8904043e50 con 0x7f88f403ca80 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.592+0000 7f890cbdf640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f88f403ca80 0x7f88f403ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.592+0000 7f890cbdf640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f88f403ca80 0x7f88f403ee70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f88f8003980 tx=0x7f88f8006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.593+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f8904043e50 con 0x7f88f403ca80 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.600+0000 7f890f644640 1 -- 172.21.15.7:0/2633021275 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8908106ab0 con 0x7f88f403ca80 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.600+0000 7f88fe7fc640 1 -- 172.21.15.7:0/2633021275 <== mgr.14150 v2:172.21.15.7:6800/3525096194 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f8908106ab0 con 0x7f88f403ca80 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 -- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f88f403ca80 msgr2=0x7f88f403ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:22.572 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f88f403ca80 0x7f88f403ee70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f88f8003980 tx=0x7f88f8006e10 comp rx=0 tx=0).stop 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 -- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 msgr2=0x7f890819b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f890819b890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89040097f0 tx=0x7f8904032740 comp rx=0 tx=0).stop 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 -- 172.21.15.7:0/2633021275 shutdown_connections 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f88f403ca80 0x7f88f403ee70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:6800/2266895831,v1:172.21.15.7:6801/2266895831] conn(0x7f88f4038fc0 0x7f88f403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 --2- 172.21.15.7:0/2633021275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8908106630 0x7f890819b890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 -- 172.21.15.7:0/2633021275 >> 172.21.15.7:0/2633021275 conn(0x7f8908101e20 msgr2=0x7f8908102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 -- 172.21.15.7:0/2633021275 shutdown_connections 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:21.601+0000 7f890f644640 1 -- 172.21.15.7:0/2633021275 wait complete. 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 9 is available 2024-09-10T10:37:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:Generating a dashboard self-signed certificate... 2024-09-10T10:37:22.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:22 smithi007 ceph-mon[30873]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T10:37:22.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:22 smithi007 ceph-mon[30873]: mgrmap e11: x(active, since 1.00596s) 2024-09-10T10:37:22.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:22 smithi007 ceph-mon[30873]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T10:37:22.826 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:23.608 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:23.285+0000 7fe0055b0640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:10:37:23] ENGINE Error in HTTPServer.serve 2024-09-10T10:37:23.608 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: Traceback (most recent call last): 2024-09-10T10:37:23.608 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:37:23.608 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self._connections.run(self.expiration_interval) 2024-09-10T10:37:23.608 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:37:23.608 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self._run(expiration_interval) 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: s = self.context.wrap_socket( 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: return self.sslsocket_class._create( 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self.do_handshake() 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self._sslobj.do_handshake() 2024-09-10T10:37:23.609 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:37:23.610 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:37:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-mon[30873]: mgrmap e12: x(active, since 2s) 2024-09-10T10:37:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-mon[30873]: [10/Sep/2024:10:37:23] ENGINE Bus STARTING 2024-09-10T10:37:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-mon[30873]: [10/Sep/2024:10:37:23] ENGINE Serving on http://172.21.15.7:8765 2024-09-10T10:37:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-mon[30873]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-mon[30873]: [10/Sep/2024:10:37:23] ENGINE Serving on https://172.21.15.7:7150 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-mon[30873]: [10/Sep/2024:10:37:23] ENGINE Bus STARTED 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:23 smithi007 ceph-mon[30873]: [10/Sep/2024:10:37:23] ENGINE Error in HTTPServer.serve 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout: Traceback (most recent call last): 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout: self._connections.run(self.expiration_interval) 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout: self._run(expiration_interval) 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:37:23.935 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:37:23.936 INFO:journalctl@ceph.mon.a.smithi007.stdout: s = self.context.wrap_socket( 2024-09-10T10:37:23.936 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:37:23.936 INFO:journalctl@ceph.mon.a.smithi007.stdout: return self.sslsocket_class._create( 2024-09-10T10:37:23.936 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:37:23.936 INFO:journalctl@ceph.mon.a.smithi007.stdout: self.do_handshake() 2024-09-10T10:37:23.936 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:37:23.936 INFO:journalctl@ceph.mon.a.smithi007.stdout: self._sslobj.do_handshake() 2024-09-10T10:37:23.936 INFO:journalctl@ceph.mon.a.smithi007.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:37:24.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T10:37:24.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.002+0000 7fd0bbc2c640 1 Processor -- start 2024-09-10T10:37:24.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.002+0000 7fd0bbc2c640 1 -- start start 2024-09-10T10:37:24.482 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.003+0000 7fd0bbc2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b410a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:24.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.003+0000 7fd0bbc2c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b407be90 con 0x7fd0b41081a0 2024-09-10T10:37:24.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.003+0000 7fd0b99c8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b410a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:24.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.003+0000 7fd0b99c8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b410a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38946/0 (socket says 172.21.15.7:38946) 2024-09-10T10:37:24.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.003+0000 7fd0b99c8640 1 -- 172.21.15.7:0/4128414682 learned_addr learned my addr 172.21.15.7:0/4128414682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:24.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.003+0000 7fd0b99c8640 1 -- 172.21.15.7:0/4128414682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b410aad0 con 0x7fd0b41081a0 2024-09-10T10:37:24.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.004+0000 7fd0b99c8640 1 --2- 172.21.15.7:0/4128414682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b410a590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009820 tx=0x7fd0b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69d619df2756de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:24.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.004+0000 7fd0b89c6640 1 -- 172.21.15.7:0/4128414682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0b0037070 con 0x7fd0b41081a0 2024-09-10T10:37:24.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.004+0000 7fd0b89c6640 1 -- 172.21.15.7:0/4128414682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b0032920 con 0x7fd0b41081a0 2024-09-10T10:37:24.483 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.004+0000 7fd0b89c6640 1 -- 172.21.15.7:0/4128414682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0b0030810 con 0x7fd0b41081a0 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.004+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/4128414682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 msgr2=0x7fd0b410a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.004+0000 7fd0bbc2c640 1 --2- 172.21.15.7:0/4128414682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b410a590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009820 tx=0x7fd0b002a270 comp rx=0 tx=0).stop 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.005+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/4128414682 shutdown_connections 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.005+0000 7fd0bbc2c640 1 --2- 172.21.15.7:0/4128414682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b410a590 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.005+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/4128414682 >> 172.21.15.7:0/4128414682 conn(0x7fd0b4101e20 msgr2=0x7fd0b4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.005+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/4128414682 shutdown_connections 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.005+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/4128414682 wait complete. 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.005+0000 7fd0bbc2c640 1 Processor -- start 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.005+0000 7fd0bbc2c640 1 -- start start 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.005+0000 7fd0bbc2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:24.484 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0bbc2c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b410bbb0 con 0x7fd0b41081a0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0b99c8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0b99c8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38960/0 (socket says 172.21.15.7:38960) 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0b99c8640 1 -- 172.21.15.7:0/1461361941 learned_addr learned my addr 172.21.15.7:0/1461361941 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0b99c8640 1 -- 172.21.15.7:0/1461361941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b419bcf0 con 0x7fd0b41081a0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0b99c8640 1 --2- 172.21.15.7:0/1461361941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b419b7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0030420 tx=0x7fd0b0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0aaffd640 1 -- 172.21.15.7:0/1461361941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0b0037040 con 0x7fd0b41081a0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0aaffd640 1 -- 172.21.15.7:0/1461361941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b0040070 con 0x7fd0b41081a0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/1461361941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b419bf80 con 0x7fd0b41081a0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0aaffd640 1 -- 172.21.15.7:0/1461361941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0b003a6e0 con 0x7fd0b41081a0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.006+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/1461361941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b419ec70 con 0x7fd0b41081a0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.007+0000 7fd0aaffd640 1 -- 172.21.15.7:0/1461361941 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fd0b003a9e0 con 0x7fd0b41081a0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.007+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/1461361941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b4108600 con 0x7fd0b41081a0 2024-09-10T10:37:24.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.007+0000 7fd0aaffd640 1 --2- 172.21.15.7:0/1461361941 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd0a0038fc0 0x7fd0a003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.007+0000 7fd0aaffd640 1 -- 172.21.15.7:0/1461361941 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd0b006c890 con 0x7fd0b41081a0 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.008+0000 7fd0b91c7640 1 --2- 172.21.15.7:0/1461361941 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd0a0038fc0 0x7fd0a003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.008+0000 7fd0b91c7640 1 --2- 172.21.15.7:0/1461361941 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd0a0038fc0 0x7fd0a003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40098c0 tx=0x7fd0a4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.016+0000 7fd0aaffd640 1 -- 172.21.15.7:0/1461361941 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0b003ae20 con 0x7fd0b41081a0 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.204+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/1461361941 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fd0b41030e0 con 0x7fd0a0038fc0 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.245+0000 7fd0aaffd640 1 -- 172.21.15.7:0/1461361941 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fd0b41030e0 con 0x7fd0a0038fc0 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.247+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/1461361941 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd0a0038fc0 msgr2=0x7fd0a003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.247+0000 7fd0bbc2c640 1 --2- 172.21.15.7:0/1461361941 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd0a0038fc0 0x7fd0a003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40098c0 tx=0x7fd0a4006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.247+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/1461361941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 msgr2=0x7fd0b419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.247+0000 7fd0bbc2c640 1 --2- 172.21.15.7:0/1461361941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b419b7b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0030420 tx=0x7fd0b0032670 comp rx=0 tx=0).stop 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.247+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/1461361941 shutdown_connections 2024-09-10T10:37:24.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.247+0000 7fd0bbc2c640 1 --2- 172.21.15.7:0/1461361941 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd0a0038fc0 0x7fd0a003b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:24.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.247+0000 7fd0bbc2c640 1 --2- 172.21.15.7:0/1461361941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b41081a0 0x7fd0b419b7b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:24.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.247+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/1461361941 >> 172.21.15.7:0/1461361941 conn(0x7fd0b4101e20 msgr2=0x7fd0b41029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:24.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.247+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/1461361941 shutdown_connections 2024-09-10T10:37:24.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:23.247+0000 7fd0bbc2c640 1 -- 172.21.15.7:0/1461361941 wait complete. 2024-09-10T10:37:24.487 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial admin user... 2024-09-10T10:37:26.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$cO2yhjWbs8K18uVoMnkb4.OyIcpPVpA2gCMl7rTQ0QmFjIo5HW2o6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725964645, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T10:37:26.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.908+0000 7fa1fb439640 1 Processor -- start 2024-09-10T10:37:26.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.908+0000 7fa1fb439640 1 -- start start 2024-09-10T10:37:26.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.908+0000 7fa1fb439640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f410a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:26.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.908+0000 7fa1fb439640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f407be10 con 0x7fa1f4108200 2024-09-10T10:37:26.008 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.909+0000 7fa1f91d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f410a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.909+0000 7fa1f91d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f410a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38964/0 (socket says 172.21.15.7:38964) 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.909+0000 7fa1f91d5640 1 -- 172.21.15.7:0/592421425 learned_addr learned my addr 172.21.15.7:0/592421425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.909+0000 7fa1f91d5640 1 -- 172.21.15.7:0/592421425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f410ab30 con 0x7fa1f4108200 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.910+0000 7fa1f91d5640 1 --2- 172.21.15.7:0/592421425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f410a5f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009820 tx=0x7fa1f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0d7a302b6d3782 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.910+0000 7fa1ebfff640 1 -- 172.21.15.7:0/592421425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f0037070 con 0x7fa1f4108200 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.910+0000 7fa1ebfff640 1 -- 172.21.15.7:0/592421425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f0032920 con 0x7fa1f4108200 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.910+0000 7fa1ebfff640 1 -- 172.21.15.7:0/592421425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f0030810 con 0x7fa1f4108200 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.910+0000 7fa1fb439640 1 -- 172.21.15.7:0/592421425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 msgr2=0x7fa1f410a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.910+0000 7fa1fb439640 1 --2- 172.21.15.7:0/592421425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f410a5f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009820 tx=0x7fa1f002a270 comp rx=0 tx=0).stop 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.910+0000 7fa1fb439640 1 -- 172.21.15.7:0/592421425 shutdown_connections 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.910+0000 7fa1fb439640 1 --2- 172.21.15.7:0/592421425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f410a5f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.910+0000 7fa1fb439640 1 -- 172.21.15.7:0/592421425 >> 172.21.15.7:0/592421425 conn(0x7fa1f4101da0 msgr2=0x7fa1f41041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:26.009 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.911+0000 7fa1fb439640 1 -- 172.21.15.7:0/592421425 shutdown_connections 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.911+0000 7fa1fb439640 1 -- 172.21.15.7:0/592421425 wait complete. 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.911+0000 7fa1fb439640 1 Processor -- start 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.911+0000 7fa1fb439640 1 -- start start 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.911+0000 7fa1fb439640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.911+0000 7fa1fb439640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f410bb50 con 0x7fa1f4108200 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.912+0000 7fa1f91d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.912+0000 7fa1f91d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f419d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38968/0 (socket says 172.21.15.7:38968) 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.912+0000 7fa1f91d5640 1 -- 172.21.15.7:0/956810803 learned_addr learned my addr 172.21.15.7:0/956810803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.912+0000 7fa1f91d5640 1 -- 172.21.15.7:0/956810803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f419de50 con 0x7fa1f4108200 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.912+0000 7fa1f91d5640 1 --2- 172.21.15.7:0/956810803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f419d910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00097f0 tx=0x7fa1f00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.912+0000 7fa1ea7fc640 1 -- 172.21.15.7:0/956810803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f0037040 con 0x7fa1f4108200 2024-09-10T10:37:26.010 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.912+0000 7fa1ea7fc640 1 -- 172.21.15.7:0/956810803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f0030cc0 con 0x7fa1f4108200 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.912+0000 7fa1fb439640 1 -- 172.21.15.7:0/956810803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f419e0e0 con 0x7fa1f4108200 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.912+0000 7fa1ea7fc640 1 -- 172.21.15.7:0/956810803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f003a6b0 con 0x7fa1f4108200 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.912+0000 7fa1fb439640 1 -- 172.21.15.7:0/956810803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f41a0dd0 con 0x7fa1f4108200 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.913+0000 7fa1ea7fc640 1 -- 172.21.15.7:0/956810803 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa1f003a850 con 0x7fa1f4108200 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.913+0000 7fa1ea7fc640 1 --2- 172.21.15.7:0/956810803 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa1e0039020 0x7fa1e003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.913+0000 7fa1ea7fc640 1 -- 172.21.15.7:0/956810803 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa1f006c5f0 con 0x7fa1f4108200 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.914+0000 7fa1fb439640 1 -- 172.21.15.7:0/956810803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f4108660 con 0x7fa1f4108200 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.914+0000 7fa1f89d4640 1 --2- 172.21.15.7:0/956810803 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa1e0039020 0x7fa1e003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.914+0000 7fa1f89d4640 1 --2- 172.21.15.7:0/956810803 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa1e0039020 0x7fa1e003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40098c0 tx=0x7fa1e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:24.919+0000 7fa1ea7fc640 1 -- 172.21.15.7:0/956810803 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1f0044920 con 0x7fa1f4108200 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.112+0000 7fa1fb439640 1 -- 172.21.15.7:0/956810803 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa1f41a0fa0 con 0x7fa1e0039020 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.361+0000 7fa1ea7fc640 1 -- 172.21.15.7:0/956810803 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa1f41a0fa0 con 0x7fa1e0039020 2024-09-10T10:37:26.011 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.364+0000 7fa1fb439640 1 -- 172.21.15.7:0/956810803 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa1e0039020 msgr2=0x7fa1e003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:26.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.364+0000 7fa1fb439640 1 --2- 172.21.15.7:0/956810803 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa1e0039020 0x7fa1e003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40098c0 tx=0x7fa1e4006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:26.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.364+0000 7fa1fb439640 1 -- 172.21.15.7:0/956810803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 msgr2=0x7fa1f419d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:26.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.364+0000 7fa1fb439640 1 --2- 172.21.15.7:0/956810803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f419d910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00097f0 tx=0x7fa1f00327b0 comp rx=0 tx=0).stop 2024-09-10T10:37:26.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.364+0000 7fa1fb439640 1 -- 172.21.15.7:0/956810803 shutdown_connections 2024-09-10T10:37:26.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.364+0000 7fa1fb439640 1 --2- 172.21.15.7:0/956810803 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa1e0039020 0x7fa1e003b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:26.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.364+0000 7fa1fb439640 1 --2- 172.21.15.7:0/956810803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f4108200 0x7fa1f419d910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:26.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.364+0000 7fa1fb439640 1 -- 172.21.15.7:0/956810803 >> 172.21.15.7:0/956810803 conn(0x7fa1f4101da0 msgr2=0x7fa1f41028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:26.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.364+0000 7fa1fb439640 1 -- 172.21.15.7:0/956810803 shutdown_connections 2024-09-10T10:37:26.012 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:25.364+0000 7fa1fb439640 1 -- 172.21.15.7:0/956810803 wait complete. 2024-09-10T10:37:26.012 INFO:teuthology.orchestra.run.smithi007.stdout:Fetching dashboard port number... 2024-09-10T10:37:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:26 smithi007 ceph-mon[30873]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:26.572 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:27.165 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.385+0000 7efd44f4b640 1 Processor -- start 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.386+0000 7efd44f4b640 1 -- start start 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.386+0000 7efd44f4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd40106370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.386+0000 7efd44f4b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd4007be90 con 0x7efd40105f70 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.386+0000 7efd3e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd40106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.386+0000 7efd3e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd40106370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38976/0 (socket says 172.21.15.7:38976) 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.386+0000 7efd3e59c640 1 -- 172.21.15.7:0/1327166609 learned_addr learned my addr 172.21.15.7:0/1327166609 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.386+0000 7efd3e59c640 1 -- 172.21.15.7:0/1327166609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd401068b0 con 0x7efd40105f70 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.387+0000 7efd3e59c640 1 --2- 172.21.15.7:0/1327166609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd40106370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd30009820 tx=0x7efd3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60759bb490ae0add server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.387+0000 7efd3dd9b640 1 -- 172.21.15.7:0/1327166609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd30037070 con 0x7efd40105f70 2024-09-10T10:37:27.166 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.387+0000 7efd3dd9b640 1 -- 172.21.15.7:0/1327166609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd30032920 con 0x7efd40105f70 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.387+0000 7efd3dd9b640 1 -- 172.21.15.7:0/1327166609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd30030810 con 0x7efd40105f70 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.388+0000 7efd44f4b640 1 -- 172.21.15.7:0/1327166609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 msgr2=0x7efd40106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.388+0000 7efd44f4b640 1 --2- 172.21.15.7:0/1327166609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd40106370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd30009820 tx=0x7efd3002a270 comp rx=0 tx=0).stop 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.388+0000 7efd44f4b640 1 -- 172.21.15.7:0/1327166609 shutdown_connections 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.388+0000 7efd44f4b640 1 --2- 172.21.15.7:0/1327166609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd40106370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.388+0000 7efd44f4b640 1 -- 172.21.15.7:0/1327166609 >> 172.21.15.7:0/1327166609 conn(0x7efd40101da0 msgr2=0x7efd401041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.388+0000 7efd44f4b640 1 -- 172.21.15.7:0/1327166609 shutdown_connections 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.388+0000 7efd44f4b640 1 -- 172.21.15.7:0/1327166609 wait complete. 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.389+0000 7efd44f4b640 1 Processor -- start 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.389+0000 7efd44f4b640 1 -- start start 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.389+0000 7efd44f4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd4019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.389+0000 7efd44f4b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd401079f0 con 0x7efd40105f70 2024-09-10T10:37:27.167 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.389+0000 7efd3e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd4019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.389+0000 7efd3e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd4019d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38978/0 (socket says 172.21.15.7:38978) 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.389+0000 7efd3e59c640 1 -- 172.21.15.7:0/1907558500 learned_addr learned my addr 172.21.15.7:0/1907558500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.389+0000 7efd3e59c640 1 -- 172.21.15.7:0/1907558500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4019de70 con 0x7efd40105f70 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.390+0000 7efd3e59c640 1 --2- 172.21.15.7:0/1907558500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd4019d930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd300097f0 tx=0x7efd300327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.390+0000 7efd377fe640 1 -- 172.21.15.7:0/1907558500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd30037040 con 0x7efd40105f70 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.390+0000 7efd377fe640 1 -- 172.21.15.7:0/1907558500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd30030cc0 con 0x7efd40105f70 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.390+0000 7efd44f4b640 1 -- 172.21.15.7:0/1907558500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4019e100 con 0x7efd40105f70 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.390+0000 7efd377fe640 1 -- 172.21.15.7:0/1907558500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd3003a6b0 con 0x7efd40105f70 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.390+0000 7efd44f4b640 1 -- 172.21.15.7:0/1907558500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd401a0df0 con 0x7efd40105f70 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.391+0000 7efd377fe640 1 -- 172.21.15.7:0/1907558500 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7efd3003b3f0 con 0x7efd40105f70 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.391+0000 7efd377fe640 1 --2- 172.21.15.7:0/1907558500 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efd2c039020 0x7efd2c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.391+0000 7efd44f4b640 1 -- 172.21.15.7:0/1907558500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd40106370 con 0x7efd40105f70 2024-09-10T10:37:27.168 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.391+0000 7efd377fe640 1 -- 172.21.15.7:0/1907558500 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efd3006c5a0 con 0x7efd40105f70 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.391+0000 7efd37fff640 1 --2- 172.21.15.7:0/1907558500 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efd2c039020 0x7efd2c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.391+0000 7efd37fff640 1 --2- 172.21.15.7:0/1907558500 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efd2c039020 0x7efd2c03b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd280098c0 tx=0x7efd28006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.396+0000 7efd377fe640 1 -- 172.21.15.7:0/1907558500 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd3003b6e0 con 0x7efd40105f70 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.570+0000 7efd44f4b640 1 -- 172.21.15.7:0/1907558500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7efd40196da0 con 0x7efd40105f70 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.570+0000 7efd377fe640 1 -- 172.21.15.7:0/1907558500 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7efd3003a850 con 0x7efd40105f70 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.572+0000 7efd44f4b640 1 -- 172.21.15.7:0/1907558500 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efd2c039020 msgr2=0x7efd2c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.572+0000 7efd44f4b640 1 --2- 172.21.15.7:0/1907558500 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efd2c039020 0x7efd2c03b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd280098c0 tx=0x7efd28006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.572+0000 7efd44f4b640 1 -- 172.21.15.7:0/1907558500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 msgr2=0x7efd4019d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.572+0000 7efd44f4b640 1 --2- 172.21.15.7:0/1907558500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd4019d930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd300097f0 tx=0x7efd300327b0 comp rx=0 tx=0).stop 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.572+0000 7efd44f4b640 1 -- 172.21.15.7:0/1907558500 shutdown_connections 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.572+0000 7efd44f4b640 1 --2- 172.21.15.7:0/1907558500 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efd2c039020 0x7efd2c03b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.572+0000 7efd44f4b640 1 --2- 172.21.15.7:0/1907558500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd40105f70 0x7efd4019d930 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:27.169 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.572+0000 7efd44f4b640 1 -- 172.21.15.7:0/1907558500 >> 172.21.15.7:0/1907558500 conn(0x7efd40101da0 msgr2=0x7efd40191c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:27.170 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.572+0000 7efd44f4b640 1 -- 172.21.15.7:0/1907558500 shutdown_connections 2024-09-10T10:37:27.170 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:26.573+0000 7efd44f4b640 1 -- 172.21.15.7:0/1907558500 wait complete. 2024-09-10T10:37:27.173 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T10:37:27.174 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-09-10T10:37:27.182 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T10:37:27.182 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-09-10T10:37:27.182 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-09-10T10:37:27.182 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T10:37:27.184 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph Dashboard is now available at: 2024-09-10T10:37:27.185 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:27.185 INFO:teuthology.orchestra.run.smithi007.stdout: URL: https://smithi007.front.sepia.ceph.com:8443/ 2024-09-10T10:37:27.185 INFO:teuthology.orchestra.run.smithi007.stdout: User: admin 2024-09-10T10:37:27.185 INFO:teuthology.orchestra.run.smithi007.stdout: Password: hr833rxm5o 2024-09-10T10:37:27.186 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:27.186 INFO:teuthology.orchestra.run.smithi007.stdout:Saving cluster configuration to /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config directory 2024-09-10T10:37:27.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:27 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1907558500' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T10:37:28.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:28 smithi007 ceph-mon[30873]: mgrmap e13: x(active, since 6s) 2024-09-10T10:37:28.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:28 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3036853519' entity='client.admin' 2024-09-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.593+0000 7fbc90932640 1 Processor -- start 2024-09-10T10:37:28.456 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.593+0000 7fbc90932640 1 -- start start 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.594+0000 7fbc90932640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c10a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.594+0000 7fbc90932640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c07bdf0 con 0x7fbc8c108220 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.594+0000 7fbc8a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c10a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.594+0000 7fbc8a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c10a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38982/0 (socket says 172.21.15.7:38982) 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.594+0000 7fbc8a59c640 1 -- 172.21.15.7:0/3257286424 learned_addr learned my addr 172.21.15.7:0/3257286424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.594+0000 7fbc8a59c640 1 -- 172.21.15.7:0/3257286424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c10ab50 con 0x7fbc8c108220 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.595+0000 7fbc8a59c640 1 --2- 172.21.15.7:0/3257286424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c10a610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009820 tx=0x7fbc7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac0c2d15f6bef57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.595+0000 7fbc8959a640 1 -- 172.21.15.7:0/3257286424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74037070 con 0x7fbc8c108220 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.595+0000 7fbc8959a640 1 -- 172.21.15.7:0/3257286424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc74032920 con 0x7fbc8c108220 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.595+0000 7fbc8959a640 1 -- 172.21.15.7:0/3257286424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74030810 con 0x7fbc8c108220 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.596+0000 7fbc90932640 1 -- 172.21.15.7:0/3257286424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 msgr2=0x7fbc8c10a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:28.457 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.596+0000 7fbc90932640 1 --2- 172.21.15.7:0/3257286424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c10a610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009820 tx=0x7fbc7402a270 comp rx=0 tx=0).stop 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.596+0000 7fbc90932640 1 -- 172.21.15.7:0/3257286424 shutdown_connections 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.596+0000 7fbc90932640 1 --2- 172.21.15.7:0/3257286424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c10a610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.596+0000 7fbc90932640 1 -- 172.21.15.7:0/3257286424 >> 172.21.15.7:0/3257286424 conn(0x7fbc8c101da0 msgr2=0x7fbc8c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.596+0000 7fbc90932640 1 -- 172.21.15.7:0/3257286424 shutdown_connections 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.596+0000 7fbc90932640 1 -- 172.21.15.7:0/3257286424 wait complete. 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.597+0000 7fbc90932640 1 Processor -- start 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.597+0000 7fbc90932640 1 -- start start 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.597+0000 7fbc90932640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.597+0000 7fbc90932640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c10bb70 con 0x7fbc8c108220 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.597+0000 7fbc8a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.597+0000 7fbc8a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38994/0 (socket says 172.21.15.7:38994) 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.597+0000 7fbc8a59c640 1 -- 172.21.15.7:0/3036853519 learned_addr learned my addr 172.21.15.7:0/3036853519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:28.458 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.598+0000 7fbc8a59c640 1 -- 172.21.15.7:0/3036853519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c19de90 con 0x7fbc8c108220 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.598+0000 7fbc8a59c640 1 --2- 172.21.15.7:0/3036853519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c19d950 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc740097f0 tx=0x7fbc740327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.598+0000 7fbc837fe640 1 -- 172.21.15.7:0/3036853519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc74037040 con 0x7fbc8c108220 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.598+0000 7fbc837fe640 1 -- 172.21.15.7:0/3036853519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc74030cc0 con 0x7fbc8c108220 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.598+0000 7fbc90932640 1 -- 172.21.15.7:0/3036853519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c19e120 con 0x7fbc8c108220 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.598+0000 7fbc837fe640 1 -- 172.21.15.7:0/3036853519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc7403a6b0 con 0x7fbc8c108220 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.598+0000 7fbc90932640 1 -- 172.21.15.7:0/3036853519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c1a0e10 con 0x7fbc8c108220 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.599+0000 7fbc837fe640 1 -- 172.21.15.7:0/3036853519 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fbc7403a8a0 con 0x7fbc8c108220 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.599+0000 7fbc90932640 1 -- 172.21.15.7:0/3036853519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c108680 con 0x7fbc8c108220 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.599+0000 7fbc837fe640 1 --2- 172.21.15.7:0/3036853519 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fbc78039060 0x7fbc7803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.599+0000 7fbc837fe640 1 -- 172.21.15.7:0/3036853519 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbc7406c870 con 0x7fbc8c108220 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.599+0000 7fbc89d9b640 1 --2- 172.21.15.7:0/3036853519 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fbc78039060 0x7fbc7803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:28.459 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.600+0000 7fbc89d9b640 1 --2- 172.21.15.7:0/3036853519 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fbc78039060 0x7fbc7803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0098c0 tx=0x7fbc7c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.604+0000 7fbc837fe640 1 -- 172.21.15.7:0/3036853519 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc7403ace0 con 0x7fbc8c108220 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.871+0000 7fbc90932640 1 -- 172.21.15.7:0/3036853519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fbc8c1a0fe0 con 0x7fbc8c108220 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.873+0000 7fbc837fe640 1 -- 172.21.15.7:0/3036853519 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fbc7402fd40 con 0x7fbc8c108220 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.875+0000 7fbc90932640 1 -- 172.21.15.7:0/3036853519 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fbc78039060 msgr2=0x7fbc7803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.875+0000 7fbc90932640 1 --2- 172.21.15.7:0/3036853519 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fbc78039060 0x7fbc7803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0098c0 tx=0x7fbc7c006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.875+0000 7fbc90932640 1 -- 172.21.15.7:0/3036853519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 msgr2=0x7fbc8c19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.875+0000 7fbc90932640 1 --2- 172.21.15.7:0/3036853519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c19d950 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbc740097f0 tx=0x7fbc740327b0 comp rx=0 tx=0).stop 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.875+0000 7fbc90932640 1 -- 172.21.15.7:0/3036853519 shutdown_connections 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.875+0000 7fbc90932640 1 --2- 172.21.15.7:0/3036853519 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fbc78039060 0x7fbc7803b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.875+0000 7fbc90932640 1 --2- 172.21.15.7:0/3036853519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc8c108220 0x7fbc8c19d950 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.875+0000 7fbc90932640 1 -- 172.21.15.7:0/3036853519 >> 172.21.15.7:0/3036853519 conn(0x7fbc8c101da0 msgr2=0x7fbc8c1028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:28.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.876+0000 7fbc90932640 1 -- 172.21.15.7:0/3036853519 shutdown_connections 2024-09-10T10:37:28.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-09-10T10:37:27.876+0000 7fbc90932640 1 -- 172.21.15.7:0/3036853519 wait complete. 2024-09-10T10:37:28.461 INFO:teuthology.orchestra.run.smithi007.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T10:37:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:37:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:28.461 INFO:teuthology.orchestra.run.smithi007.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T10:37:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T10:37:28.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:28.462 INFO:teuthology.orchestra.run.smithi007.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T10:37:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: ceph telemetry on 2024-09-10T10:37:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:28.462 INFO:teuthology.orchestra.run.smithi007.stdout:For more information see: 2024-09-10T10:37:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T10:37:28.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:28.462 INFO:teuthology.orchestra.run.smithi007.stdout:Bootstrap complete. 2024-09-10T10:37:28.502 INFO:tasks.cephadm:Fetching config... 2024-09-10T10:37:28.502 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:37:28.503 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T10:37:28.521 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T10:37:28.521 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:37:28.521 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T10:37:28.579 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T10:37:28.579 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:37:28.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T10:37:28.646 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T10:37:28.646 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:37:28.646 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T10:37:28.703 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T10:37:28.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBAC63hyxwWCbZDFBBncXvIvACXJ9bZwRTNm7ZJo53XA+IUttcctyl1oBI8hFRQnV8USmOE/ZYIV2GeZlYm7Ayyjo3Nssd0u0EUOjrhYFcSijbkvaUcl+pPCEVTD7p1Co7UxFPldEls5URR9FBMLfDEx0Xkd6w3ozI+jURIQJIEdIjQiKpinq4Db1K3QONW4Oipcmoa/QKS0Q9txQ5P/yWu2Tx52xglLm6Nd/T7mcvuT1boec7M7Y5YOduyrWmZ2Nn1C6VEXpH/kttwOOR6aMQ7eTIlMn+Eed8NwCwMvB6VRnLVGHGnoMoqAb0Sf6MRvQsO9CQOpQ0u9vgLYZRJQj7zIc/xu5uMyU6cg9p4S3w9tRadOVpL7AN9H/3xcZf2x+MfJ0X0I/E0Lrz2pHq/vheNma4aO5fLH5S2sUMWBWBP8Sqfktio1fmhHWyFj/m0lw+bfvrGQTLvSfnr1Qsn90r9o4UlC1rrxFBcte5xOg6MBLkxtev99arvBQqgBAaWN8= ceph-65af4d96-6f60-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T10:37:28.781 INFO:teuthology.orchestra.run.smithi007.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBAC63hyxwWCbZDFBBncXvIvACXJ9bZwRTNm7ZJo53XA+IUttcctyl1oBI8hFRQnV8USmOE/ZYIV2GeZlYm7Ayyjo3Nssd0u0EUOjrhYFcSijbkvaUcl+pPCEVTD7p1Co7UxFPldEls5URR9FBMLfDEx0Xkd6w3ozI+jURIQJIEdIjQiKpinq4Db1K3QONW4Oipcmoa/QKS0Q9txQ5P/yWu2Tx52xglLm6Nd/T7mcvuT1boec7M7Y5YOduyrWmZ2Nn1C6VEXpH/kttwOOR6aMQ7eTIlMn+Eed8NwCwMvB6VRnLVGHGnoMoqAb0Sf6MRvQsO9CQOpQ0u9vgLYZRJQj7zIc/xu5uMyU6cg9p4S3w9tRadOVpL7AN9H/3xcZf2x+MfJ0X0I/E0Lrz2pHq/vheNma4aO5fLH5S2sUMWBWBP8Sqfktio1fmhHWyFj/m0lw+bfvrGQTLvSfnr1Qsn90r9o4UlC1rrxFBcte5xOg6MBLkxtev99arvBQqgBAaWN8= ceph-65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:37:28.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBAC63hyxwWCbZDFBBncXvIvACXJ9bZwRTNm7ZJo53XA+IUttcctyl1oBI8hFRQnV8USmOE/ZYIV2GeZlYm7Ayyjo3Nssd0u0EUOjrhYFcSijbkvaUcl+pPCEVTD7p1Co7UxFPldEls5URR9FBMLfDEx0Xkd6w3ozI+jURIQJIEdIjQiKpinq4Db1K3QONW4Oipcmoa/QKS0Q9txQ5P/yWu2Tx52xglLm6Nd/T7mcvuT1boec7M7Y5YOduyrWmZ2Nn1C6VEXpH/kttwOOR6aMQ7eTIlMn+Eed8NwCwMvB6VRnLVGHGnoMoqAb0Sf6MRvQsO9CQOpQ0u9vgLYZRJQj7zIc/xu5uMyU6cg9p4S3w9tRadOVpL7AN9H/3xcZf2x+MfJ0X0I/E0Lrz2pHq/vheNma4aO5fLH5S2sUMWBWBP8Sqfktio1fmhHWyFj/m0lw+bfvrGQTLvSfnr1Qsn90r9o4UlC1rrxFBcte5xOg6MBLkxtev99arvBQqgBAaWN8= ceph-65af4d96-6f60-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T10:37:28.848 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBAC63hyxwWCbZDFBBncXvIvACXJ9bZwRTNm7ZJo53XA+IUttcctyl1oBI8hFRQnV8USmOE/ZYIV2GeZlYm7Ayyjo3Nssd0u0EUOjrhYFcSijbkvaUcl+pPCEVTD7p1Co7UxFPldEls5URR9FBMLfDEx0Xkd6w3ozI+jURIQJIEdIjQiKpinq4Db1K3QONW4Oipcmoa/QKS0Q9txQ5P/yWu2Tx52xglLm6Nd/T7mcvuT1boec7M7Y5YOduyrWmZ2Nn1C6VEXpH/kttwOOR6aMQ7eTIlMn+Eed8NwCwMvB6VRnLVGHGnoMoqAb0Sf6MRvQsO9CQOpQ0u9vgLYZRJQj7zIc/xu5uMyU6cg9p4S3w9tRadOVpL7AN9H/3xcZf2x+MfJ0X0I/E0Lrz2pHq/vheNma4aO5fLH5S2sUMWBWBP8Sqfktio1fmhHWyFj/m0lw+bfvrGQTLvSfnr1Qsn90r9o4UlC1rrxFBcte5xOg6MBLkxtev99arvBQqgBAaWN8= ceph-65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:37:28.860 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBAC63hyxwWCbZDFBBncXvIvACXJ9bZwRTNm7ZJo53XA+IUttcctyl1oBI8hFRQnV8USmOE/ZYIV2GeZlYm7Ayyjo3Nssd0u0EUOjrhYFcSijbkvaUcl+pPCEVTD7p1Co7UxFPldEls5URR9FBMLfDEx0Xkd6w3ozI+jURIQJIEdIjQiKpinq4Db1K3QONW4Oipcmoa/QKS0Q9txQ5P/yWu2Tx52xglLm6Nd/T7mcvuT1boec7M7Y5YOduyrWmZ2Nn1C6VEXpH/kttwOOR6aMQ7eTIlMn+Eed8NwCwMvB6VRnLVGHGnoMoqAb0Sf6MRvQsO9CQOpQ0u9vgLYZRJQj7zIc/xu5uMyU6cg9p4S3w9tRadOVpL7AN9H/3xcZf2x+MfJ0X0I/E0Lrz2pHq/vheNma4aO5fLH5S2sUMWBWBP8Sqfktio1fmhHWyFj/m0lw+bfvrGQTLvSfnr1Qsn90r9o4UlC1rrxFBcte5xOg6MBLkxtev99arvBQqgBAaWN8= ceph-65af4d96-6f60-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T10:37:28.902 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBAC63hyxwWCbZDFBBncXvIvACXJ9bZwRTNm7ZJo53XA+IUttcctyl1oBI8hFRQnV8USmOE/ZYIV2GeZlYm7Ayyjo3Nssd0u0EUOjrhYFcSijbkvaUcl+pPCEVTD7p1Co7UxFPldEls5URR9FBMLfDEx0Xkd6w3ozI+jURIQJIEdIjQiKpinq4Db1K3QONW4Oipcmoa/QKS0Q9txQ5P/yWu2Tx52xglLm6Nd/T7mcvuT1boec7M7Y5YOduyrWmZ2Nn1C6VEXpH/kttwOOR6aMQ7eTIlMn+Eed8NwCwMvB6VRnLVGHGnoMoqAb0Sf6MRvQsO9CQOpQ0u9vgLYZRJQj7zIc/xu5uMyU6cg9p4S3w9tRadOVpL7AN9H/3xcZf2x+MfJ0X0I/E0Lrz2pHq/vheNma4aO5fLH5S2sUMWBWBP8Sqfktio1fmhHWyFj/m0lw+bfvrGQTLvSfnr1Qsn90r9o4UlC1rrxFBcte5xOg6MBLkxtev99arvBQqgBAaWN8= ceph-65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:37:28.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T10:37:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:37:29.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.673+0000 7fb7be054640 1 -- 172.21.15.7:0/175007595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102150 msgr2=0x7fb7b8104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:29.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.673+0000 7fb7be054640 1 --2- 172.21.15.7:0/175007595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102150 0x7fb7b8104540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009880 tx=0x7fb7a802a5d0 comp rx=0 tx=0).stop 2024-09-10T10:37:29.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.673+0000 7fb7be054640 1 -- 172.21.15.7:0/175007595 shutdown_connections 2024-09-10T10:37:29.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.673+0000 7fb7be054640 1 --2- 172.21.15.7:0/175007595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102150 0x7fb7b8104540 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:29.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.673+0000 7fb7be054640 1 -- 172.21.15.7:0/175007595 >> 172.21.15.7:0/175007595 conn(0x7fb7b80fbcd0 msgr2=0x7fb7b80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:29.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.673+0000 7fb7be054640 1 -- 172.21.15.7:0/175007595 shutdown_connections 2024-09-10T10:37:29.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.673+0000 7fb7be054640 1 -- 172.21.15.7:0/175007595 wait complete. 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.674+0000 7fb7be054640 1 Processor -- start 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.674+0000 7fb7be054640 1 -- start start 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7be054640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102150 0x7fb7b8196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7be054640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b81076d0 con 0x7fb7b8102150 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7b77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102150 0x7fb7b8196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7b77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102150 0x7fb7b8196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39012/0 (socket says 172.21.15.7:39012) 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7b77fe640 1 -- 172.21.15.7:0/3544421054 learned_addr learned my addr 172.21.15.7:0/3544421054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7b77fe640 1 -- 172.21.15.7:0/3544421054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b81972d0 con 0x7fb7b8102150 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7b77fe640 1 --2- 172.21.15.7:0/3544421054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102150 0x7fb7b8196d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8004f60 tx=0x7fb7a80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7b57fa640 1 -- 172.21.15.7:0/3544421054 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a803f070 con 0x7fb7b8102150 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7b57fa640 1 -- 172.21.15.7:0/3544421054 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a8003ad0 con 0x7fb7b8102150 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7be054640 1 -- 172.21.15.7:0/3544421054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b8197560 con 0x7fb7b8102150 2024-09-10T10:37:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.675+0000 7fb7b57fa640 1 -- 172.21.15.7:0/3544421054 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7a8035030 con 0x7fb7b8102150 2024-09-10T10:37:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.676+0000 7fb7be054640 1 -- 172.21.15.7:0/3544421054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b819a210 con 0x7fb7b8102150 2024-09-10T10:37:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.676+0000 7fb7b57fa640 1 -- 172.21.15.7:0/3544421054 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fb7a8042070 con 0x7fb7b8102150 2024-09-10T10:37:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.677+0000 7fb7b57fa640 1 --2- 172.21.15.7:0/3544421054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7a40390c0 0x7fb7a403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.677+0000 7fb7be054640 1 -- 172.21.15.7:0/3544421054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b8107e50 con 0x7fb7b8102150 2024-09-10T10:37:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.677+0000 7fb7b57fa640 1 -- 172.21.15.7:0/3544421054 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb7a806dd80 con 0x7fb7b8102150 2024-09-10T10:37:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.677+0000 7fb7aeffd640 1 --2- 172.21.15.7:0/3544421054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7a40390c0 0x7fb7a403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.677+0000 7fb7aeffd640 1 --2- 172.21.15.7:0/3544421054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7a40390c0 0x7fb7a403b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00098c0 tx=0x7fb7a0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.681+0000 7fb7b57fa640 1 -- 172.21.15.7:0/3544421054 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7a8039350 con 0x7fb7b8102150 2024-09-10T10:37:29.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.860+0000 7fb7be054640 1 -- 172.21.15.7:0/3544421054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb7b802d8d0 con 0x7fb7b8102150 2024-09-10T10:37:29.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.863+0000 7fb7b57fa640 1 -- 172.21.15.7:0/3544421054 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb7a8003c80 con 0x7fb7b8102150 2024-09-10T10:37:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.865+0000 7fb7be054640 1 -- 172.21.15.7:0/3544421054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7a40390c0 msgr2=0x7fb7a403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.865+0000 7fb7be054640 1 --2- 172.21.15.7:0/3544421054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7a40390c0 0x7fb7a403b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00098c0 tx=0x7fb7a0006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.865+0000 7fb7be054640 1 -- 172.21.15.7:0/3544421054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102150 msgr2=0x7fb7b8196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.865+0000 7fb7be054640 1 --2- 172.21.15.7:0/3544421054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102150 0x7fb7b8196d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8004f60 tx=0x7fb7a80048f0 comp rx=0 tx=0).stop 2024-09-10T10:37:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.865+0000 7fb7be054640 1 -- 172.21.15.7:0/3544421054 shutdown_connections 2024-09-10T10:37:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.865+0000 7fb7be054640 1 --2- 172.21.15.7:0/3544421054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7a40390c0 0x7fb7a403b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.865+0000 7fb7be054640 1 --2- 172.21.15.7:0/3544421054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b8102150 0x7fb7b8196d90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:29.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.865+0000 7fb7be054640 1 -- 172.21.15.7:0/3544421054 >> 172.21.15.7:0/3544421054 conn(0x7fb7b80fbcd0 msgr2=0x7fb7b80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.866+0000 7fb7be054640 1 -- 172.21.15.7:0/3544421054 shutdown_connections 2024-09-10T10:37:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:29.866+0000 7fb7be054640 1 -- 172.21.15.7:0/3544421054 wait complete. 2024-09-10T10:37:30.981 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T10:37:30.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T10:37:31.081 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:30 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3544421054' entity='client.admin' 2024-09-10T10:37:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:37:31.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.922+0000 7f059c508640 1 -- 172.21.15.7:0/1347225041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594102130 msgr2=0x7f0594104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:31.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.922+0000 7f059c508640 1 --2- 172.21.15.7:0/1347225041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594102130 0x7f0594104520 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0590009880 tx=0x7f059002a5d0 comp rx=0 tx=0).stop 2024-09-10T10:37:31.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.922+0000 7f059c508640 1 -- 172.21.15.7:0/1347225041 shutdown_connections 2024-09-10T10:37:31.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.922+0000 7f059c508640 1 --2- 172.21.15.7:0/1347225041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594102130 0x7f0594104520 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.922+0000 7f059c508640 1 -- 172.21.15.7:0/1347225041 >> 172.21.15.7:0/1347225041 conn(0x7f05940fbcd0 msgr2=0x7f05940fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.922+0000 7f059c508640 1 -- 172.21.15.7:0/1347225041 shutdown_connections 2024-09-10T10:37:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.922+0000 7f059c508640 1 -- 172.21.15.7:0/1347225041 wait complete. 2024-09-10T10:37:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.923+0000 7f059c508640 1 Processor -- start 2024-09-10T10:37:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.923+0000 7f059c508640 1 -- start start 2024-09-10T10:37:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.923+0000 7f059c508640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594102130 0x7f05941019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.923+0000 7f059c508640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05941076e0 con 0x7f0594102130 2024-09-10T10:37:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.923+0000 7f059a2a4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594102130 0x7f05941019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.923+0000 7f059a2a4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594102130 0x7f05941019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56290/0 (socket says 172.21.15.7:56290) 2024-09-10T10:37:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.923+0000 7f059a2a4640 1 -- 172.21.15.7:0/3881201083 learned_addr learned my addr 172.21.15.7:0/3881201083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.923+0000 7f059a2a4640 1 -- 172.21.15.7:0/3881201083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0594101ef0 con 0x7f0594102130 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.924+0000 7f059a2a4640 1 --2- 172.21.15.7:0/3881201083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594102130 0x7f05941019b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0590004f60 tx=0x7f05900048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.924+0000 7f058b7fe640 1 -- 172.21.15.7:0/3881201083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f059003f070 con 0x7f0594102130 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.924+0000 7f058b7fe640 1 -- 172.21.15.7:0/3881201083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0590003ad0 con 0x7f0594102130 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.924+0000 7f059c508640 1 -- 172.21.15.7:0/3881201083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05941000e0 con 0x7f0594102130 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.924+0000 7f058b7fe640 1 -- 172.21.15.7:0/3881201083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0590035030 con 0x7f0594102130 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.924+0000 7f059c508640 1 -- 172.21.15.7:0/3881201083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0594100620 con 0x7f0594102130 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.925+0000 7f059c508640 1 -- 172.21.15.7:0/3881201083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0594107e60 con 0x7f0594102130 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.925+0000 7f058b7fe640 1 -- 172.21.15.7:0/3881201083 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0590042070 con 0x7f0594102130 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.925+0000 7f058b7fe640 1 --2- 172.21.15.7:0/3881201083 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05800390c0 0x7f058003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.925+0000 7f058b7fe640 1 -- 172.21.15.7:0/3881201083 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f059006cd70 con 0x7f0594102130 2024-09-10T10:37:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.925+0000 7f0599aa3640 1 --2- 172.21.15.7:0/3881201083 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05800390c0 0x7f058003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.926+0000 7f0599aa3640 1 --2- 172.21.15.7:0/3881201083 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05800390c0 0x7f058003b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f05840098c0 tx=0x7f0584006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:31.930+0000 7f058b7fe640 1 -- 172.21.15.7:0/3881201083 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0590071070 con 0x7f0594102130 2024-09-10T10:37:32.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.116+0000 7f059c508640 1 -- 172.21.15.7:0/3881201083 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f0594190180 con 0x7f05800390c0 2024-09-10T10:37:32.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.119+0000 7f058b7fe640 1 -- 172.21.15.7:0/3881201083 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0594190180 con 0x7f05800390c0 2024-09-10T10:37:32.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.122+0000 7f059c508640 1 -- 172.21.15.7:0/3881201083 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05800390c0 msgr2=0x7f058003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:32.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.122+0000 7f059c508640 1 --2- 172.21.15.7:0/3881201083 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05800390c0 0x7f058003b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f05840098c0 tx=0x7f0584006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:32.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.122+0000 7f059c508640 1 -- 172.21.15.7:0/3881201083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594102130 msgr2=0x7f05941019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:32.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.122+0000 7f059c508640 1 --2- 172.21.15.7:0/3881201083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594102130 0x7f05941019b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0590004f60 tx=0x7f05900048f0 comp rx=0 tx=0).stop 2024-09-10T10:37:32.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.122+0000 7f059c508640 1 -- 172.21.15.7:0/3881201083 shutdown_connections 2024-09-10T10:37:32.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.122+0000 7f059c508640 1 --2- 172.21.15.7:0/3881201083 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05800390c0 0x7f058003b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.122+0000 7f059c508640 1 --2- 172.21.15.7:0/3881201083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0594102130 0x7f05941019b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.122+0000 7f059c508640 1 -- 172.21.15.7:0/3881201083 >> 172.21.15.7:0/3881201083 conn(0x7f05940fbcd0 msgr2=0x7f05940fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.122+0000 7f059c508640 1 -- 172.21.15.7:0/3881201083 shutdown_connections 2024-09-10T10:37:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:32.122+0000 7f059c508640 1 -- 172.21.15.7:0/3881201083 wait complete. 2024-09-10T10:37:32.791 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi103 2024-09-10T10:37:32.792 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:37:32.792 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.conf 2024-09-10T10:37:32.812 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:37:32.812 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:37:32.870 INFO:tasks.cephadm:Adding host smithi103 to orchestrator... 2024-09-10T10:37:32.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch host add smithi103 2024-09-10T10:37:33.315 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:37:33.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:33 smithi007 ceph-mon[30873]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:33.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:33 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.861+0000 7f0861ee1640 1 -- 172.21.15.7:0/3115419783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f085c102150 msgr2=0x7f085c104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.861+0000 7f0861ee1640 1 --2- 172.21.15.7:0/3115419783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f085c102150 0x7f085c104540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f084c009880 tx=0x7f084c02a5d0 comp rx=0 tx=0).stop 2024-09-10T10:37:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.861+0000 7f0861ee1640 1 -- 172.21.15.7:0/3115419783 shutdown_connections 2024-09-10T10:37:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.861+0000 7f0861ee1640 1 --2- 172.21.15.7:0/3115419783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f085c102150 0x7f085c104540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.861+0000 7f0861ee1640 1 -- 172.21.15.7:0/3115419783 >> 172.21.15.7:0/3115419783 conn(0x7f085c0fbcd0 msgr2=0x7f085c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.861+0000 7f0861ee1640 1 -- 172.21.15.7:0/3115419783 shutdown_connections 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.861+0000 7f0861ee1640 1 -- 172.21.15.7:0/3115419783 wait complete. 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.861+0000 7f0861ee1640 1 Processor -- start 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.862+0000 7f0861ee1640 1 -- start start 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.862+0000 7f0861ee1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f085c102150 0x7f085c196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.862+0000 7f0861ee1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f085c1076d0 con 0x7f085c102150 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.862+0000 7f085b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f085c102150 0x7f085c196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.862+0000 7f085b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f085c102150 0x7f085c196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56306/0 (socket says 172.21.15.7:56306) 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.862+0000 7f085b7fe640 1 -- 172.21.15.7:0/3473035641 learned_addr learned my addr 172.21.15.7:0/3473035641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.862+0000 7f085b7fe640 1 -- 172.21.15.7:0/3473035641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085c1972d0 con 0x7f085c102150 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.863+0000 7f085b7fe640 1 --2- 172.21.15.7:0/3473035641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f085c102150 0x7f085c196d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f084c004f60 tx=0x7f084c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.863+0000 7f0858ff9640 1 -- 172.21.15.7:0/3473035641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f084c03f070 con 0x7f085c102150 2024-09-10T10:37:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.863+0000 7f0858ff9640 1 -- 172.21.15.7:0/3473035641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f084c003ad0 con 0x7f085c102150 2024-09-10T10:37:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.863+0000 7f0858ff9640 1 -- 172.21.15.7:0/3473035641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f084c035030 con 0x7f085c102150 2024-09-10T10:37:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.863+0000 7f0861ee1640 1 -- 172.21.15.7:0/3473035641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085c197560 con 0x7f085c102150 2024-09-10T10:37:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.863+0000 7f0861ee1640 1 -- 172.21.15.7:0/3473035641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085c19a210 con 0x7f085c102150 2024-09-10T10:37:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.864+0000 7f0858ff9640 1 -- 172.21.15.7:0/3473035641 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f084c042070 con 0x7f085c102150 2024-09-10T10:37:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.864+0000 7f0861ee1640 1 -- 172.21.15.7:0/3473035641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f085c107e50 con 0x7f085c102150 2024-09-10T10:37:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.864+0000 7f0858ff9640 1 --2- 172.21.15.7:0/3473035641 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0844039060 0x7f084403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.864+0000 7f0858ff9640 1 -- 172.21.15.7:0/3473035641 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f084c06cde0 con 0x7f085c102150 2024-09-10T10:37:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.864+0000 7f085affd640 1 --2- 172.21.15.7:0/3473035641 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0844039060 0x7f084403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.865+0000 7f085affd640 1 --2- 172.21.15.7:0/3473035641 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0844039060 0x7f084403b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f08500098c0 tx=0x7f0850006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:33.869+0000 7f0858ff9640 1 -- 172.21.15.7:0/3473035641 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f084c039350 con 0x7f085c102150 2024-09-10T10:37:34.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:34.054+0000 7f0861ee1640 1 -- 172.21.15.7:0/3473035641 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}) -- 0x7f085c02d8a0 con 0x7f0844039060 2024-09-10T10:37:34.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:34.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:34.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:37:34.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:34.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:34.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-10T10:37:34.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:37:34.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:34.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:34.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:34.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:34.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:34.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:34.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:35.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:35 smithi007 ceph-mon[30873]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:37:35.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:35 smithi007 ceph-mon[30873]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:37:35.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:35 smithi007 ceph-mon[30873]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:36.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:36 smithi007 ceph-mon[30873]: Deploying cephadm binary to smithi103 2024-09-10T10:37:38.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.015+0000 7f0858ff9640 1 -- 172.21.15.7:0/3473035641 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f085c02d8a0 con 0x7f0844039060 2024-09-10T10:37:38.017 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi103' with addr '172.21.15.103' 2024-09-10T10:37:38.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.017+0000 7f0861ee1640 1 -- 172.21.15.7:0/3473035641 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0844039060 msgr2=0x7f084403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:38.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.017+0000 7f0861ee1640 1 --2- 172.21.15.7:0/3473035641 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0844039060 0x7f084403b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f08500098c0 tx=0x7f0850006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:38.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.017+0000 7f0861ee1640 1 -- 172.21.15.7:0/3473035641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f085c102150 msgr2=0x7f085c196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:38.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.017+0000 7f0861ee1640 1 --2- 172.21.15.7:0/3473035641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f085c102150 0x7f085c196d90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f084c004f60 tx=0x7f084c0048f0 comp rx=0 tx=0).stop 2024-09-10T10:37:38.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.017+0000 7f0861ee1640 1 -- 172.21.15.7:0/3473035641 shutdown_connections 2024-09-10T10:37:38.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.017+0000 7f0861ee1640 1 --2- 172.21.15.7:0/3473035641 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0844039060 0x7f084403b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:38.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.017+0000 7f0861ee1640 1 --2- 172.21.15.7:0/3473035641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f085c102150 0x7f085c196d90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:38.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.017+0000 7f0861ee1640 1 -- 172.21.15.7:0/3473035641 >> 172.21.15.7:0/3473035641 conn(0x7f085c0fbcd0 msgr2=0x7f085c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:38.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.017+0000 7f0861ee1640 1 -- 172.21.15.7:0/3473035641 shutdown_connections 2024-09-10T10:37:38.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:38.018+0000 7f0861ee1640 1 -- 172.21.15.7:0/3473035641 wait complete. 2024-09-10T10:37:38.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T10:37:39.099 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:37:39.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:39 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:39.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:39 smithi007 ceph-mon[30873]: Added host smithi103 2024-09-10T10:37:39.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:39 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:39.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:39 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:39.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:39 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.566+0000 7fb94ca2e640 1 -- 172.21.15.7:0/1557517840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb948102730 msgr2=0x7fb948102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.566+0000 7fb94ca2e640 1 --2- 172.21.15.7:0/1557517840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb948102730 0x7fb948102b10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb938009c10 tx=0x7fb93802aa10 comp rx=0 tx=0).stop 2024-09-10T10:37:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.566+0000 7fb94ca2e640 1 -- 172.21.15.7:0/1557517840 shutdown_connections 2024-09-10T10:37:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.566+0000 7fb94ca2e640 1 --2- 172.21.15.7:0/1557517840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb948102730 0x7fb948102b10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.566+0000 7fb94ca2e640 1 -- 172.21.15.7:0/1557517840 >> 172.21.15.7:0/1557517840 conn(0x7fb948077310 msgr2=0x7fb9480fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.566+0000 7fb94ca2e640 1 -- 172.21.15.7:0/1557517840 shutdown_connections 2024-09-10T10:37:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.566+0000 7fb94ca2e640 1 -- 172.21.15.7:0/1557517840 wait complete. 2024-09-10T10:37:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.567+0000 7fb94ca2e640 1 Processor -- start 2024-09-10T10:37:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.567+0000 7fb94ca2e640 1 -- start start 2024-09-10T10:37:39.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.567+0000 7fb94ca2e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb948102730 0x7fb948199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:39.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.567+0000 7fb94ca2e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810bc80 con 0x7fb948102730 2024-09-10T10:37:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.567+0000 7fb94659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb948102730 0x7fb948199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.567+0000 7fb94659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb948102730 0x7fb948199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56328/0 (socket says 172.21.15.7:56328) 2024-09-10T10:37:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.567+0000 7fb94659c640 1 -- 172.21.15.7:0/61149511 learned_addr learned my addr 172.21.15.7:0/61149511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.567+0000 7fb94659c640 1 -- 172.21.15.7:0/61149511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb948199950 con 0x7fb948102730 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.567+0000 7fb94659c640 1 --2- 172.21.15.7:0/61149511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb948102730 0x7fb948199410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb938009d30 tx=0x7fb938030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.568+0000 7fb93f7fe640 1 -- 172.21.15.7:0/61149511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb938034070 con 0x7fb948102730 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.568+0000 7fb93f7fe640 1 -- 172.21.15.7:0/61149511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb93802f8c0 con 0x7fb948102730 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.568+0000 7fb94ca2e640 1 -- 172.21.15.7:0/61149511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb948199be0 con 0x7fb948102730 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.568+0000 7fb93f7fe640 1 -- 172.21.15.7:0/61149511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9380044d0 con 0x7fb948102730 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.568+0000 7fb94ca2e640 1 -- 172.21.15.7:0/61149511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb948199fe0 con 0x7fb948102730 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.569+0000 7fb93f7fe640 1 -- 172.21.15.7:0/61149511 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fb938037020 con 0x7fb948102730 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.569+0000 7fb93f7fe640 1 --2- 172.21.15.7:0/61149511 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb934039060 0x7fb93403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.569+0000 7fb94ca2e640 1 -- 172.21.15.7:0/61149511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb948069f90 con 0x7fb948102730 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.569+0000 7fb93f7fe640 1 -- 172.21.15.7:0/61149511 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb93806cea0 con 0x7fb948102730 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.569+0000 7fb945d9b640 1 --2- 172.21.15.7:0/61149511 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb934039060 0x7fb93403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.569+0000 7fb945d9b640 1 --2- 172.21.15.7:0/61149511 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb934039060 0x7fb93403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb9300098c0 tx=0x7fb930006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.574+0000 7fb93f7fe640 1 -- 172.21.15.7:0/61149511 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb93803f070 con 0x7fb948102730 2024-09-10T10:37:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.754+0000 7fb94ca2e640 1 -- 172.21.15.7:0/61149511 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb948107fd0 con 0x7fb934039060 2024-09-10T10:37:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.755+0000 7fb93f7fe640 1 -- 172.21.15.7:0/61149511 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fb948107fd0 con 0x7fb934039060 2024-09-10T10:37:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:39.756 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}] 2024-09-10T10:37:39.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.757+0000 7fb94ca2e640 1 -- 172.21.15.7:0/61149511 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb934039060 msgr2=0x7fb93403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:39.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.757+0000 7fb94ca2e640 1 --2- 172.21.15.7:0/61149511 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb934039060 0x7fb93403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb9300098c0 tx=0x7fb930006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:39.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.757+0000 7fb94ca2e640 1 -- 172.21.15.7:0/61149511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb948102730 msgr2=0x7fb948199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.757+0000 7fb94ca2e640 1 --2- 172.21.15.7:0/61149511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb948102730 0x7fb948199410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb938009d30 tx=0x7fb938030c30 comp rx=0 tx=0).stop 2024-09-10T10:37:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.757+0000 7fb94ca2e640 1 -- 172.21.15.7:0/61149511 shutdown_connections 2024-09-10T10:37:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.757+0000 7fb94ca2e640 1 --2- 172.21.15.7:0/61149511 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb934039060 0x7fb93403b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.757+0000 7fb94ca2e640 1 --2- 172.21.15.7:0/61149511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb948102730 0x7fb948199410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.757+0000 7fb94ca2e640 1 -- 172.21.15.7:0/61149511 >> 172.21.15.7:0/61149511 conn(0x7fb948077310 msgr2=0x7fb9481078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.757+0000 7fb94ca2e640 1 -- 172.21.15.7:0/61149511 shutdown_connections 2024-09-10T10:37:39.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:39.757+0000 7fb94ca2e640 1 -- 172.21.15.7:0/61149511 wait complete. 2024-09-10T10:37:40.379 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi133 2024-09-10T10:37:40.379 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:37:40.379 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.conf 2024-09-10T10:37:40.396 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:37:40.396 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:37:40.453 INFO:tasks.cephadm:Adding host smithi133 to orchestrator... 2024-09-10T10:37:40.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch host add smithi133 2024-09-10T10:37:40.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:40.717 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:37:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.147+0000 7fa06787d640 1 -- 172.21.15.7:0/4144176162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa060106570 msgr2=0x7fa060106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.147+0000 7fa06787d640 1 --2- 172.21.15.7:0/4144176162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa060106570 0x7fa060106950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009880 tx=0x7fa05c02a5d0 comp rx=0 tx=0).stop 2024-09-10T10:37:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.147+0000 7fa06787d640 1 -- 172.21.15.7:0/4144176162 shutdown_connections 2024-09-10T10:37:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.147+0000 7fa06787d640 1 --2- 172.21.15.7:0/4144176162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa060106570 0x7fa060106950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.147+0000 7fa06787d640 1 -- 172.21.15.7:0/4144176162 >> 172.21.15.7:0/4144176162 conn(0x7fa0600fc1a0 msgr2=0x7fa0600fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.147+0000 7fa06787d640 1 -- 172.21.15.7:0/4144176162 shutdown_connections 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.147+0000 7fa06787d640 1 -- 172.21.15.7:0/4144176162 wait complete. 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.147+0000 7fa06787d640 1 Processor -- start 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.148+0000 7fa06787d640 1 -- start start 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.148+0000 7fa06787d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa060106570 0x7fa060197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.148+0000 7fa06787d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa06010aba0 con 0x7fa060106570 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.148+0000 7fa065619640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa060106570 0x7fa060197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.148+0000 7fa065619640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa060106570 0x7fa060197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38320/0 (socket says 172.21.15.7:38320) 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.148+0000 7fa065619640 1 -- 172.21.15.7:0/2214702167 learned_addr learned my addr 172.21.15.7:0/2214702167 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:41.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.148+0000 7fa065619640 1 -- 172.21.15.7:0/2214702167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0601984f0 con 0x7fa060106570 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.148+0000 7fa065619640 1 --2- 172.21.15.7:0/2214702167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa060106570 0x7fa060197fb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009230 tx=0x7fa05c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.149+0000 7fa0567fc640 1 -- 172.21.15.7:0/2214702167 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa05c03f070 con 0x7fa060106570 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.149+0000 7fa0567fc640 1 -- 172.21.15.7:0/2214702167 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05c03bd40 con 0x7fa060106570 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.149+0000 7fa06787d640 1 -- 172.21.15.7:0/2214702167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa060198780 con 0x7fa060106570 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.149+0000 7fa0567fc640 1 -- 172.21.15.7:0/2214702167 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa05c035030 con 0x7fa060106570 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.149+0000 7fa06787d640 1 -- 172.21.15.7:0/2214702167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa060198be0 con 0x7fa060106570 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.149+0000 7fa0567fc640 1 -- 172.21.15.7:0/2214702167 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fa05c042070 con 0x7fa060106570 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.150+0000 7fa06787d640 1 -- 172.21.15.7:0/2214702167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa060101be0 con 0x7fa060106570 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.150+0000 7fa0567fc640 1 --2- 172.21.15.7:0/2214702167 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa04c039060 0x7fa04c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.150+0000 7fa0567fc640 1 -- 172.21.15.7:0/2214702167 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa05c06ccf0 con 0x7fa060106570 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.150+0000 7fa064e18640 1 --2- 172.21.15.7:0/2214702167 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa04c039060 0x7fa04c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:41.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.150+0000 7fa064e18640 1 --2- 172.21.15.7:0/2214702167 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa04c039060 0x7fa04c03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098c0 tx=0x7fa050006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.154+0000 7fa0567fc640 1 -- 172.21.15.7:0/2214702167 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa05c071070 con 0x7fa060106570 2024-09-10T10:37:41.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:41.334+0000 7fa06787d640 1 -- 172.21.15.7:0/2214702167 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}) -- 0x7fa0601069d0 con 0x7fa04c039060 2024-09-10T10:37:41.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:41 smithi007 ceph-mon[30873]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:37:42.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:42 smithi007 ceph-mon[30873]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:42.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:42 smithi007 ceph-mon[30873]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:42.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:42.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:42.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:42.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:42.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:37:42.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:42.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:43.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:43 smithi007 ceph-mon[30873]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T10:37:43.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:43 smithi007 ceph-mon[30873]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:37:43.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:43.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:43.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:44.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:44 smithi007 ceph-mon[30873]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:44.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:44 smithi007 ceph-mon[30873]: Deploying cephadm binary to smithi133 2024-09-10T10:37:44.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:44 smithi007 ceph-mon[30873]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:37:44.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:44 smithi007 ceph-mon[30873]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:37:45.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.361+0000 7fa0567fc640 1 -- 172.21.15.7:0/2214702167 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa0601069d0 con 0x7fa04c039060 2024-09-10T10:37:45.362 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi133' with addr '172.21.15.133' 2024-09-10T10:37:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.363+0000 7fa06787d640 1 -- 172.21.15.7:0/2214702167 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa04c039060 msgr2=0x7fa04c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.363+0000 7fa06787d640 1 --2- 172.21.15.7:0/2214702167 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa04c039060 0x7fa04c03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0500098c0 tx=0x7fa050006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.363+0000 7fa06787d640 1 -- 172.21.15.7:0/2214702167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa060106570 msgr2=0x7fa060197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.363+0000 7fa06787d640 1 --2- 172.21.15.7:0/2214702167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa060106570 0x7fa060197fb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009230 tx=0x7fa05c003d80 comp rx=0 tx=0).stop 2024-09-10T10:37:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.363+0000 7fa06787d640 1 -- 172.21.15.7:0/2214702167 shutdown_connections 2024-09-10T10:37:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.363+0000 7fa06787d640 1 --2- 172.21.15.7:0/2214702167 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa04c039060 0x7fa04c03b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.363+0000 7fa06787d640 1 --2- 172.21.15.7:0/2214702167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa060106570 0x7fa060197fb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.363+0000 7fa06787d640 1 -- 172.21.15.7:0/2214702167 >> 172.21.15.7:0/2214702167 conn(0x7fa0600fc1a0 msgr2=0x7fa0600fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.363+0000 7fa06787d640 1 -- 172.21.15.7:0/2214702167 shutdown_connections 2024-09-10T10:37:45.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:45.363+0000 7fa06787d640 1 -- 172.21.15.7:0/2214702167 wait complete. 2024-09-10T10:37:45.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T10:37:46.216 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:37:46.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:46 smithi007 ceph-mon[30873]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:46.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:46.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:46 smithi007 ceph-mon[30873]: Added host smithi133 2024-09-10T10:37:46.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:46.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:46.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:46.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.687+0000 7fd213142640 1 -- 172.21.15.7:0/2847790242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c107590 msgr2=0x7fd20c107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:46.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.687+0000 7fd213142640 1 --2- 172.21.15.7:0/2847790242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c107590 0x7fd20c107970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd200009880 tx=0x7fd20002a5d0 comp rx=0 tx=0).stop 2024-09-10T10:37:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.687+0000 7fd213142640 1 -- 172.21.15.7:0/2847790242 shutdown_connections 2024-09-10T10:37:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.687+0000 7fd213142640 1 --2- 172.21.15.7:0/2847790242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c107590 0x7fd20c107970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.687+0000 7fd213142640 1 -- 172.21.15.7:0/2847790242 >> 172.21.15.7:0/2847790242 conn(0x7fd20c0fb930 msgr2=0x7fd20c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.687+0000 7fd213142640 1 -- 172.21.15.7:0/2847790242 shutdown_connections 2024-09-10T10:37:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.687+0000 7fd213142640 1 -- 172.21.15.7:0/2847790242 wait complete. 2024-09-10T10:37:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.688+0000 7fd213142640 1 Processor -- start 2024-09-10T10:37:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.688+0000 7fd213142640 1 -- start start 2024-09-10T10:37:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.688+0000 7fd213142640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c107590 0x7fd20c100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.688+0000 7fd213142640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c10bc50 con 0x7fd20c107590 2024-09-10T10:37:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.688+0000 7fd210ede640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c107590 0x7fd20c100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.688+0000 7fd210ede640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c107590 0x7fd20c100cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38332/0 (socket says 172.21.15.7:38332) 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.688+0000 7fd210ede640 1 -- 172.21.15.7:0/2707110354 learned_addr learned my addr 172.21.15.7:0/2707110354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.688+0000 7fd210ede640 1 -- 172.21.15.7:0/2707110354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c102bb0 con 0x7fd20c107590 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.688+0000 7fd210ede640 1 --2- 172.21.15.7:0/2707110354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c107590 0x7fd20c100cc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd200009230 tx=0x7fd200003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.688+0000 7fd209ffb640 1 -- 172.21.15.7:0/2707110354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd20003f070 con 0x7fd20c107590 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.689+0000 7fd209ffb640 1 -- 172.21.15.7:0/2707110354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd20003bd40 con 0x7fd20c107590 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.689+0000 7fd213142640 1 -- 172.21.15.7:0/2707110354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20c101260 con 0x7fd20c107590 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.689+0000 7fd209ffb640 1 -- 172.21.15.7:0/2707110354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd200035030 con 0x7fd20c107590 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.689+0000 7fd213142640 1 -- 172.21.15.7:0/2707110354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20c101720 con 0x7fd20c107590 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.690+0000 7fd209ffb640 1 -- 172.21.15.7:0/2707110354 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fd200042070 con 0x7fd20c107590 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.690+0000 7fd213142640 1 -- 172.21.15.7:0/2707110354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20c1079f0 con 0x7fd20c107590 2024-09-10T10:37:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.690+0000 7fd209ffb640 1 --2- 172.21.15.7:0/2707110354 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1fc039060 0x7fd1fc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.690+0000 7fd209ffb640 1 -- 172.21.15.7:0/2707110354 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd20006ccc0 con 0x7fd20c107590 2024-09-10T10:37:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.690+0000 7fd20bfff640 1 --2- 172.21.15.7:0/2707110354 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1fc039060 0x7fd1fc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.690+0000 7fd20bfff640 1 --2- 172.21.15.7:0/2707110354 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1fc039060 0x7fd1fc03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80098c0 tx=0x7fd1f8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.695+0000 7fd209ffb640 1 -- 172.21.15.7:0/2707110354 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd20003a890 con 0x7fd20c107590 2024-09-10T10:37:46.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.874+0000 7fd213142640 1 -- 172.21.15.7:0/2707110354 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd20c102de0 con 0x7fd1fc039060 2024-09-10T10:37:46.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.875+0000 7fd209ffb640 1 -- 172.21.15.7:0/2707110354 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fd20c102de0 con 0x7fd1fc039060 2024-09-10T10:37:46.876 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:37:46.876 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}] 2024-09-10T10:37:46.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.877+0000 7fd213142640 1 -- 172.21.15.7:0/2707110354 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1fc039060 msgr2=0x7fd1fc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:46.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.877+0000 7fd213142640 1 --2- 172.21.15.7:0/2707110354 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1fc039060 0x7fd1fc03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80098c0 tx=0x7fd1f8006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:46.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.877+0000 7fd213142640 1 -- 172.21.15.7:0/2707110354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c107590 msgr2=0x7fd20c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:46.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.877+0000 7fd213142640 1 --2- 172.21.15.7:0/2707110354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c107590 0x7fd20c100cc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd200009230 tx=0x7fd200003d80 comp rx=0 tx=0).stop 2024-09-10T10:37:46.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.877+0000 7fd213142640 1 -- 172.21.15.7:0/2707110354 shutdown_connections 2024-09-10T10:37:46.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.877+0000 7fd213142640 1 --2- 172.21.15.7:0/2707110354 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1fc039060 0x7fd1fc03b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.877+0000 7fd213142640 1 --2- 172.21.15.7:0/2707110354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd20c107590 0x7fd20c100cc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.877+0000 7fd213142640 1 -- 172.21.15.7:0/2707110354 >> 172.21.15.7:0/2707110354 conn(0x7fd20c0fb930 msgr2=0x7fd20c0fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.877+0000 7fd213142640 1 -- 172.21.15.7:0/2707110354 shutdown_connections 2024-09-10T10:37:46.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:46.877+0000 7fd213142640 1 -- 172.21.15.7:0/2707110354 wait complete. 2024-09-10T10:37:47.598 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T10:37:47.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T10:37:47.850 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:37:47.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:47 smithi007 ceph-mon[30873]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:47.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:47 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:47.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:47 smithi007 ceph-mon[30873]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:37:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.470+0000 7feb800af640 1 -- 172.21.15.7:0/1736890828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb78074590 msgr2=0x7feb78074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.470+0000 7feb800af640 1 --2- 172.21.15.7:0/1736890828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb78074590 0x7feb78074970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb74009880 tx=0x7feb7402a670 comp rx=0 tx=0).stop 2024-09-10T10:37:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.470+0000 7feb800af640 1 -- 172.21.15.7:0/1736890828 shutdown_connections 2024-09-10T10:37:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.470+0000 7feb800af640 1 --2- 172.21.15.7:0/1736890828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb78074590 0x7feb78074970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.470+0000 7feb800af640 1 -- 172.21.15.7:0/1736890828 >> 172.21.15.7:0/1736890828 conn(0x7feb780fd290 msgr2=0x7feb780ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:48.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.470+0000 7feb800af640 1 -- 172.21.15.7:0/1736890828 shutdown_connections 2024-09-10T10:37:48.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.470+0000 7feb800af640 1 -- 172.21.15.7:0/1736890828 wait complete. 2024-09-10T10:37:48.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.471+0000 7feb800af640 1 Processor -- start 2024-09-10T10:37:48.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.471+0000 7feb800af640 1 -- start start 2024-09-10T10:37:48.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.471+0000 7feb800af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb78074590 0x7feb781a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:48.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.471+0000 7feb800af640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feb781122b0 con 0x7feb78074590 2024-09-10T10:37:48.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.471+0000 7feb7de4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb78074590 0x7feb781a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:48.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.471+0000 7feb7de4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb78074590 0x7feb781a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38356/0 (socket says 172.21.15.7:38356) 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.471+0000 7feb7de4b640 1 -- 172.21.15.7:0/4049443031 learned_addr learned my addr 172.21.15.7:0/4049443031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.472+0000 7feb7de4b640 1 -- 172.21.15.7:0/4049443031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb781a3a60 con 0x7feb78074590 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.472+0000 7feb7de4b640 1 --2- 172.21.15.7:0/4049443031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb78074590 0x7feb781a3520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb74009230 tx=0x7feb74003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.472+0000 7feb6effd640 1 -- 172.21.15.7:0/4049443031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb7403f070 con 0x7feb78074590 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.472+0000 7feb6effd640 1 -- 172.21.15.7:0/4049443031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb7403bd40 con 0x7feb78074590 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.472+0000 7feb800af640 1 -- 172.21.15.7:0/4049443031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb781a3cf0 con 0x7feb78074590 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.472+0000 7feb6effd640 1 -- 172.21.15.7:0/4049443031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb74035030 con 0x7feb78074590 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.472+0000 7feb800af640 1 -- 172.21.15.7:0/4049443031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7819d680 con 0x7feb78074590 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.473+0000 7feb6effd640 1 -- 172.21.15.7:0/4049443031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7feb74042070 con 0x7feb78074590 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.473+0000 7feb800af640 1 -- 172.21.15.7:0/4049443031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb78075cd0 con 0x7feb78074590 2024-09-10T10:37:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.473+0000 7feb6effd640 1 --2- 172.21.15.7:0/4049443031 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7feb640390c0 0x7feb6403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:48.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.473+0000 7feb6effd640 1 -- 172.21.15.7:0/4049443031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feb7406cd00 con 0x7feb78074590 2024-09-10T10:37:48.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.473+0000 7feb7d64a640 1 --2- 172.21.15.7:0/4049443031 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7feb640390c0 0x7feb6403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:48.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.474+0000 7feb7d64a640 1 --2- 172.21.15.7:0/4049443031 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7feb640390c0 0x7feb6403b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feb680098c0 tx=0x7feb68006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:48.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.478+0000 7feb6effd640 1 -- 172.21.15.7:0/4049443031 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb7403b860 con 0x7feb78074590 2024-09-10T10:37:48.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.663+0000 7feb800af640 1 -- 172.21.15.7:0/4049443031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7feb780749f0 con 0x7feb78074590 2024-09-10T10:37:48.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.703+0000 7feb6effd640 1 -- 172.21.15.7:0/4049443031 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7feb7403a8e0 con 0x7feb78074590 2024-09-10T10:37:48.705 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-09-10T10:37:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.706+0000 7feb800af640 1 -- 172.21.15.7:0/4049443031 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7feb640390c0 msgr2=0x7feb6403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.706+0000 7feb800af640 1 --2- 172.21.15.7:0/4049443031 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7feb640390c0 0x7feb6403b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feb680098c0 tx=0x7feb68006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.706+0000 7feb800af640 1 -- 172.21.15.7:0/4049443031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb78074590 msgr2=0x7feb781a3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.706+0000 7feb800af640 1 --2- 172.21.15.7:0/4049443031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb78074590 0x7feb781a3520 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb74009230 tx=0x7feb74003d80 comp rx=0 tx=0).stop 2024-09-10T10:37:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.706+0000 7feb800af640 1 -- 172.21.15.7:0/4049443031 shutdown_connections 2024-09-10T10:37:48.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.706+0000 7feb800af640 1 --2- 172.21.15.7:0/4049443031 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7feb640390c0 0x7feb6403b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:48.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.706+0000 7feb800af640 1 --2- 172.21.15.7:0/4049443031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb78074590 0x7feb781a3520 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:48.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.706+0000 7feb800af640 1 -- 172.21.15.7:0/4049443031 >> 172.21.15.7:0/4049443031 conn(0x7feb780fd290 msgr2=0x7feb780fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:48.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.706+0000 7feb800af640 1 -- 172.21.15.7:0/4049443031 shutdown_connections 2024-09-10T10:37:48.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:37:48.706+0000 7feb800af640 1 -- 172.21.15.7:0/4049443031 wait complete. 2024-09-10T10:37:48.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:48 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4049443031' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T10:37:49.322 INFO:tasks.cephadm:Adding mon.a on smithi007 2024-09-10T10:37:49.322 INFO:tasks.cephadm:Adding mon.b on smithi103 2024-09-10T10:37:49.322 INFO:tasks.cephadm:Adding mon.c on smithi133 2024-09-10T10:37:49.322 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi007:172.21.15.7=a;smithi103:172.21.15.103=b;smithi133:172.21.15.133=c' 2024-09-10T10:37:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T10:37:49.703 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T10:37:50.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.157+0000 7f5e5756c640 1 -- 172.21.15.133:0/3878682508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50102130 msgr2=0x7f5e50104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:50.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.157+0000 7f5e5756c640 1 --2- 172.21.15.133:0/3878682508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50102130 0x7f5e50104520 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009880 tx=0x7f5e4c02a5d0 comp rx=0 tx=0).stop 2024-09-10T10:37:50.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.157+0000 7f5e5756c640 1 -- 172.21.15.133:0/3878682508 shutdown_connections 2024-09-10T10:37:50.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.157+0000 7f5e5756c640 1 --2- 172.21.15.133:0/3878682508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50102130 0x7f5e50104520 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:50.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.157+0000 7f5e5756c640 1 -- 172.21.15.133:0/3878682508 >> 172.21.15.133:0/3878682508 conn(0x7f5e500fbcd0 msgr2=0x7f5e500fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.158+0000 7f5e5756c640 1 -- 172.21.15.133:0/3878682508 shutdown_connections 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.158+0000 7f5e5756c640 1 -- 172.21.15.133:0/3878682508 wait complete. 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.158+0000 7f5e5756c640 1 Processor -- start 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.158+0000 7f5e5756c640 1 -- start start 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.158+0000 7f5e5756c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50102130 0x7f5e50196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.158+0000 7f5e5756c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e501076e0 con 0x7f5e50102130 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.159+0000 7f5e55308640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50102130 0x7f5e50196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.159+0000 7f5e55308640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50102130 0x7f5e50196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.133:33756/0 (socket says 172.21.15.133:33756) 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.159+0000 7f5e55308640 1 -- 172.21.15.133:0/2280172686 learned_addr learned my addr 172.21.15.133:0/2280172686 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.159+0000 7f5e55308640 1 -- 172.21.15.133:0/2280172686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e501972e0 con 0x7f5e50102130 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.159+0000 7f5e55308640 1 --2- 172.21.15.133:0/2280172686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50102130 0x7f5e50196da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c004f60 tx=0x7f5e4c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.159+0000 7f5e467fc640 1 -- 172.21.15.133:0/2280172686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e4c03f070 con 0x7f5e50102130 2024-09-10T10:37:50.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.159+0000 7f5e5756c640 1 -- 172.21.15.133:0/2280172686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e50197570 con 0x7f5e50102130 2024-09-10T10:37:50.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.159+0000 7f5e467fc640 1 -- 172.21.15.133:0/2280172686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e4c003ad0 con 0x7f5e50102130 2024-09-10T10:37:50.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.159+0000 7f5e467fc640 1 -- 172.21.15.133:0/2280172686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e4c035030 con 0x7f5e50102130 2024-09-10T10:37:50.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.159+0000 7f5e5756c640 1 -- 172.21.15.133:0/2280172686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5019a220 con 0x7f5e50102130 2024-09-10T10:37:50.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.160+0000 7f5e467fc640 1 -- 172.21.15.133:0/2280172686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f5e4c042070 con 0x7f5e50102130 2024-09-10T10:37:50.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.160+0000 7f5e467fc640 1 --2- 172.21.15.133:0/2280172686 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5e3c039060 0x7f5e3c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:50.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.160+0000 7f5e467fc640 1 -- 172.21.15.133:0/2280172686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5e4c06d020 con 0x7f5e50102130 2024-09-10T10:37:50.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.161+0000 7f5e54b07640 1 --2- 172.21.15.133:0/2280172686 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5e3c039060 0x7f5e3c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:50.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.161+0000 7f5e5756c640 1 -- 172.21.15.133:0/2280172686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e50107e60 con 0x7f5e50102130 2024-09-10T10:37:50.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.161+0000 7f5e54b07640 1 --2- 172.21.15.133:0/2280172686 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5e3c039060 0x7f5e3c03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5e400098c0 tx=0x7f5e40006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:50.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.165+0000 7f5e467fc640 1 -- 172.21.15.133:0/2280172686 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e4c003cf0 con 0x7f5e50102130 2024-09-10T10:37:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4049443031' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T10:37:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:37:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:37:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:50.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:50.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:49 smithi007 ceph-mon[30873]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T10:37:50.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.353+0000 7f5e5756c640 1 -- 172.21.15.133:0/2280172686 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi103:172.21.15.103=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}) -- 0x7f5e500626f0 con 0x7f5e3c039060 2024-09-10T10:37:50.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.358+0000 7f5e467fc640 1 -- 172.21.15.133:0/2280172686 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5e500626f0 con 0x7f5e3c039060 2024-09-10T10:37:50.360 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mon update... 2024-09-10T10:37:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.360+0000 7f5e5756c640 1 -- 172.21.15.133:0/2280172686 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5e3c039060 msgr2=0x7f5e3c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.360+0000 7f5e5756c640 1 --2- 172.21.15.133:0/2280172686 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5e3c039060 0x7f5e3c03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5e400098c0 tx=0x7f5e40006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.360+0000 7f5e5756c640 1 -- 172.21.15.133:0/2280172686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50102130 msgr2=0x7f5e50196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.360+0000 7f5e5756c640 1 --2- 172.21.15.133:0/2280172686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50102130 0x7f5e50196da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c004f60 tx=0x7f5e4c0048f0 comp rx=0 tx=0).stop 2024-09-10T10:37:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.360+0000 7f5e5756c640 1 -- 172.21.15.133:0/2280172686 shutdown_connections 2024-09-10T10:37:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.360+0000 7f5e5756c640 1 --2- 172.21.15.133:0/2280172686 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5e3c039060 0x7f5e3c03b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.360+0000 7f5e5756c640 1 --2- 172.21.15.133:0/2280172686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e50102130 0x7f5e50196da0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.360+0000 7f5e5756c640 1 -- 172.21.15.133:0/2280172686 >> 172.21.15.133:0/2280172686 conn(0x7f5e500fbcd0 msgr2=0x7f5e500fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.360+0000 7f5e5756c640 1 -- 172.21.15.133:0/2280172686 shutdown_connections 2024-09-10T10:37:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:50.360+0000 7f5e5756c640 1 -- 172.21.15.133:0/2280172686 wait complete. 2024-09-10T10:37:50.949 DEBUG:teuthology.orchestra.run.smithi103:mon.b> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T10:37:50.952 DEBUG:teuthology.orchestra.run.smithi133:mon.c> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T10:37:50.954 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T10:37:50.955 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T10:37:51.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:37:51.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:37:51.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi103:172.21.15.103=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi103:172.21.15.103=b;smithi133:172.21.15.133=c;count:3 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:37:51.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:37:51.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:51 smithi007 ceph-mon[30873]: Deploying daemon mon.c on smithi133 2024-09-10T10:37:51.954 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:37:52.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:52 smithi007 ceph-mon[30873]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:52.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.960+0000 7fd048c54640 1 -- 172.21.15.133:0/4145380942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441075e0 msgr2=0x7fd0441079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:52.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.960+0000 7fd048c54640 1 --2- 172.21.15.133:0/4145380942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441075e0 0x7fd0441079c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd034009880 tx=0x7fd03402a5d0 comp rx=0 tx=0).stop 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.961+0000 7fd048c54640 1 -- 172.21.15.133:0/4145380942 shutdown_connections 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.961+0000 7fd048c54640 1 --2- 172.21.15.133:0/4145380942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441075e0 0x7fd0441079c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.961+0000 7fd048c54640 1 -- 172.21.15.133:0/4145380942 >> 172.21.15.133:0/4145380942 conn(0x7fd0440fb8a0 msgr2=0x7fd0440fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.961+0000 7fd048c54640 1 -- 172.21.15.133:0/4145380942 shutdown_connections 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.961+0000 7fd048c54640 1 -- 172.21.15.133:0/4145380942 wait complete. 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.961+0000 7fd048c54640 1 Processor -- start 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.961+0000 7fd048c54640 1 -- start start 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.962+0000 7fd048c54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441075e0 0x7fd044197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.962+0000 7fd048c54640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd04410bb90 con 0x7fd0441075e0 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.962+0000 7fd04259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441075e0 0x7fd044197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.962+0000 7fd04259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441075e0 0x7fd044197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.133:34380/0 (socket says 172.21.15.133:34380) 2024-09-10T10:37:52.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.962+0000 7fd04259c640 1 -- 172.21.15.133:0/453924777 learned_addr learned my addr 172.21.15.133:0/453924777 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:37:52.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.962+0000 7fd04259c640 1 -- 172.21.15.133:0/453924777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd044197560 con 0x7fd0441075e0 2024-09-10T10:37:52.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.962+0000 7fd04259c640 1 --2- 172.21.15.133:0/453924777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441075e0 0x7fd044197020 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd034009230 tx=0x7fd034003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:52.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.963+0000 7fd0337fe640 1 -- 172.21.15.133:0/453924777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd03403f070 con 0x7fd0441075e0 2024-09-10T10:37:52.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.963+0000 7fd048c54640 1 -- 172.21.15.133:0/453924777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd044197850 con 0x7fd0441075e0 2024-09-10T10:37:52.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.963+0000 7fd0337fe640 1 -- 172.21.15.133:0/453924777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd03403bc80 con 0x7fd0441075e0 2024-09-10T10:37:52.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.963+0000 7fd0337fe640 1 -- 172.21.15.133:0/453924777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd034035030 con 0x7fd0441075e0 2024-09-10T10:37:52.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.963+0000 7fd048c54640 1 -- 172.21.15.133:0/453924777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04419b8f0 con 0x7fd0441075e0 2024-09-10T10:37:52.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.964+0000 7fd0337fe640 1 -- 172.21.15.133:0/453924777 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fd034042070 con 0x7fd0441075e0 2024-09-10T10:37:52.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.964+0000 7fd048c54640 1 -- 172.21.15.133:0/453924777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd044107a40 con 0x7fd0441075e0 2024-09-10T10:37:52.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.964+0000 7fd0337fe640 1 --2- 172.21.15.133:0/453924777 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd02c039060 0x7fd02c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:37:52.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.964+0000 7fd0337fe640 1 -- 172.21.15.133:0/453924777 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd03406ca50 con 0x7fd0441075e0 2024-09-10T10:37:52.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.964+0000 7fd041d9b640 1 --2- 172.21.15.133:0/453924777 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd02c039060 0x7fd02c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:37:52.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.965+0000 7fd041d9b640 1 --2- 172.21.15.133:0/453924777 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd02c039060 0x7fd02c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0380098c0 tx=0x7fd038006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:37:52.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:52.969+0000 7fd0337fe640 1 -- 172.21.15.133:0/453924777 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd034031570 con 0x7fd0441075e0 2024-09-10T10:37:53.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.198+0000 7fd048c54640 1 -- 172.21.15.133:0/453924777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd04419bc30 con 0x7fd0441075e0 2024-09-10T10:37:53.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.199+0000 7fd0337fe640 1 -- 172.21.15.133:0/453924777 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7fd034044450 con 0x7fd0441075e0 2024-09-10T10:37:53.200 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:37:53.200 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":1,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","modified":"2024-09-10T10:36:29.861118Z","created":"2024-09-10T10:36:29.861118Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T10:37:53.201 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 1 2024-09-10T10:37:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.201+0000 7fd048c54640 1 -- 172.21.15.133:0/453924777 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd02c039060 msgr2=0x7fd02c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:53.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.201+0000 7fd048c54640 1 --2- 172.21.15.133:0/453924777 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd02c039060 0x7fd02c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0380098c0 tx=0x7fd038006db0 comp rx=0 tx=0).stop 2024-09-10T10:37:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.201+0000 7fd048c54640 1 -- 172.21.15.133:0/453924777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441075e0 msgr2=0x7fd044197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:37:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.201+0000 7fd048c54640 1 --2- 172.21.15.133:0/453924777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441075e0 0x7fd044197020 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd034009230 tx=0x7fd034003d40 comp rx=0 tx=0).stop 2024-09-10T10:37:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.201+0000 7fd048c54640 1 -- 172.21.15.133:0/453924777 shutdown_connections 2024-09-10T10:37:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.201+0000 7fd048c54640 1 --2- 172.21.15.133:0/453924777 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd02c039060 0x7fd02c03b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.201+0000 7fd048c54640 1 --2- 172.21.15.133:0/453924777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0441075e0 0x7fd044197020 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:37:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.201+0000 7fd048c54640 1 -- 172.21.15.133:0/453924777 >> 172.21.15.133:0/453924777 conn(0x7fd0440fb8a0 msgr2=0x7fd0440fbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:37:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.201+0000 7fd048c54640 1 -- 172.21.15.133:0/453924777 shutdown_connections 2024-09-10T10:37:53.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:37:53.201+0000 7fd048c54640 1 -- 172.21.15.133:0/453924777 wait complete. 2024-09-10T10:37:53.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:53 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/453924777' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:37:54.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:37:54 smithi007 ceph-mon[30873]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:37:55.255 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T10:37:55.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T10:37:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:38:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.414+0000 7f3593610640 1 -- 172.21.15.133:0/559121124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3578003530 msgr2=0x7f35780059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.414+0000 7f3593610640 1 --2- 172.21.15.133:0/559121124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3578003530 0x7f35780059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f358c1016a0 tx=0x7f358802c490 comp rx=0 tx=0).stop 2024-09-10T10:38:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.414+0000 7f3593610640 1 -- 172.21.15.133:0/559121124 shutdown_connections 2024-09-10T10:38:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.414+0000 7f3593610640 1 --2- 172.21.15.133:0/559121124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3578003530 0x7f35780059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.414+0000 7f3593610640 1 -- 172.21.15.133:0/559121124 >> 172.21.15.133:0/559121124 conn(0x7f358c0fc220 msgr2=0x7f358c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:00.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.414+0000 7f3593610640 1 -- 172.21.15.133:0/559121124 shutdown_connections 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.414+0000 7f3593610640 1 -- 172.21.15.133:0/559121124 wait complete. 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.415+0000 7f3593610640 1 Processor -- start 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.415+0000 7f3593610640 1 -- start start 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3593610640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3578003530 0x7f358c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3593610640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358c1977e0 0x7f358c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3593610640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f358c108670 con 0x7f3578003530 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3593610640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f358c1084f0 con 0x7f358c1977e0 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3590bab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358c1977e0 0x7f358c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3590bab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358c1977e0 0x7f358c19bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37214/0 (socket says 172.21.15.133:37214) 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3590bab640 1 -- 172.21.15.133:0/1148599844 learned_addr learned my addr 172.21.15.133:0/1148599844 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f35913ac640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3578003530 0x7f358c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:00.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3590bab640 1 -- 172.21.15.133:0/1148599844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358c1977e0 msgr2=0x7f358c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T10:38:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3590bab640 1 -- 172.21.15.133:0/1148599844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358c1977e0 msgr2=0x7f358c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T10:38:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3590bab640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358c1977e0 0x7f358c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:38:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f3590bab640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358c1977e0 0x7f358c19bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:38:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f35913ac640 1 -- 172.21.15.133:0/1148599844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358c1977e0 msgr2=0x7f358c19bb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:38:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f35913ac640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358c1977e0 0x7f358c19bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.416+0000 7f35913ac640 1 -- 172.21.15.133:0/1148599844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c197e00 con 0x7f3578003530 2024-09-10T10:38:00.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.417+0000 7f35913ac640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3578003530 0x7f358c1972a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f358802ca80 tx=0x7f35880339a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:00.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.417+0000 7f35827fc640 1 -- 172.21.15.133:0/1148599844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f35880329e0 con 0x7f3578003530 2024-09-10T10:38:00.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.417+0000 7f3593610640 1 -- 172.21.15.133:0/1148599844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358c19c0b0 con 0x7f3578003530 2024-09-10T10:38:00.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.417+0000 7f35827fc640 1 -- 172.21.15.133:0/1148599844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3588005520 con 0x7f3578003530 2024-09-10T10:38:00.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.417+0000 7f35827fc640 1 -- 172.21.15.133:0/1148599844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3588004bc0 con 0x7f3578003530 2024-09-10T10:38:00.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.417+0000 7f3593610640 1 -- 172.21.15.133:0/1148599844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358c19c640 con 0x7f3578003530 2024-09-10T10:38:00.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.418+0000 7f35827fc640 1 -- 172.21.15.133:0/1148599844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f3588043020 con 0x7f3578003530 2024-09-10T10:38:00.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.419+0000 7f35827fc640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3570038e00 0x7f357003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:00.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.419+0000 7f35827fc640 1 -- 172.21.15.133:0/1148599844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3588041020 con 0x7f3578003530 2024-09-10T10:38:00.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.419+0000 7f3593610640 1 -- 172.21.15.133:0/1148599844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358c19c9b0 con 0x7f3578003530 2024-09-10T10:38:00.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.419+0000 7f3590bab640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3570038e00 0x7f357003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:00.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.419+0000 7f3590bab640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3570038e00 0x7f357003b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f357c009100 tx=0x7f357c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:00.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.425+0000 7f35827fc640 1 -- 172.21.15.133:0/1148599844 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f358804a020 con 0x7f3578003530 2024-09-10T10:38:00.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.660+0000 7f3593610640 1 -- 172.21.15.133:0/1148599844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f358c198600 con 0x7f3578003530 2024-09-10T10:38:00.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.661+0000 7f35827fc640 1 -- 172.21.15.133:0/1148599844 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f358803abb0 con 0x7f3578003530 2024-09-10T10:38:00.663 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:38:00.663 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","modified":"2024-09-10T10:37:55.125716Z","created":"2024-09-10T10:36:29.861118Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T10:38:00.664 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 2 2024-09-10T10:38:00.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 -- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3570038e00 msgr2=0x7f357003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:00.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3570038e00 0x7f357003b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f357c009100 tx=0x7f357c008040 comp rx=0 tx=0).stop 2024-09-10T10:38:00.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 -- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3578003530 msgr2=0x7f358c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:00.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3578003530 0x7f358c1972a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f358802ca80 tx=0x7f35880339a0 comp rx=0 tx=0).stop 2024-09-10T10:38:00.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 -- 172.21.15.133:0/1148599844 shutdown_connections 2024-09-10T10:38:00.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3570038e00 0x7f357003b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:00.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f358c1977e0 0x7f358c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:00.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 --2- 172.21.15.133:0/1148599844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3578003530 0x7f358c1972a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:00.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 -- 172.21.15.133:0/1148599844 >> 172.21.15.133:0/1148599844 conn(0x7f358c0fc220 msgr2=0x7f358c0fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:00.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 -- 172.21.15.133:0/1148599844 shutdown_connections 2024-09-10T10:38:00.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:00.663+0000 7f3593610640 1 -- 172.21.15.133:0/1148599844 wait complete. 2024-09-10T10:38:00.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: mon.a calling monitor election 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: mon.c calling monitor election 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:00.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: monmap epoch 2 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: last_changed 2024-09-10T10:37:55.125716+0000 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: created 2024-09-10T10:36:29.861118+0000 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: min_mon_release 19 (squid) 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: election_strategy: 1 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: fsmap 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: mgrmap e13: x(active, since 39s) 2024-09-10T10:38:00.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: overall HEALTH_OK 2024-09-10T10:38:00.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:00.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:00.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:00.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:38:00.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:01.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:01 smithi007 ceph-mon[30873]: Deploying daemon mon.b on smithi103 2024-09-10T10:38:01.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:01 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/1148599844' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:38:01.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:02.262 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T10:38:02.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T10:38:02.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:02 smithi007 ceph-mon[30873]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:02.434 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:38:02 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:38:02.126+0000 7fe03e830640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T10:38:02.516 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:38:02.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 -- 172.21.15.133:0/376417828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4107660 msgr2=0x7f0ed4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:02.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 --2- 172.21.15.133:0/376417828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4107660 0x7f0ed4107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009880 tx=0x7f0ebc02a5d0 comp rx=0 tx=0).stop 2024-09-10T10:38:02.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 -- 172.21.15.133:0/376417828 shutdown_connections 2024-09-10T10:38:02.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 --2- 172.21.15.133:0/376417828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4107660 0x7f0ed4107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:02.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 -- 172.21.15.133:0/376417828 >> 172.21.15.133:0/376417828 conn(0x7f0ed40fd320 msgr2=0x7f0ed40ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:02.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 -- 172.21.15.133:0/376417828 shutdown_connections 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 -- 172.21.15.133:0/376417828 wait complete. 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 Processor -- start 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 -- start start 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed4107660 0x7f0ed41973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4197900 0x7f0ed419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed410bd20 con 0x7f0ed4197900 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.948+0000 7f0ed90cd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed410bba0 con 0x7f0ed4107660 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed4107660 0x7f0ed41973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4197900 0x7f0ed419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed4107660 0x7f0ed41973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57444/0 (socket says 172.21.15.133:57444) 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed2d9d640 1 -- 172.21.15.133:0/632715837 learned_addr learned my addr 172.21.15.133:0/632715837 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:38:02.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4197900 0x7f0ed419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.133:36244/0 (socket says 172.21.15.133:36244) 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed2d9d640 1 -- 172.21.15.133:0/632715837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed4107660 msgr2=0x7f0ed41973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed2d9d640 1 -- 172.21.15.133:0/632715837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed4107660 msgr2=0x7f0ed41973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed2d9d640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed4107660 0x7f0ed41973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed2d9d640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed4107660 0x7f0ed41973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed259c640 1 -- 172.21.15.133:0/632715837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed4107660 msgr2=0x7f0ed41973c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed259c640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed4107660 0x7f0ed41973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed259c640 1 -- 172.21.15.133:0/632715837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed4197f20 con 0x7f0ed4197900 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed259c640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4197900 0x7f0ed419bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400b3d0 tx=0x7f0ec400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ecbfff640 1 -- 172.21.15.133:0/632715837 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0ec4011630 con 0x7f0ed4197900 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ecbfff640 1 -- 172.21.15.133:0/632715837 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ec4011cb0 con 0x7f0ed4197900 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed90cd640 1 -- 172.21.15.133:0/632715837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed419c1d0 con 0x7f0ed4197900 2024-09-10T10:38:02.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ecbfff640 1 -- 172.21.15.133:0/632715837 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0ec4010df0 con 0x7f0ed4197900 2024-09-10T10:38:02.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.949+0000 7f0ed90cd640 1 -- 172.21.15.133:0/632715837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed419c710 con 0x7f0ed4197900 2024-09-10T10:38:02.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.950+0000 7f0ecbfff640 1 -- 172.21.15.133:0/632715837 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0ec4018020 con 0x7f0ed4197900 2024-09-10T10:38:02.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.950+0000 7f0ed90cd640 1 -- 172.21.15.133:0/632715837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed4102da0 con 0x7f0ed4197900 2024-09-10T10:38:02.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.951+0000 7f0ecbfff640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0ec0038e00 0x7f0ec003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:02.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.951+0000 7f0ecbfff640 1 -- 172.21.15.133:0/632715837 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0ec404be20 con 0x7f0ed4197900 2024-09-10T10:38:02.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.951+0000 7f0ed2d9d640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0ec0038e00 0x7f0ec003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:02.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.951+0000 7f0ed2d9d640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0ec0038e00 0x7f0ec003b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0096c0 tx=0x7f0ebc005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:02.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:02.956+0000 7f0ecbfff640 1 -- 172.21.15.133:0/632715837 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ec401f020 con 0x7f0ed4197900 2024-09-10T10:38:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.186+0000 7f0ed90cd640 1 -- 172.21.15.133:0/632715837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0ed419ca20 con 0x7f0ed4197900 2024-09-10T10:38:03.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.187+0000 7f0ecbfff640 1 -- 172.21.15.133:0/632715837 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f0ec4018310 con 0x7f0ed4197900 2024-09-10T10:38:03.189 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:38:03.190 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","modified":"2024-09-10T10:37:55.125716Z","created":"2024-09-10T10:36:29.861118Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T10:38:03.190 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 2 2024-09-10T10:38:03.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 -- 172.21.15.133:0/632715837 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0ec0038e00 msgr2=0x7f0ec003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0ec0038e00 0x7f0ec003b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0096c0 tx=0x7f0ebc005a40 comp rx=0 tx=0).stop 2024-09-10T10:38:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 -- 172.21.15.133:0/632715837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4197900 msgr2=0x7f0ed419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4197900 0x7f0ed419bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400b3d0 tx=0x7f0ec400b890 comp rx=0 tx=0).stop 2024-09-10T10:38:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 -- 172.21.15.133:0/632715837 shutdown_connections 2024-09-10T10:38:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0ec0038e00 0x7f0ec003b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ed4197900 0x7f0ed419bc90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 --2- 172.21.15.133:0/632715837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ed4107660 0x7f0ed41973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 -- 172.21.15.133:0/632715837 >> 172.21.15.133:0/632715837 conn(0x7f0ed40fd320 msgr2=0x7f0ed40fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 -- 172.21.15.133:0/632715837 shutdown_connections 2024-09-10T10:38:03.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:03.189+0000 7f0ed90cd640 1 -- 172.21.15.133:0/632715837 wait complete. 2024-09-10T10:38:03.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:03 smithi133 ceph-mon[35282]: from='client.? 172.21.15.133:0/632715837' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:38:03.633 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:03 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/632715837' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:38:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:03 smithi103 systemd[1]: Started Ceph mon.b for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:38:04.759 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T10:38:04.759 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T10:38:05.012 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:38:09.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.023+0000 7f55b1051640 1 -- 172.21.15.133:0/3315376126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5594003530 msgr2=0x7f55940059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:09.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.023+0000 7f55b1051640 1 --2- 172.21.15.133:0/3315376126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5594003530 0x7f55940059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55ac102af0 tx=0x7f559c02c3f0 comp rx=0 tx=0).stop 2024-09-10T10:38:09.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.024+0000 7f55b1051640 1 -- 172.21.15.133:0/3315376126 shutdown_connections 2024-09-10T10:38:09.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.024+0000 7f55b1051640 1 --2- 172.21.15.133:0/3315376126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5594003530 0x7f55940059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:09.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.024+0000 7f55b1051640 1 -- 172.21.15.133:0/3315376126 >> 172.21.15.133:0/3315376126 conn(0x7f55ac0fd320 msgr2=0x7f55ac0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:09.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.024+0000 7f55b1051640 1 -- 172.21.15.133:0/3315376126 shutdown_connections 2024-09-10T10:38:09.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.024+0000 7f55b1051640 1 -- 172.21.15.133:0/3315376126 wait complete. 2024-09-10T10:38:09.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55b1051640 1 Processor -- start 2024-09-10T10:38:09.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55b1051640 1 -- start start 2024-09-10T10:38:09.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55b1051640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5594003530 0x7f55ac1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:09.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55b1051640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac197830 0x7f55ac19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:09.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55b1051640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac19c100 0x7f55ac19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:09.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55b1051640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac109720 con 0x7f5594003530 2024-09-10T10:38:09.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55b1051640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac1095a0 con 0x7f55ac19c100 2024-09-10T10:38:09.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55b1051640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac1098a0 con 0x7f55ac197830 2024-09-10T10:38:09.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55aa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac197830 0x7f55ac19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:09.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55aa59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac197830 0x7f55ac19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57446/0 (socket says 172.21.15.133:57446) 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55aa59c640 1 -- 172.21.15.133:0/2947937646 learned_addr learned my addr 172.21.15.133:0/2947937646 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.025+0000 7f55aad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5594003530 0x7f55ac1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55ab59e640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac19c100 0x7f55ac19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55aa59c640 1 -- 172.21.15.133:0/2947937646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac197830 msgr2=0x7f55ac19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55aa59c640 1 -- 172.21.15.133:0/2947937646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac197830 msgr2=0x7f55ac19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55aa59c640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac197830 0x7f55ac19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55aa59c640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac197830 0x7f55ac19bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55ab59e640 1 -- 172.21.15.133:0/2947937646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac19c100 msgr2=0x7f55ac19c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55ab59e640 1 -- 172.21.15.133:0/2947937646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac19c100 msgr2=0x7f55ac19c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55ab59e640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac19c100 0x7f55ac19c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55ab59e640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac19c100 0x7f55ac19c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:38:09.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55aad9d640 1 -- 172.21.15.133:0/2947937646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac197830 msgr2=0x7f55ac19bbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55aad9d640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac197830 0x7f55ac19bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55aad9d640 1 -- 172.21.15.133:0/2947937646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac19c100 msgr2=0x7f55ac19c5b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55aad9d640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac19c100 0x7f55ac19c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55aad9d640 1 -- 172.21.15.133:0/2947937646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ac1a3e70 con 0x7f5594003530 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55aad9d640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5594003530 0x7f55ac1972f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f559c036040 tx=0x7f559c034c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f5593fff640 1 -- 172.21.15.133:0/2947937646 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c0054e0 con 0x7f5594003530 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f5593fff640 1 -- 172.21.15.133:0/2947937646 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f559c003ad0 con 0x7f5594003530 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55b1051640 1 -- 172.21.15.133:0/2947937646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ac074420 con 0x7f5594003530 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f5593fff640 1 -- 172.21.15.133:0/2947937646 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c012a30 con 0x7f5594003530 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.026+0000 7f55b1051640 1 -- 172.21.15.133:0/2947937646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ac0749b0 con 0x7f5594003530 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.027+0000 7f5593fff640 1 -- 172.21.15.133:0/2947937646 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f559c012bd0 con 0x7f5594003530 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.027+0000 7f5593fff640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f55980394f0 0x7f559803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:09.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.027+0000 7f55b1051640 1 -- 172.21.15.133:0/2947937646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ac10a850 con 0x7f5594003530 2024-09-10T10:38:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.027+0000 7f5593fff640 1 -- 172.21.15.133:0/2947937646 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f559c06e520 con 0x7f5594003530 2024-09-10T10:38:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.028+0000 7f55aa59c640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f55980394f0 0x7f559803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.028+0000 7f55aa59c640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f55980394f0 0x7f559803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009100 tx=0x7f55a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:09.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.033+0000 7f5593fff640 1 -- 172.21.15.133:0/2947937646 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f559c044300 con 0x7f5594003530 2024-09-10T10:38:09.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: mon.c calling monitor election 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: mon.a calling monitor election 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: monmap epoch 3 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: last_changed 2024-09-10T10:38:03.825596+0000 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: created 2024-09-10T10:36:29.861118+0000 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: min_mon_release 19 (squid) 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: election_strategy: 1 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: fsmap 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:38:09.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: mgrmap e13: x(active, since 48s) 2024-09-10T10:38:09.187 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: overall HEALTH_OK 2024-09-10T10:38:09.187 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:09.187 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:09.187 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:09.187 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:09.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:09.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: mon.c calling monitor election 2024-09-10T10:38:09.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: mon.a calling monitor election 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: monmap epoch 3 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: last_changed 2024-09-10T10:38:03.825596+0000 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: created 2024-09-10T10:36:29.861118+0000 2024-09-10T10:38:09.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: min_mon_release 19 (squid) 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: election_strategy: 1 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: fsmap 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: mgrmap e13: x(active, since 48s) 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: overall HEALTH_OK 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:09.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:09.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.282+0000 7f55b1051640 1 -- 172.21.15.133:0/2947937646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f55ac074c10 con 0x7f5594003530 2024-09-10T10:38:09.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.283+0000 7f5593fff640 1 -- 172.21.15.133:0/2947937646 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f559c031d40 con 0x7f5594003530 2024-09-10T10:38:09.285 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:38:09.285 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","modified":"2024-09-10T10:38:03.825596Z","created":"2024-09-10T10:36:29.861118Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T10:38:09.285 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 3 2024-09-10T10:38:09.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 -- 172.21.15.133:0/2947937646 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f55980394f0 msgr2=0x7f559803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:09.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f55980394f0 0x7f559803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009100 tx=0x7f55a0008040 comp rx=0 tx=0).stop 2024-09-10T10:38:09.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 -- 172.21.15.133:0/2947937646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5594003530 msgr2=0x7f55ac1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:09.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5594003530 0x7f55ac1972f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f559c036040 tx=0x7f559c034c50 comp rx=0 tx=0).stop 2024-09-10T10:38:09.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 -- 172.21.15.133:0/2947937646 shutdown_connections 2024-09-10T10:38:09.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f55980394f0 0x7f559803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:09.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55ac19c100 0x7f55ac19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:09.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac197830 0x7f55ac19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:09.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 --2- 172.21.15.133:0/2947937646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5594003530 0x7f55ac1972f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:09.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 -- 172.21.15.133:0/2947937646 >> 172.21.15.133:0/2947937646 conn(0x7f55ac0fd320 msgr2=0x7f55ac0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:09.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.286+0000 7f55b1051640 1 -- 172.21.15.133:0/2947937646 shutdown_connections 2024-09-10T10:38:09.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:09.287+0000 7f55b1051640 1 -- 172.21.15.133:0/2947937646 wait complete. 2024-09-10T10:38:09.940 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T10:38:09.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T10:38:10.147 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:09 smithi133 ceph-mon[35282]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:10.148 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:09 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:10.148 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:09 smithi133 ceph-mon[35282]: from='client.? 172.21.15.133:0/2947937646' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:38:10.148 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:09 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:10.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:09 smithi007 ceph-mon[30873]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:10.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:09 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:10.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:09 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/2947937646' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:38:10.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:09 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:10.207 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:38:10.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.610+0000 7f41fcad9640 1 -- 172.21.15.7:0/3246531423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f81027d0 msgr2=0x7f41f8102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:10.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.610+0000 7f41fcad9640 1 --2- 172.21.15.7:0/3246531423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f81027d0 0x7f41f8102bb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009810 tx=0x7f41e802a540 comp rx=0 tx=0).stop 2024-09-10T10:38:10.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.610+0000 7f41fcad9640 1 -- 172.21.15.7:0/3246531423 shutdown_connections 2024-09-10T10:38:10.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.610+0000 7f41fcad9640 1 --2- 172.21.15.7:0/3246531423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f81027d0 0x7f41f8102bb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.610+0000 7f41fcad9640 1 -- 172.21.15.7:0/3246531423 >> 172.21.15.7:0/3246531423 conn(0x7f41f8077310 msgr2=0x7f41f80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.610+0000 7f41fcad9640 1 -- 172.21.15.7:0/3246531423 shutdown_connections 2024-09-10T10:38:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.610+0000 7f41fcad9640 1 -- 172.21.15.7:0/3246531423 wait complete. 2024-09-10T10:38:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.611+0000 7f41fcad9640 1 Processor -- start 2024-09-10T10:38:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.611+0000 7f41fcad9640 1 -- start start 2024-09-10T10:38:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41fcad9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f81027d0 0x7f41f81973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41fcad9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f81978e0 0x7f41f819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41fcad9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41f819c1b0 0x7f41f819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41fcad9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f41f810bc30 con 0x7f41f81027d0 2024-09-10T10:38:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41fcad9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41f810bab0 con 0x7f41f81978e0 2024-09-10T10:38:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41fcad9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41f810bdb0 con 0x7f41f819c1b0 2024-09-10T10:38:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f81027d0 0x7f41f81973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f81027d0 0x7f41f81973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60072/0 (socket says 172.21.15.7:60072) 2024-09-10T10:38:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f659c640 1 -- 172.21.15.7:0/818200000 learned_addr learned my addr 172.21.15.7:0/818200000 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:38:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f5d9b640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f81978e0 0x7f41f819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f6d9d640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41f819c1b0 0x7f41f819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f5d9b640 1 -- 172.21.15.7:0/818200000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f81978e0 msgr2=0x7f41f819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T10:38:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f5d9b640 1 -- 172.21.15.7:0/818200000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f81978e0 msgr2=0x7f41f819bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T10:38:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f5d9b640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f81978e0 0x7f41f819bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:38:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f5d9b640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f81978e0 0x7f41f819bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:38:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f659c640 1 -- 172.21.15.7:0/818200000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41f819c1b0 msgr2=0x7f41f819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f659c640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41f819c1b0 0x7f41f819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f659c640 1 -- 172.21.15.7:0/818200000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f81978e0 msgr2=0x7f41f819bc70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:38:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f659c640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f81978e0 0x7f41f819bc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.612+0000 7f41f659c640 1 -- 172.21.15.7:0/818200000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f81a3f20 con 0x7f41f81027d0 2024-09-10T10:38:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.613+0000 7f41f659c640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f81027d0 0x7f41f81973a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f41e80307a0 tx=0x7f41e80307d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.613+0000 7f41e77fe640 1 -- 172.21.15.7:0/818200000 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8040070 con 0x7f41f81027d0 2024-09-10T10:38:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.613+0000 7f41e77fe640 1 -- 172.21.15.7:0/818200000 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e8004c70 con 0x7f41f81027d0 2024-09-10T10:38:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.613+0000 7f41fcad9640 1 -- 172.21.15.7:0/818200000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f81a40f0 con 0x7f41f81027d0 2024-09-10T10:38:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.613+0000 7f41e77fe640 1 -- 172.21.15.7:0/818200000 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e802f570 con 0x7f41f81027d0 2024-09-10T10:38:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.613+0000 7f41fcad9640 1 -- 172.21.15.7:0/818200000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f81a45a0 con 0x7f41f81027d0 2024-09-10T10:38:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.614+0000 7f41e77fe640 1 -- 172.21.15.7:0/818200000 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f41e8039040 con 0x7f41f81027d0 2024-09-10T10:38:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.614+0000 7f41e77fe640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f41e00394f0 0x7f41e003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:10.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.614+0000 7f41fcad9640 1 -- 172.21.15.7:0/818200000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f8069ef0 con 0x7f41f81027d0 2024-09-10T10:38:10.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.615+0000 7f41e77fe640 1 -- 172.21.15.7:0/818200000 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f41e806d3c0 con 0x7f41f81027d0 2024-09-10T10:38:10.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.615+0000 7f41f5d9b640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f41e00394f0 0x7f41e003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:10.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.615+0000 7f41f5d9b640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f41e00394f0 0x7f41e003b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f41ec006fd0 tx=0x7f41ec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:10.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.619+0000 7f41e77fe640 1 -- 172.21.15.7:0/818200000 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41e8004e20 con 0x7f41f81027d0 2024-09-10T10:38:10.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.800+0000 7f41fcad9640 1 -- 172.21.15.7:0/818200000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f41f81a4800 con 0x7f41f81027d0 2024-09-10T10:38:10.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.800+0000 7f41e77fe640 1 -- 172.21.15.7:0/818200000 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f41e804c2f0 con 0x7f41f81027d0 2024-09-10T10:38:10.802 INFO:teuthology.orchestra.run.smithi007.stdout:# minimal ceph.conf for 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:38:10.802 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-10T10:38:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:38:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: mon_host = [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] 2024-09-10T10:38:10.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.802+0000 7f41fcad9640 1 -- 172.21.15.7:0/818200000 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f41e00394f0 msgr2=0x7f41e003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:10.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.802+0000 7f41fcad9640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f41e00394f0 0x7f41e003b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f41ec006fd0 tx=0x7f41ec008040 comp rx=0 tx=0).stop 2024-09-10T10:38:10.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.802+0000 7f41fcad9640 1 -- 172.21.15.7:0/818200000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f81027d0 msgr2=0x7f41f81973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:10.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.802+0000 7f41fcad9640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f81027d0 0x7f41f81973a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f41e80307a0 tx=0x7f41e80307d0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.802+0000 7f41fcad9640 1 -- 172.21.15.7:0/818200000 shutdown_connections 2024-09-10T10:38:10.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.802+0000 7f41fcad9640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f41e00394f0 0x7f41e003b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.802+0000 7f41fcad9640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41f819c1b0 0x7f41f819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.802+0000 7f41fcad9640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41f81978e0 0x7f41f819bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.802+0000 7f41fcad9640 1 --2- 172.21.15.7:0/818200000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f81027d0 0x7f41f81973a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:10.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.803+0000 7f41fcad9640 1 -- 172.21.15.7:0/818200000 >> 172.21.15.7:0/818200000 conn(0x7f41f8077310 msgr2=0x7f41f8103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:10.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.803+0000 7f41fcad9640 1 -- 172.21.15.7:0/818200000 shutdown_connections 2024-09-10T10:38:10.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:10.803+0000 7f41fcad9640 1 -- 172.21.15.7:0/818200000 wait complete. 2024-09-10T10:38:11.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: mon.c calling monitor election 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: mon.a calling monitor election 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:11.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: monmap epoch 3 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: last_changed 2024-09-10T10:38:03.825596+0000 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: created 2024-09-10T10:36:29.861118+0000 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: min_mon_release 19 (squid) 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: election_strategy: 1 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: fsmap 2024-09-10T10:38:11.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:38:11.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: mgrmap e13: x(active, since 48s) 2024-09-10T10:38:11.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: overall HEALTH_OK 2024-09-10T10:38:11.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:11.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:11.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='client.? 172.21.15.133:0/2947937646' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T10:38:11.175 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:11.472 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T10:38:11.472 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:38:11.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T10:38:11.501 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:38:11.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:38:11.567 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:38:11.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T10:38:11.595 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:38:11.595 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:38:11.661 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:38:11.662 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T10:38:11.689 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:38:11.689 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:38:11.755 INFO:tasks.cephadm:Adding mgr.x on smithi007 2024-09-10T10:38:11.755 INFO:tasks.cephadm:Adding mgr.y on smithi103 2024-09-10T10:38:11.755 INFO:tasks.cephadm:Adding mgr.z on smithi133 2024-09-10T10:38:11.755 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi007=x;smithi103=y;smithi133=z' 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: mon.b calling monitor election 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: mon.b calling monitor election 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: mon.c calling monitor election 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: mon.a calling monitor election 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:38:11.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: monmap epoch 3 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: last_changed 2024-09-10T10:38:03.825596+0000 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: created 2024-09-10T10:36:29.861118+0000 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: min_mon_release 19 (squid) 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: election_strategy: 1 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: fsmap 2024-09-10T10:38:11.912 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: mgrmap e13: x(active, since 50s) 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: overall HEALTH_OK 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.913 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T10:38:11.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:38:11.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:38:11.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:11.914 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:11 smithi007 ceph-mon[30873]: Reconfiguring daemon mon.a on smithi007 2024-09-10T10:38:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: mon.b calling monitor election 2024-09-10T10:38:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-10T10:38:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T10:38:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: mon.b calling monitor election 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: mon.c calling monitor election 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: mon.a calling monitor election 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: monmap epoch 3 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: last_changed 2024-09-10T10:38:03.825596+0000 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: created 2024-09-10T10:36:29.861118+0000 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: min_mon_release 19 (squid) 2024-09-10T10:38:11.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: election_strategy: 1 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: fsmap 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: mgrmap e13: x(active, since 50s) 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: overall HEALTH_OK 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:11.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:11 smithi103 ceph-mon[35160]: Reconfiguring daemon mon.a on smithi007 2024-09-10T10:38:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: mon.b calling monitor election 2024-09-10T10:38:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-10T10:38:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: mon.b calling monitor election 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: mon.c calling monitor election 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: mon.a calling monitor election 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: monmap epoch 3 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: last_changed 2024-09-10T10:38:03.825596+0000 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: created 2024-09-10T10:36:29.861118+0000 2024-09-10T10:38:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: min_mon_release 19 (squid) 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: election_strategy: 1 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: fsmap 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: mgrmap e13: x(active, since 50s) 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: overall HEALTH_OK 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:11.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T10:38:11.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:38:11.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:38:11.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:11.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:11 smithi133 ceph-mon[35282]: Reconfiguring daemon mon.a on smithi007 2024-09-10T10:38:12.034 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:38:12.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.451+0000 7f3767fdb640 1 -- 172.21.15.133:0/1020923459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601005f0 msgr2=0x7f37601009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:12.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.451+0000 7f3767fdb640 1 --2- 172.21.15.133:0/1020923459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601005f0 0x7f37601009f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f375c009880 tx=0x7f375c02a5d0 comp rx=0 tx=0).stop 2024-09-10T10:38:12.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.451+0000 7f3767fdb640 1 -- 172.21.15.133:0/1020923459 shutdown_connections 2024-09-10T10:38:12.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.451+0000 7f3767fdb640 1 --2- 172.21.15.133:0/1020923459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601005f0 0x7f37601009f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:12.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.451+0000 7f3767fdb640 1 -- 172.21.15.133:0/1020923459 >> 172.21.15.133:0/1020923459 conn(0x7f37600fbda0 msgr2=0x7f37600fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:12.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.452+0000 7f3767fdb640 1 -- 172.21.15.133:0/1020923459 shutdown_connections 2024-09-10T10:38:12.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.452+0000 7f3767fdb640 1 -- 172.21.15.133:0/1020923459 wait complete. 2024-09-10T10:38:12.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.452+0000 7f3767fdb640 1 Processor -- start 2024-09-10T10:38:12.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.452+0000 7f3767fdb640 1 -- start start 2024-09-10T10:38:12.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.452+0000 7f3767fdb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601005f0 0x7f376006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:12.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3767fdb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3760199cf0 0x7f376019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:12.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3767fdb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f376019c670 0x7f376019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:12.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3767fdb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37601066e0 con 0x7f37601005f0 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3767fdb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3760106560 con 0x7f376019c670 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3767fdb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3760106860 con 0x7f3760199cf0 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765576640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3760199cf0 0x7f376019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765d77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601005f0 0x7f376006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765d77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601005f0 0x7f376006afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.133:37900/0 (socket says 172.21.15.133:37900) 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765d77640 1 -- 172.21.15.133:0/3474863863 learned_addr learned my addr 172.21.15.133:0/3474863863 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765576640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3760199cf0 0x7f376019c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41174/0 (socket says 172.21.15.133:41174) 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3766578640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f376019c670 0x7f376019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765576640 1 -- 172.21.15.133:0/3474863863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f376019c670 msgr2=0x7f376019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765576640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f376019c670 0x7f376019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765576640 1 -- 172.21.15.133:0/3474863863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601005f0 msgr2=0x7f376006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:12.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765576640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601005f0 0x7f376006afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765576640 1 -- 172.21.15.133:0/3474863863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376019f3d0 con 0x7f3760199cf0 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3766578640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f376019c670 0x7f376019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765d77640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601005f0 0x7f376006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3765576640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3760199cf0 0x7f376019c0e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f375000b450 tx=0x7f375000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3756ffd640 1 -- 172.21.15.133:0/3474863863 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750004230 con 0x7f3760199cf0 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3756ffd640 1 -- 172.21.15.133:0/3474863863 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37500043d0 con 0x7f3760199cf0 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3767fdb640 1 -- 172.21.15.133:0/3474863863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376019f6c0 con 0x7f3760199cf0 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3756ffd640 1 -- 172.21.15.133:0/3474863863 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750003cb0 con 0x7f3760199cf0 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.453+0000 7f3767fdb640 1 -- 172.21.15.133:0/3474863863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376019fc80 con 0x7f3760199cf0 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.454+0000 7f3756ffd640 1 -- 172.21.15.133:0/3474863863 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f375000f050 con 0x7f3760199cf0 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.454+0000 7f3767fdb640 1 -- 172.21.15.133:0/3474863863 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376018f4a0 con 0x7f3760199cf0 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.454+0000 7f3756ffd640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f374c0394f0 0x7f374c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:12.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.455+0000 7f3756ffd640 1 -- 172.21.15.133:0/3474863863 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f375004bdc0 con 0x7f3760199cf0 2024-09-10T10:38:12.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.455+0000 7f3765d77640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f374c0394f0 0x7f374c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:12.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.455+0000 7f3765d77640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f374c0394f0 0x7f374c03b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f375c0314e0 tx=0x7f375c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:12.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.460+0000 7f3756ffd640 1 -- 172.21.15.133:0/3474863863 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f375000d040 con 0x7f3760199cf0 2024-09-10T10:38:12.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.637+0000 7f3767fdb640 1 -- 172.21.15.133:0/3474863863 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi103=y;smithi133=z", "target": ["mon-mgr", ""]}) -- 0x7f37600626c0 con 0x7f374c0394f0 2024-09-10T10:38:12.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.643+0000 7f3756ffd640 1 -- 172.21.15.133:0/3474863863 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f37600626c0 con 0x7f374c0394f0 2024-09-10T10:38:12.644 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mgr update... 2024-09-10T10:38:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 -- 172.21.15.133:0/3474863863 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f374c0394f0 msgr2=0x7f374c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f374c0394f0 0x7f374c03b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f375c0314e0 tx=0x7f375c0058d0 comp rx=0 tx=0).stop 2024-09-10T10:38:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 -- 172.21.15.133:0/3474863863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3760199cf0 msgr2=0x7f376019c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3760199cf0 0x7f376019c0e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f375000b450 tx=0x7f375000b910 comp rx=0 tx=0).stop 2024-09-10T10:38:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 -- 172.21.15.133:0/3474863863 shutdown_connections 2024-09-10T10:38:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f374c0394f0 0x7f374c03b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:12.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f376019c670 0x7f376019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3760199cf0 0x7f376019c0e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 --2- 172.21.15.133:0/3474863863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601005f0 0x7f376006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 -- 172.21.15.133:0/3474863863 >> 172.21.15.133:0/3474863863 conn(0x7f37600fbda0 msgr2=0x7f37600fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 -- 172.21.15.133:0/3474863863 shutdown_connections 2024-09-10T10:38:12.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:38:12.645+0000 7f3767fdb640 1 -- 172.21.15.133:0/3474863863 wait complete. 2024-09-10T10:38:12.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:12.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:12.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:12.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:38:12.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:38:12.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:12.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:12.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:12.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:12.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:38:12.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:38:12.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:12.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:38:12 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:38:12.825+0000 7fe03e830640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T10:38:12.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:38:12.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:12.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:12.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:38:12.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:38:12.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:13.149 DEBUG:teuthology.orchestra.run.smithi103:mgr.y> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T10:38:13.151 DEBUG:teuthology.orchestra.run.smithi133:mgr.z> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T10:38:13.153 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T10:38:13.154 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:38:13.154 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T10:38:13.173 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T10:38:13.173 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-09-10T10:38:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T10:38:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:13.230 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-10T10:38:13.231 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:13.231 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:13.231 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-10 10:37:31.767866416 +0000 2024-09-10T10:38:13.231 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-10 10:32:50.823105354 +0000 2024-09-10T10:38:13.231 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-10 10:32:50.823105354 +0000 2024-09-10T10:38:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-10 10:32:50.823105354 +0000 2024-09-10T10:38:13.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T10:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-10T10:38:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-10T10:38:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00012258 s, 4.2 MB/s 2024-09-10T10:38:13.299 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T10:38:13.357 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-09-10T10:38:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T10:38:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:13.414 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-10T10:38:13.414 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:13.414 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:13.414 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-10 10:37:31.802865902 +0000 2024-09-10T10:38:13.414 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-10 10:32:51.132100839 +0000 2024-09-10T10:38:13.414 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-10 10:32:51.132100839 +0000 2024-09-10T10:38:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-10 10:32:51.132100839 +0000 2024-09-10T10:38:13.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T10:38:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-10T10:38:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-10T10:38:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00011429 s, 4.5 MB/s 2024-09-10T10:38:13.481 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T10:38:13.538 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-09-10T10:38:13.594 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T10:38:13.594 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:13.594 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T10:38:13.594 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:13.595 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:13.595 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-10 10:37:31.839865358 +0000 2024-09-10T10:38:13.595 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-10 10:32:51.454096133 +0000 2024-09-10T10:38:13.595 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-10 10:32:51.454096133 +0000 2024-09-10T10:38:13.595 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-10 10:32:51.454096133 +0000 2024-09-10T10:38:13.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T10:38:13.660 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-10T10:38:13.660 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-10T10:38:13.660 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000111064 s, 4.6 MB/s 2024-09-10T10:38:13.661 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T10:38:13.719 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-09-10T10:38:13.774 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T10:38:13.775 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:13.775 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T10:38:13.775 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:13.775 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:13.775 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-10 10:37:31.873864858 +0000 2024-09-10T10:38:13.775 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-10 10:32:51.788091252 +0000 2024-09-10T10:38:13.775 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-10 10:32:51.788091252 +0000 2024-09-10T10:38:13.775 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-10 10:32:51.788091252 +0000 2024-09-10T10:38:13.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T10:38:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-10T10:38:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-10T10:38:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000105678 s, 4.8 MB/s 2024-09-10T10:38:13.840 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T10:38:13.897 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-10T10:38:13.897 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T10:38:13.913 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T10:38:13.913 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-09-10T10:38:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:13 smithi103 ceph-mon[35160]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:13 smithi103 ceph-mon[35160]: Reconfiguring mon.b (monmap changed)... 2024-09-10T10:38:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:13 smithi103 ceph-mon[35160]: Reconfiguring daemon mon.b on smithi103 2024-09-10T10:38:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:13 smithi103 ceph-mon[35160]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi103=y;smithi133=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:13 smithi103 ceph-mon[35160]: Saving service mgr spec with placement smithi007=x;smithi103=y;smithi133=z;count:3 2024-09-10T10:38:13.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:13 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:13.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:13 smithi007 ceph-mon[30873]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:13.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:13 smithi007 ceph-mon[30873]: Reconfiguring mon.b (monmap changed)... 2024-09-10T10:38:13.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:13 smithi007 ceph-mon[30873]: Reconfiguring daemon mon.b on smithi103 2024-09-10T10:38:13.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:13 smithi007 ceph-mon[30873]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi103=y;smithi133=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:13.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:13 smithi007 ceph-mon[30873]: Saving service mgr spec with placement smithi007=x;smithi103=y;smithi133=z;count:3 2024-09-10T10:38:13.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:13 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:13.937 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T10:38:13.937 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:13.937 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T10:38:13.937 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:13.938 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:13.938 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-10 10:38:03.275251750 +0000 2024-09-10T10:38:13.938 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-10 10:33:07.544385456 +0000 2024-09-10T10:38:13.938 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-10 10:33:07.544385456 +0000 2024-09-10T10:38:13.938 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-10 10:33:07.544385456 +0000 2024-09-10T10:38:13.938 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T10:38:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:13 smithi133 ceph-mon[35282]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:13 smithi133 ceph-mon[35282]: Reconfiguring mon.b (monmap changed)... 2024-09-10T10:38:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:13 smithi133 ceph-mon[35282]: Reconfiguring daemon mon.b on smithi103 2024-09-10T10:38:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:13 smithi133 ceph-mon[35282]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi103=y;smithi133=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:13 smithi133 ceph-mon[35282]: Saving service mgr spec with placement smithi007=x;smithi103=y;smithi133=z;count:3 2024-09-10T10:38:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:13 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:14.001 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-10T10:38:14.002 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-10T10:38:14.002 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000221839 s, 2.3 MB/s 2024-09-10T10:38:14.003 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T10:38:14.060 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-09-10T10:38:14.116 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T10:38:14.116 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T10:38:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-10 10:38:03.275251750 +0000 2024-09-10T10:38:14.116 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-10 10:33:07.886378600 +0000 2024-09-10T10:38:14.117 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-10 10:33:07.886378600 +0000 2024-09-10T10:38:14.117 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-10 10:33:07.886378600 +0000 2024-09-10T10:38:14.117 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T10:38:14.180 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-10T10:38:14.181 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-10T10:38:14.181 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.00019821 s, 2.6 MB/s 2024-09-10T10:38:14.182 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T10:38:14.238 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-09-10T10:38:14.293 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T10:38:14.293 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:14.293 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-10T10:38:14.293 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:14.293 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:14.294 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-10 10:38:03.275251750 +0000 2024-09-10T10:38:14.294 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-10 10:33:08.230371703 +0000 2024-09-10T10:38:14.294 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-10 10:33:08.230371703 +0000 2024-09-10T10:38:14.294 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-10 10:33:08.230371703 +0000 2024-09-10T10:38:14.294 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T10:38:14.357 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-10T10:38:14.357 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-10T10:38:14.357 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000227486 s, 2.3 MB/s 2024-09-10T10:38:14.358 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T10:38:14.415 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-09-10T10:38:14.471 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T10:38:14.471 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:14.471 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-10T10:38:14.471 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:14.471 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:14.471 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-10 10:38:03.275251750 +0000 2024-09-10T10:38:14.471 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-10 10:33:08.549365308 +0000 2024-09-10T10:38:14.471 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-10 10:33:08.549365308 +0000 2024-09-10T10:38:14.471 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-10 10:33:08.549365308 +0000 2024-09-10T10:38:14.472 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T10:38:14.534 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-10T10:38:14.534 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-10T10:38:14.534 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000219477 s, 2.3 MB/s 2024-09-10T10:38:14.535 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T10:38:14.591 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T10:38:14.592 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T10:38:14.608 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T10:38:14.608 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-09-10T10:38:14.664 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T10:38:14.664 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:14.664 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-10T10:38:14.664 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:14.664 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:14.664 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 10:37:54.578038392 +0000 2024-09-10T10:38:14.664 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 10:33:07.253805385 +0000 2024-09-10T10:38:14.664 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 10:33:07.253805385 +0000 2024-09-10T10:38:14.665 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 10:33:07.253805385 +0000 2024-09-10T10:38:14.665 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T10:38:14.728 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T10:38:14.729 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T10:38:14.729 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000340676 s, 1.5 MB/s 2024-09-10T10:38:14.730 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T10:38:14.786 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-09-10T10:38:14.841 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T10:38:14.841 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:14.841 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-10T10:38:14.841 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:14.841 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:14.841 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 10:37:54.578038392 +0000 2024-09-10T10:38:14.841 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 10:33:07.611798728 +0000 2024-09-10T10:38:14.842 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 10:33:07.611798728 +0000 2024-09-10T10:38:14.842 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 10:33:07.611798728 +0000 2024-09-10T10:38:14.842 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T10:38:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T10:38:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T10:38:14.905 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000182736 s, 2.8 MB/s 2024-09-10T10:38:14.906 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T10:38:14.962 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-09-10T10:38:15.017 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T10:38:15.017 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:15.017 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-10T10:38:15.017 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:15.017 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:15.017 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 10:37:54.578038392 +0000 2024-09-10T10:38:15.017 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 10:33:07.941792592 +0000 2024-09-10T10:38:15.018 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 10:33:07.941792592 +0000 2024-09-10T10:38:15.018 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 10:33:07.941792592 +0000 2024-09-10T10:38:15.018 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T10:38:15.080 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T10:38:15.081 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T10:38:15.081 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000195931 s, 2.6 MB/s 2024-09-10T10:38:15.082 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T10:38:15.138 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-09-10T10:38:15.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:15.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:15.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:14 smithi103 ceph-mon[35160]: Reconfiguring mon.c (monmap changed)... 2024-09-10T10:38:15.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:38:15.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:38:15.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:15.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:14 smithi103 ceph-mon[35160]: Reconfiguring daemon mon.c on smithi133 2024-09-10T10:38:15.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:15.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:15.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:14 smithi007 ceph-mon[30873]: Reconfiguring mon.c (monmap changed)... 2024-09-10T10:38:15.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:38:15.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:38:15.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:15.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:14 smithi007 ceph-mon[30873]: Reconfiguring daemon mon.c on smithi133 2024-09-10T10:38:15.194 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T10:38:15.194 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:38:15.194 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T10:38:15.194 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:38:15.194 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:38:15.194 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 10:37:54.579038374 +0000 2024-09-10T10:38:15.195 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 10:33:08.258786698 +0000 2024-09-10T10:38:15.195 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 10:33:08.258786698 +0000 2024-09-10T10:38:15.195 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 10:33:08.258786698 +0000 2024-09-10T10:38:15.195 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T10:38:15.196 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:15.196 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:15.196 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:14 smithi133 ceph-mon[35282]: Reconfiguring mon.c (monmap changed)... 2024-09-10T10:38:15.196 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T10:38:15.196 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T10:38:15.196 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:15.196 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:14 smithi133 ceph-mon[35282]: Reconfiguring daemon mon.c on smithi133 2024-09-10T10:38:15.220 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T10:38:15.220 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T10:38:15.220 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000211599 s, 2.4 MB/s 2024-09-10T10:38:15.222 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T10:38:15.279 INFO:tasks.cephadm:Deploying osd.0 on smithi007 with /dev/vg_nvme/lv_4... 2024-09-10T10:38:15.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T10:38:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:38:16.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:16.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:16.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:16.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:16.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:16.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:16.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:16.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:38:16.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:38:16.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:38:16.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:16.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:16 smithi133 ceph-mon[35282]: Deploying daemon mgr.z on smithi133 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:16.554 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:16 smithi007 ceph-mon[30873]: Deploying daemon mgr.z on smithi133 2024-09-10T10:38:16.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:16.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:16.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:16.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:16.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:16.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:16.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:16.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:38:16.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:38:16.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:38:16.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:16.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:16 smithi103 ceph-mon[35160]: Deploying daemon mgr.z on smithi133 2024-09-10T10:38:17.212 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 systemd[1]: Starting Ceph mgr.z for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:38:17.468 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 podman[37079]: 2024-09-10 10:38:17.113765221 +0000 UTC m=+0.015581800 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:38:17.468 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 podman[37079]: 2024-09-10 10:38:17.236871859 +0000 UTC m=+0.138688428 container create fda7baca4a5d1a35cf4369cb8b1be257bb5f1631e7ff6dead3d06e24bdbd08c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T10:38:17.468 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 podman[37079]: 2024-09-10 10:38:17.439227276 +0000 UTC m=+0.341043846 container init fda7baca4a5d1a35cf4369cb8b1be257bb5f1631e7ff6dead3d06e24bdbd08c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T10:38:17.468 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 podman[37079]: 2024-09-10 10:38:17.441840631 +0000 UTC m=+0.343657202 container start fda7baca4a5d1a35cf4369cb8b1be257bb5f1631e7ff6dead3d06e24bdbd08c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T10:38:17.469 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 ceph-mgr[37098]: -- 172.21.15.133:0/476480763 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610492c10e0 con 0x56104929e800 2024-09-10T10:38:17.725 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 bash[37079]: fda7baca4a5d1a35cf4369cb8b1be257bb5f1631e7ff6dead3d06e24bdbd08c1 2024-09-10T10:38:17.725 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 systemd[1]: Started Ceph mgr.z for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:38:17.725 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:17.563+0000 7ff1fa70b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:38:17.725 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:17.624+0000 7ff1fa70b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:38:17.922 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:38:17.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_4 2024-09-10T10:38:18.128 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:17.773+0000 7ff1fa70b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:38:18.128 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:17.838+0000 7ff1fa70b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:38:18.129 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:17.987+0000 7ff1fa70b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:38:18.129 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:18.127+0000 7ff1fa70b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:38:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:38:18.365 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:18 smithi103 ceph-mon[35160]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:18.365 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:18.365 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:18.365 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:18.366 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:38:18.366 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:38:18.366 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:38:18.366 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:18.446 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-mon[35282]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:18.446 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:18.446 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:18.446 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:18.446 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:38:18.447 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:38:18.447 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:38:18.447 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:18.590 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:18 smithi007 ceph-mon[30873]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:18.590 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:18.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:18.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:18.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:38:18.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T10:38:18.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:38:18.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:18.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/2445972473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b8102690 msgr2=0x7fa7b8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 --2- 172.21.15.7:0/2445972473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b8102690 0x7fa7b8108f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009ca0 tx=0x7fa7b002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:38:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/2445972473 shutdown_connections 2024-09-10T10:38:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 --2- 172.21.15.7:0/2445972473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b8102690 0x7fa7b8108f20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 --2- 172.21.15.7:0/2445972473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b8101cd0 0x7fa7b8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 --2- 172.21.15.7:0/2445972473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b8100ad0 0x7fa7b8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/2445972473 >> 172.21.15.7:0/2445972473 conn(0x7fa7b80fc280 msgr2=0x7fa7b80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/2445972473 shutdown_connections 2024-09-10T10:38:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/2445972473 wait complete. 2024-09-10T10:38:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 Processor -- start 2024-09-10T10:38:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.675+0000 7fa7bf3a4640 1 -- start start 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bf3a4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b8100ad0 0x7fa7b806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bf3a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b8101cd0 0x7fa7b819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bf3a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b8102690 0x7fa7b819f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bf3a4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810c730 con 0x7fa7b8100ad0 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bf3a4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810c5b0 con 0x7fa7b8101cd0 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bf3a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b810c8b0 con 0x7fa7b8102690 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd140640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b8100ad0 0x7fa7b806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd140640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b8100ad0 0x7fa7b806aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60094/0 (socket says 172.21.15.7:60094) 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd140640 1 -- 172.21.15.7:0/589551363 learned_addr learned my addr 172.21.15.7:0/589551363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd941640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b8102690 0x7fa7b819f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bc93f640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b8101cd0 0x7fa7b819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd140640 1 -- 172.21.15.7:0/589551363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b8102690 msgr2=0x7fa7b819f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd140640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b8102690 0x7fa7b819f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd140640 1 -- 172.21.15.7:0/589551363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b8101cd0 msgr2=0x7fa7b819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd140640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b8101cd0 0x7fa7b819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd140640 1 -- 172.21.15.7:0/589551363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b819fb10 con 0x7fa7b8100ad0 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bc93f640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b8101cd0 0x7fa7b819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd941640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b8102690 0x7fa7b819f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bd140640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b8100ad0 0x7fa7b806aad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400d950 tx=0x7fa7b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7ae7fc640 1 -- 172.21.15.7:0/589551363 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4017020 con 0x7fa7b8100ad0 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7ae7fc640 1 -- 172.21.15.7:0/589551363 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4009c30 con 0x7fa7b8100ad0 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7ae7fc640 1 -- 172.21.15.7:0/589551363 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400f720 con 0x7fa7b8100ad0 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/589551363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b819fda0 con 0x7fa7b8100ad0 2024-09-10T10:38:18.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.676+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/589551363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b81a0260 con 0x7fa7b8100ad0 2024-09-10T10:38:18.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.677+0000 7fa7ae7fc640 1 -- 172.21.15.7:0/589551363 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fa7b401a020 con 0x7fa7b8100ad0 2024-09-10T10:38:18.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.678+0000 7fa7ae7fc640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7a4039290 0x7fa7a403b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:18.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.678+0000 7fa7ae7fc640 1 -- 172.21.15.7:0/589551363 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa7b404c2b0 con 0x7fa7b8100ad0 2024-09-10T10:38:18.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.678+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/589551363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b8107ef0 con 0x7fa7b8100ad0 2024-09-10T10:38:18.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.678+0000 7fa7bc93f640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7a4039290 0x7fa7a403b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:18.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.678+0000 7fa7bc93f640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7a4039290 0x7fa7a403b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005fd0 tx=0x7fa7a8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:18.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.683+0000 7fa7ae7fc640 1 -- 172.21.15.7:0/589551363 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7b4021020 con 0x7fa7b8100ad0 2024-09-10T10:38:18.725 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:18.444+0000 7ff1fa70b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:38:18.725 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:18.505+0000 7ff1fa70b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:38:18.725 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:18 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:18.688+0000 7ff1fa70b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:38:18.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:18.867+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/589551363 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa7b8062630 con 0x7fa7a4039290 2024-09-10T10:38:19.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:19 smithi007 ceph-mon[30873]: Deploying daemon mgr.y on smithi103 2024-09-10T10:38:19.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:19 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:19.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:19 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:19.576 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:19 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:19.577 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:19 smithi133 ceph-mon[35282]: Deploying daemon mgr.y on smithi103 2024-09-10T10:38:19.577 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:19 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:19.577 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:19 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:19.577 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:19 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:19.620 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:19 smithi103 systemd[1]: Starting Ceph mgr.y for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:38:19.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:19 smithi103 ceph-mon[35160]: Deploying daemon mgr.y on smithi103 2024-09-10T10:38:19.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:19 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:19.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:19 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:19.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:19 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:19.915 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:19 smithi103 podman[36508]: 2024-09-10 10:38:19.522287495 +0000 UTC m=+0.015973977 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:38:19.915 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:19 smithi103 podman[36508]: 2024-09-10 10:38:19.636694107 +0000 UTC m=+0.130380619 container create 70049c4f604ab28f5e9f8f20e1f193c10703dcbafe0e39e3eaabfbd21ea6dbea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:38:19.915 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:19 smithi103 podman[36508]: 2024-09-10 10:38:19.839032611 +0000 UTC m=+0.332719078 container init 70049c4f604ab28f5e9f8f20e1f193c10703dcbafe0e39e3eaabfbd21ea6dbea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:38:19.915 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:19 smithi103 podman[36508]: 2024-09-10 10:38:19.841834417 +0000 UTC m=+0.335520893 container start 70049c4f604ab28f5e9f8f20e1f193c10703dcbafe0e39e3eaabfbd21ea6dbea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T10:38:19.915 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:19 smithi103 ceph-mgr[36528]: -- 172.21.15.103:0/2509745553 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a8843f0e0 con 0x563a8841d400 2024-09-10T10:38:19.915 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:19 smithi103 bash[36508]: 70049c4f604ab28f5e9f8f20e1f193c10703dcbafe0e39e3eaabfbd21ea6dbea 2024-09-10T10:38:19.975 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:19 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:19.575+0000 7ff1fa70b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:38:19.975 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:19 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:19.643+0000 7ff1fa70b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:38:20.172 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:19 smithi103 systemd[1]: Started Ceph mgr.y for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:38:20.172 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:19 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:19.964+0000 7f4b06bea040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:38:20.172 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:20.027+0000 7f4b06bea040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:38:20.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:20 smithi007 ceph-mon[30873]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:20.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:20 smithi007 ceph-mon[30873]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:20.515 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:20.517 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-mon[35282]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:20.517 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-mon[35282]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:20.517 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.517 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.517 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.517 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.517 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:20.518 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:20.385+0000 7ff1fa70b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:38:20.518 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:20.516+0000 7ff1fa70b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:38:20.572 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:20.182+0000 7f4b06bea040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:38:20.572 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:20.251+0000 7f4b06bea040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:38:20.572 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:20.414+0000 7f4b06bea040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:38:20.572 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:20.569+0000 7f4b06bea040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:38:20.572 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-mon[35160]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:20.572 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-mon[35160]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:20.573 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.573 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.573 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.573 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:20.573 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:20.812 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:20.661+0000 7ff1fa70b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:38:20.812 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:20.727+0000 7ff1fa70b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:38:21.161 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:20.899+0000 7f4b06bea040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:38:21.161 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:20.962+0000 7f4b06bea040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:38:21.187 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:20.884+0000 7ff1fa70b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:38:21.187 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:20.954+0000 7ff1fa70b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:38:21.187 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:21.015+0000 7ff1fa70b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:38:21.187 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:21.185+0000 7ff1fa70b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:38:21.422 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:21.158+0000 7f4b06bea040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:38:21.473 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:21.258+0000 7ff1fa70b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:38:21.473 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:21.328+0000 7ff1fa70b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:38:21.473 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:21.411+0000 7ff1fa70b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:38:21.473 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:21.471+0000 7ff1fa70b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:38:21.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:21.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4102924049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9239e99-3a62-4883-b2f1-8964ebb54492"}]: dispatch 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4102924049' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9239e99-3a62-4883-b2f1-8964ebb54492"}]': finished 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:21.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:21 smithi007 ceph-mon[30873]: Reconfiguring daemon mgr.x on smithi007 2024-09-10T10:38:21.725 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:21.530+0000 7ff1fa70b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:38:21.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:21.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4102924049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9239e99-3a62-4883-b2f1-8964ebb54492"}]: dispatch 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4102924049' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9239e99-3a62-4883-b2f1-8964ebb54492"}]': finished 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T10:38:21.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:38:21.727 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:38:21.727 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:21.727 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:21 smithi133 ceph-mon[35282]: Reconfiguring daemon mgr.x on smithi007 2024-09-10T10:38:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4102924049' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9239e99-3a62-4883-b2f1-8964ebb54492"}]: dispatch 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4102924049' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9239e99-3a62-4883-b2f1-8964ebb54492"}]': finished 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T10:38:21.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:21.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T10:38:21.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T10:38:21.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T10:38:21.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:21.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:21 smithi103 ceph-mon[35160]: Reconfiguring daemon mgr.x on smithi007 2024-09-10T10:38:22.422 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:22 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:22.052+0000 7f4b06bea040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:38:22.422 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:22 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:22.120+0000 7f4b06bea040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:38:22.638 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:38:22 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:38:22.312+0000 7ff1fa70b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:38:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:22.640+0000 7fa7ae7fc640 1 -- 172.21.15.7:0/589551363 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7fa7b403ad10 con 0x7fa7b8100ad0 2024-09-10T10:38:22.922 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:22 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:22.836+0000 7f4b06bea040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:38:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:22 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2470495142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:22 smithi103 ceph-mon[35160]: Standby manager daemon z started 2024-09-10T10:38:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:22 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:38:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:22 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:38:22.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:22 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:38:22.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:22 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:38:22.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:22 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2470495142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:22.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:22 smithi007 ceph-mon[30873]: Standby manager daemon z started 2024-09-10T10:38:22.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:22 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:38:22.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:22 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:38:22.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:22 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:38:22.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:22 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:38:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:22 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2470495142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:22 smithi133 ceph-mon[35282]: Standby manager daemon z started 2024-09-10T10:38:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:22 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:38:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:22 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:38:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:22 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:38:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:22 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.133:0/4107531358' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:38:23.302 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:22 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:22.951+0000 7f4b06bea040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:38:23.302 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.092+0000 7f4b06bea040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:38:23.303 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.152+0000 7f4b06bea040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:38:23.303 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.300+0000 7f4b06bea040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:38:23.587 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.367+0000 7f4b06bea040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:38:23.587 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.427+0000 7f4b06bea040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:38:23.587 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.585+0000 7f4b06bea040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:38:23.862 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-mon[35160]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:23.862 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-mon[35160]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-10T10:38:23.862 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:38:23.863 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.651+0000 7f4b06bea040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:38:23.863 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.717+0000 7f4b06bea040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:38:23.863 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.800+0000 7f4b06bea040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:38:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:23 smithi007 ceph-mon[30873]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:23 smithi007 ceph-mon[30873]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-10T10:38:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:23 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:38:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:23 smithi133 ceph-mon[35282]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:23 smithi133 ceph-mon[35282]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-10T10:38:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:23 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:38:24.172 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.860+0000 7f4b06bea040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:38:24.172 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:23.918+0000 7f4b06bea040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:38:25.172 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:38:24 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:38:24.703+0000 7f4b06bea040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:38:25.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:25.651+0000 7fa7ae7fc640 1 -- 172.21.15.7:0/589551363 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fa7b4024be0 con 0x7fa7b8100ad0 2024-09-10T10:38:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:25 smithi103 ceph-mon[35160]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:25 smithi103 ceph-mon[35160]: Standby manager daemon y started 2024-09-10T10:38:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:25 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:38:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:25 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:38:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:25 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:38:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:25 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:38:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:25 smithi007 ceph-mon[30873]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:25.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:25 smithi007 ceph-mon[30873]: Standby manager daemon y started 2024-09-10T10:38:25.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:25 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:38:25.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:25 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:38:25.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:25 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:38:25.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:25 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:38:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:25 smithi133 ceph-mon[35282]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:25 smithi133 ceph-mon[35282]: Standby manager daemon y started 2024-09-10T10:38:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:25 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:38:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:25 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:38:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:25 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:38:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:25 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.103:0/1435560421' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:38:26.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:26 smithi103 ceph-mon[35160]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-10T10:38:26.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:26 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:38:26.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:26 smithi007 ceph-mon[30873]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-10T10:38:26.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:38:26.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:26 smithi133 ceph-mon[35282]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-10T10:38:26.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:26 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:38:27.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:27 smithi007 ceph-mon[30873]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:27 smithi103 ceph-mon[35160]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:27 smithi133 ceph-mon[35282]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:29.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:29 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:29.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:29 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:29.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:29 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:29.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:29 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:29.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:29 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:29.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:29 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:29 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:29 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:29 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:29.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:29 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:29.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:29 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:29.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:29 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:29 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:29 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:29 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:29 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:29.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:29 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:29.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:29 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:30.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:30 smithi103 ceph-mon[35160]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:30.876 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:30 smithi133 ceph-mon[35282]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:30.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:30 smithi007 ceph-mon[30873]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:31.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:31 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T10:38:31.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:31 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:31 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T10:38:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:31 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:31 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T10:38:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:31 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:32.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:32 smithi007 ceph-mon[30873]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:32.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:32 smithi007 ceph-mon[30873]: Deploying daemon osd.0 on smithi007 2024-09-10T10:38:32.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:32 smithi103 ceph-mon[35160]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:32.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:32 smithi103 ceph-mon[35160]: Deploying daemon osd.0 on smithi007 2024-09-10T10:38:32.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:32 smithi133 ceph-mon[35282]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:32.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:32 smithi133 ceph-mon[35282]: Deploying daemon osd.0 on smithi007 2024-09-10T10:38:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:34 smithi103 ceph-mon[35160]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:34.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:34 smithi007 ceph-mon[30873]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:34.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:34 smithi133 ceph-mon[35282]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:36.759 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:36 smithi007 ceph-mon[30873]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:36 smithi103 ceph-mon[35160]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:36.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:36 smithi133 ceph-mon[35282]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:37.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:37.853 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:37.853 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:38.803 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:38 smithi007 ceph-mon[30873]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:38 smithi103 ceph-mon[35160]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:38.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:38 smithi133 ceph-mon[35282]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:40 smithi103 ceph-mon[35160]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:40.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:40.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:40 smithi007 ceph-mon[30873]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:40.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:40.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:40.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:40.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:40.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:40.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:40 smithi133 ceph-mon[35282]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:40.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:40.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:40.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:40.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:40.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:41.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.402+0000 7fa7ae7fc640 1 -- 172.21.15.7:0/589551363 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa7b8062630 con 0x7fa7a4039290 2024-09-10T10:38:41.405 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 0 on host 'smithi007' 2024-09-10T10:38:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/589551363 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7a4039290 msgr2=0x7fa7a403b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7a4039290 0x7fa7a403b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8005fd0 tx=0x7fa7a8005850 comp rx=0 tx=0).stop 2024-09-10T10:38:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/589551363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b8100ad0 msgr2=0x7fa7b806aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b8100ad0 0x7fa7b806aad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400d950 tx=0x7fa7b400de10 comp rx=0 tx=0).stop 2024-09-10T10:38:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/589551363 shutdown_connections 2024-09-10T10:38:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7a4039290 0x7fa7a403b750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b8102690 0x7fa7b819f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b8101cd0 0x7fa7b819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 --2- 172.21.15.7:0/589551363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b8100ad0 0x7fa7b806aad0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/589551363 >> 172.21.15.7:0/589551363 conn(0x7fa7b80fc280 msgr2=0x7fa7b81050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/589551363 shutdown_connections 2024-09-10T10:38:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:38:41.405+0000 7fa7bf3a4640 1 -- 172.21.15.7:0/589551363 wait complete. 2024-09-10T10:38:41.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:41 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:41.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:41 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:41.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:41 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:41.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:41 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:41.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:41 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:41.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:41 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:41 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:38:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:41 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:41 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:42.069 DEBUG:teuthology.orchestra.run.smithi007:osd.0> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T10:38:42.072 INFO:tasks.cephadm:Deploying osd.1 on smithi103 with /dev/vg_nvme/lv_4... 2024-09-10T10:38:42.072 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T10:38:42.312 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:38:42.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:42 smithi007 ceph-mon[30873]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:42.738 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:42 smithi103 ceph-mon[35160]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:42.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:42 smithi133 ceph-mon[35282]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:43.794 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:43 smithi103 ceph-mon[35160]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:43.794 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:43 smithi103 ceph-mon[35160]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T10:38:43.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:43 smithi007 ceph-mon[30873]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:43.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:43 smithi007 ceph-mon[30873]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T10:38:43.934 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:38:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0[44756]: 2024-09-10T10:38:43.501+0000 7f343069c640 -1 osd.0 0 log_to_monitors true 2024-09-10T10:38:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:43 smithi133 ceph-mon[35282]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:43 smithi133 ceph-mon[35282]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T10:38:44.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:44 smithi103 ceph-mon[35160]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T10:38:44.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:44 smithi103 ceph-mon[35160]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T10:38:44.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:44 smithi103 ceph-mon[35160]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:38:44.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:44.752 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:38:44.777 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-09-10T10:38:44.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:44 smithi007 ceph-mon[30873]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T10:38:44.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:44 smithi007 ceph-mon[30873]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T10:38:44.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:44 smithi007 ceph-mon[30873]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:38:44.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:44.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:44 smithi133 ceph-mon[35282]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T10:38:44.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:44 smithi133 ceph-mon[35282]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T10:38:44.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:44 smithi133 ceph-mon[35282]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:38:44.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:45.032 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:38:45.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.469+0000 7f8edb015640 1 -- 172.21.15.103:0/2720837830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4102760 msgr2=0x7f8ed4104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:45.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.469+0000 7f8edb015640 1 --2- 172.21.15.103:0/2720837830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4102760 0x7f8ed4104b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009900 tx=0x7f8ec802a410 comp rx=0 tx=0).stop 2024-09-10T10:38:45.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.469+0000 7f8edb015640 1 -- 172.21.15.103:0/2720837830 shutdown_connections 2024-09-10T10:38:45.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.469+0000 7f8edb015640 1 --2- 172.21.15.103:0/2720837830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ed41079c0 0x7f8ed4109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:45.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.469+0000 7f8edb015640 1 --2- 172.21.15.103:0/2720837830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed4105090 0x7f8ed4107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.469+0000 7f8edb015640 1 --2- 172.21.15.103:0/2720837830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4102760 0x7f8ed4104b50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.469+0000 7f8edb015640 1 -- 172.21.15.103:0/2720837830 >> 172.21.15.103:0/2720837830 conn(0x7f8ed40fc2a0 msgr2=0x7f8ed40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.469+0000 7f8edb015640 1 -- 172.21.15.103:0/2720837830 shutdown_connections 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.469+0000 7f8edb015640 1 -- 172.21.15.103:0/2720837830 wait complete. 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8edb015640 1 Processor -- start 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8edb015640 1 -- start start 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8edb015640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4102760 0x7f8ed4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8edb015640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ed4105090 0x7f8ed41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8edb015640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed41079c0 0x7f8ed419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8edb015640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410d8d0 con 0x7f8ed41079c0 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8edb015640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410d750 con 0x7f8ed4102760 2024-09-10T10:38:45.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8edb015640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410da50 con 0x7f8ed4105090 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8ed8db1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4102760 0x7f8ed4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8ed8db1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4102760 0x7f8ed4197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60610/0 (socket says 172.21.15.103:60610) 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8ed8db1640 1 -- 172.21.15.103:0/333211637 learned_addr learned my addr 172.21.15.103:0/333211637 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8ed3fff640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ed4105090 0x7f8ed41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8ed95b2640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed41079c0 0x7f8ed419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8ed8db1640 1 -- 172.21.15.103:0/333211637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ed4105090 msgr2=0x7f8ed41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8ed8db1640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ed4105090 0x7f8ed41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8ed8db1640 1 -- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed41079c0 msgr2=0x7f8ed419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8ed8db1640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed41079c0 0x7f8ed419f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.470+0000 7f8ed8db1640 1 -- 172.21.15.103:0/333211637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed419fb90 con 0x7f8ed4102760 2024-09-10T10:38:45.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.471+0000 7f8ed95b2640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed41079c0 0x7f8ed419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.471+0000 7f8ed8db1640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4102760 0x7f8ed4197f60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009230 tx=0x7f8ec8004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.471+0000 7f8ed1ffb640 1 -- 172.21.15.103:0/333211637 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec803f030 con 0x7f8ed4102760 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.471+0000 7f8ed1ffb640 1 -- 172.21.15.103:0/333211637 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec8003ce0 con 0x7f8ed4102760 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.471+0000 7f8edb015640 1 -- 172.21.15.103:0/333211637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed419fe20 con 0x7f8ed4102760 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.471+0000 7f8ed1ffb640 1 -- 172.21.15.103:0/333211637 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8035070 con 0x7f8ed4102760 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.471+0000 7f8edb015640 1 -- 172.21.15.103:0/333211637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41a0270 con 0x7f8ed4102760 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.472+0000 7f8edb015640 1 -- 172.21.15.103:0/333211637 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed40fdd90 con 0x7f8ed4102760 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.473+0000 7f8ed1ffb640 1 -- 172.21.15.103:0/333211637 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f8ec8042030 con 0x7f8ed4102760 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.474+0000 7f8ed1ffb640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8ec40a2ba0 0x7f8ec40a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.474+0000 7f8ed1ffb640 1 -- 172.21.15.103:0/333211637 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f8ec80eda50 con 0x7f8ed4102760 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.474+0000 7f8ed3fff640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8ec40a2ba0 0x7f8ec40a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:38:45.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.475+0000 7f8ed3fff640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8ec40a2ba0 0x7f8ec40a5060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4199480 tx=0x7f8ec000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:38:45.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.478+0000 7f8ed1ffb640 1 -- 172.21.15.103:0/333211637 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ec80bc520 con 0x7f8ed4102760 2024-09-10T10:38:45.557 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0[44756]: 2024-09-10T10:38:45.249+0000 7f342ce33640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T10:38:45.557 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0[44756]: 2024-09-10T10:38:45.252+0000 7f342442e640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:38:45.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:38:45.653+0000 7f8edb015640 1 -- 172.21.15.103:0/333211637 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8ed4062630 con 0x7f8ec40a2ba0 2024-09-10T10:38:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:38:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T10:38:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi007 2024-09-10T10:38:45.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:45.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi007 to 17092M 2024-09-10T10:38:45.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:45.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:45.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:45 smithi103 ceph-mon[35160]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' 2024-09-10T10:38:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:38:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T10:38:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi007 2024-09-10T10:38:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:45.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi007 to 17092M 2024-09-10T10:38:45.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:45.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:45.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:45 smithi007 ceph-mon[30873]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' 2024-09-10T10:38:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:38:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T10:38:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi007 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi007 to 17092M 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:38:45.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:45 smithi133 ceph-mon[35282]: from='osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390]' entity='osd.0' 2024-09-10T10:38:46.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:46 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:38:46.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:46 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:38:46.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:46.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:46 smithi103 ceph-mon[35160]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:46.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:46.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:46.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:46.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:46 smithi103 ceph-mon[35160]: osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] boot 2024-09-10T10:38:46.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:46 smithi103 ceph-mon[35160]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T10:38:46.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:46 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:38:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:46 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:38:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:46 smithi007 ceph-mon[30873]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:46 smithi007 ceph-mon[30873]: osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] boot 2024-09-10T10:38:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:46 smithi007 ceph-mon[30873]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T10:38:46.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:46.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:46 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:38:46.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:46 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:38:46.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:46.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:46 smithi133 ceph-mon[35282]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:38:46.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:38:46.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:38:46.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:46.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:46 smithi133 ceph-mon[35282]: osd.0 [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] boot 2024-09-10T10:38:46.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:46 smithi133 ceph-mon[35282]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T10:38:46.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:38:48.404 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:48 smithi103 ceph-mon[35160]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:48.404 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:48 smithi103 ceph-mon[35160]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T10:38:48.404 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:48 smithi103 ceph-mon[35160]: from='client.? 172.21.15.103:0/543574527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d750ff16-48c8-421d-b46b-49f0d2b7f4e7"}]: dispatch 2024-09-10T10:38:48.404 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:48 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d750ff16-48c8-421d-b46b-49f0d2b7f4e7"}]: dispatch 2024-09-10T10:38:48.404 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:48 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d750ff16-48c8-421d-b46b-49f0d2b7f4e7"}]': finished 2024-09-10T10:38:48.404 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:48 smithi103 ceph-mon[35160]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T10:38:48.404 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:48 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:38:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:48 smithi007 ceph-mon[30873]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:48 smithi007 ceph-mon[30873]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T10:38:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:48 smithi007 ceph-mon[30873]: from='client.? 172.21.15.103:0/543574527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d750ff16-48c8-421d-b46b-49f0d2b7f4e7"}]: dispatch 2024-09-10T10:38:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:48 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d750ff16-48c8-421d-b46b-49f0d2b7f4e7"}]: dispatch 2024-09-10T10:38:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:48 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d750ff16-48c8-421d-b46b-49f0d2b7f4e7"}]': finished 2024-09-10T10:38:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:48 smithi007 ceph-mon[30873]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T10:38:48.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:48 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:38:48.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:48 smithi133 ceph-mon[35282]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:48.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:48 smithi133 ceph-mon[35282]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T10:38:48.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:48 smithi133 ceph-mon[35282]: from='client.? 172.21.15.103:0/543574527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d750ff16-48c8-421d-b46b-49f0d2b7f4e7"}]: dispatch 2024-09-10T10:38:48.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:48 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d750ff16-48c8-421d-b46b-49f0d2b7f4e7"}]: dispatch 2024-09-10T10:38:48.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:48 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d750ff16-48c8-421d-b46b-49f0d2b7f4e7"}]': finished 2024-09-10T10:38:48.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:48 smithi133 ceph-mon[35282]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T10:38:48.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:48 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:38:49.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:49 smithi103 ceph-mon[35160]: from='client.? 172.21.15.103:0/3572306504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:49.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:49 smithi007 ceph-mon[30873]: from='client.? 172.21.15.103:0/3572306504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:49.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:49 smithi133 ceph-mon[35282]: from='client.? 172.21.15.103:0/3572306504' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:38:50.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:50 smithi103 ceph-mon[35160]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:50.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:50 smithi007 ceph-mon[30873]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:50.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:50 smithi133 ceph-mon[35282]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:52.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:52 smithi103 ceph-mon[35160]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:52.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:52 smithi007 ceph-mon[30873]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:52.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:52 smithi133 ceph-mon[35282]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:54.521 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:54 smithi103 ceph-mon[35160]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:54.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:54 smithi007 ceph-mon[30873]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:54.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:54 smithi133 ceph-mon[35282]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:56.549 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:56 smithi103 ceph-mon[35160]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:56.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:56 smithi007 ceph-mon[30873]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:56.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:56 smithi133 ceph-mon[35282]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:57.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T10:38:57.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:57.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T10:38:57.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T10:38:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:38:58.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:58 smithi103 ceph-mon[35160]: Deploying daemon osd.1 on smithi103 2024-09-10T10:38:58.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:38:58 smithi103 ceph-mon[35160]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:58.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:58 smithi007 ceph-mon[30873]: Deploying daemon osd.1 on smithi103 2024-09-10T10:38:58.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:38:58 smithi007 ceph-mon[30873]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:38:58.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:58 smithi133 ceph-mon[35282]: Deploying daemon osd.1 on smithi103 2024-09-10T10:38:58.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:38:58 smithi133 ceph-mon[35282]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:00.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:00 smithi103 ceph-mon[35160]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:00.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:00 smithi007 ceph-mon[30873]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:00.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:00 smithi133 ceph-mon[35282]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:02.578 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:02 smithi103 ceph-mon[35160]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:02.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:02 smithi007 ceph-mon[30873]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:02.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:02 smithi133 ceph-mon[35282]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:03.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:03.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:03.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:03.644 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:03.645 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:03.645 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:03.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:03.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:03.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:04.398 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:04 smithi103 ceph-mon[35160]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:04.602 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:04 smithi007 ceph-mon[30873]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:04 smithi133 ceph-mon[35282]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:06.549 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:06 smithi103 ceph-mon[35160]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:06.549 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:06 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:06.549 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:06 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:06.549 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:06 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:06.550 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:06 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:06.550 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:06 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:06.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:06 smithi007 ceph-mon[30873]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:06.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:06 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:06.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:06 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:06.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:06 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:06.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:06 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:06.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:06 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:06.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:06 smithi133 ceph-mon[35282]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:06.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:06 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:06.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:06 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:06.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:06 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:06.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:06 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:06.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:06 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:06.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.905+0000 7f8ed1ffb640 1 -- 172.21.15.103:0/333211637 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8ed4062630 con 0x7f8ec40a2ba0 2024-09-10T10:39:06.908 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 1 on host 'smithi103' 2024-09-10T10:39:06.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.907+0000 7f8edb015640 1 -- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8ec40a2ba0 msgr2=0x7f8ec40a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.907+0000 7f8edb015640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8ec40a2ba0 0x7f8ec40a5060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4199480 tx=0x7f8ec000b390 comp rx=0 tx=0).stop 2024-09-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.907+0000 7f8edb015640 1 -- 172.21.15.103:0/333211637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4102760 msgr2=0x7f8ed4197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.907+0000 7f8edb015640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4102760 0x7f8ed4197f60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009230 tx=0x7f8ec8004360 comp rx=0 tx=0).stop 2024-09-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.907+0000 7f8edb015640 1 -- 172.21.15.103:0/333211637 shutdown_connections 2024-09-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.907+0000 7f8edb015640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8ec40a2ba0 0x7f8ec40a5060 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.907+0000 7f8edb015640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed41079c0 0x7f8ed419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.907+0000 7f8edb015640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ed4105090 0x7f8ed41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:06.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.907+0000 7f8edb015640 1 --2- 172.21.15.103:0/333211637 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ed4102760 0x7f8ed4197f60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:06.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.908+0000 7f8edb015640 1 -- 172.21.15.103:0/333211637 >> 172.21.15.103:0/333211637 conn(0x7f8ed40fc2a0 msgr2=0x7f8ed41030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:39:06.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.908+0000 7f8edb015640 1 -- 172.21.15.103:0/333211637 shutdown_connections 2024-09-10T10:39:06.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:39:06.908+0000 7f8edb015640 1 -- 172.21.15.103:0/333211637 wait complete. 2024-09-10T10:39:07.478 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:07.478 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:07.478 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:07.505 DEBUG:teuthology.orchestra.run.smithi103:osd.1> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T10:39:07.507 INFO:tasks.cephadm:Deploying osd.2 on smithi133 with /dev/vg_nvme/lv_4... 2024-09-10T10:39:07.507 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T10:39:07.535 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:07.535 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:07.535 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:07.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:07.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:07.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:07.759 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:39:08.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:08 smithi103 ceph-mon[35160]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:08.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:08 smithi133 ceph-mon[35282]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:08.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:08 smithi007 ceph-mon[30873]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:09.271 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:09 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1[39659]: 2024-09-10T10:39:09.113+0000 7f7acc7e9640 -1 osd.1 0 log_to_monitors true 2024-09-10T10:39:09.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:09 smithi103 ceph-mon[35160]: from='osd.1 [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:39:09.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:09 smithi103 ceph-mon[35160]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:39:09.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:09 smithi007 ceph-mon[30873]: from='osd.1 [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:39:09.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:09 smithi007 ceph-mon[30873]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:39:09.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:09 smithi133 ceph-mon[35282]: from='osd.1 [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:39:09.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:09 smithi133 ceph-mon[35282]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T10:39:10.021 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:39:10.046 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2024-09-10T10:39:10.335 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:39:10.532 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:10 smithi133 ceph-mon[35282]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:10.532 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:10 smithi133 ceph-mon[35282]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T10:39:10.533 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:10 smithi133 ceph-mon[35282]: from='osd.1 [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:39:10.533 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:10 smithi133 ceph-mon[35282]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T10:39:10.533 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:10 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:10.533 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:10 smithi133 ceph-mon[35282]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:39:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:10 smithi103 ceph-mon[35160]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:10 smithi103 ceph-mon[35160]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T10:39:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:10 smithi103 ceph-mon[35160]: from='osd.1 [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:39:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:10 smithi103 ceph-mon[35160]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T10:39:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:10 smithi103 ceph-mon[35160]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:39:10.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:10 smithi007 ceph-mon[30873]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:10.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:10 smithi007 ceph-mon[30873]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T10:39:10.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:10 smithi007 ceph-mon[30873]: from='osd.1 [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:39:10.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:10 smithi007 ceph-mon[30873]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T10:39:10.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:10 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:10.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:10 smithi007 ceph-mon[30873]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:39:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.838+0000 7fda0c348640 1 -- 172.21.15.133:0/3798275033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04105050 msgr2=0x7fda04107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.838+0000 7fda0c348640 1 --2- 172.21.15.133:0/3798275033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04105050 0x7fda04107440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009900 tx=0x7fd9f402a410 comp rx=0 tx=0).stop 2024-09-10T10:39:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.838+0000 7fda0c348640 1 -- 172.21.15.133:0/3798275033 shutdown_connections 2024-09-10T10:39:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.838+0000 7fda0c348640 1 --2- 172.21.15.133:0/3798275033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda04107980 0x7fda04109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.838+0000 7fda0c348640 1 --2- 172.21.15.133:0/3798275033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04105050 0x7fda04107440 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:10.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.838+0000 7fda0c348640 1 --2- 172.21.15.133:0/3798275033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04102720 0x7fda04104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.838+0000 7fda0c348640 1 -- 172.21.15.133:0/3798275033 >> 172.21.15.133:0/3798275033 conn(0x7fda040fc280 msgr2=0x7fda040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.839+0000 7fda0c348640 1 -- 172.21.15.133:0/3798275033 shutdown_connections 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.839+0000 7fda0c348640 1 -- 172.21.15.133:0/3798275033 wait complete. 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.839+0000 7fda0c348640 1 Processor -- start 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.839+0000 7fda0c348640 1 -- start start 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.839+0000 7fda0c348640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda04102720 0x7fda04102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.839+0000 7fda0c348640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04105050 0x7fda041006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.839+0000 7fda0c348640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04107980 0x7fda04100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.839+0000 7fda0c348640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda0410d890 con 0x7fda04107980 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.839+0000 7fda0c348640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda0410d710 con 0x7fda04102720 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.839+0000 7fda0c348640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda0410da10 con 0x7fda04105050 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda0a0e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda04102720 0x7fda04102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:10.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda098e3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04105050 0x7fda041006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda0a0e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda04102720 0x7fda04102080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.133:34196/0 (socket says 172.21.15.133:34196) 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda0a0e4640 1 -- 172.21.15.133:0/1427484295 learned_addr learned my addr 172.21.15.133:0/1427484295 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda0a8e5640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04107980 0x7fda04100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda098e3640 1 -- 172.21.15.133:0/1427484295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda04102720 msgr2=0x7fda04102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda098e3640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda04102720 0x7fda04102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda098e3640 1 -- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04107980 msgr2=0x7fda04100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda098e3640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04107980 0x7fda04100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda098e3640 1 -- 172.21.15.133:0/1427484295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda041014d0 con 0x7fda04105050 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda0a8e5640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04107980 0x7fda04100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda098e3640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04105050 0x7fda041006d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009230 tx=0x7fd9f4032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fd9fb7fe640 1 -- 172.21.15.133:0/1427484295 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f403f030 con 0x7fda04105050 2024-09-10T10:39:10.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fd9fb7fe640 1 -- 172.21.15.133:0/1427484295 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f4003ce0 con 0x7fda04105050 2024-09-10T10:39:10.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda0c348640 1 -- 172.21.15.133:0/1427484295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda041a2040 con 0x7fda04105050 2024-09-10T10:39:10.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fd9fb7fe640 1 -- 172.21.15.133:0/1427484295 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4035070 con 0x7fda04105050 2024-09-10T10:39:10.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.840+0000 7fda0c348640 1 -- 172.21.15.133:0/1427484295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda041a24f0 con 0x7fda04105050 2024-09-10T10:39:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.841+0000 7fda0c348640 1 -- 172.21.15.133:0/1427484295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda040fdd70 con 0x7fda04105050 2024-09-10T10:39:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.843+0000 7fd9fb7fe640 1 -- 172.21.15.133:0/1427484295 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fd9f4042030 con 0x7fda04105050 2024-09-10T10:39:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.843+0000 7fd9fb7fe640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd9f00a2af0 0x7fd9f00a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.843+0000 7fd9fb7fe640 1 -- 172.21.15.133:0/1427484295 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fd9f40edc10 con 0x7fda04105050 2024-09-10T10:39:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.844+0000 7fda0a0e4640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd9f00a2af0 0x7fd9f00a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:10.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.844+0000 7fda0a0e4640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd9f00a2af0 0x7fd9f00a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda00005fd0 tx=0x7fda0000b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:39:10.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:10.851+0000 7fd9fb7fe640 1 -- 172.21.15.133:0/1427484295 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9f40bc4e0 con 0x7fda04105050 2024-09-10T10:39:11.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:11.031+0000 7fda0c348640 1 -- 172.21.15.133:0/1427484295 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fda04062630 con 0x7fd9f00a2af0 2024-09-10T10:39:11.424 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1[39659]: 2024-09-10T10:39:11.070+0000 7f7ac7f49640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T10:39:11.424 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1[39659]: 2024-09-10T10:39:11.074+0000 7f7ac057b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:39:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:39:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T10:39:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:11.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi103 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:11.425 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:11 smithi103 ceph-mon[35160]: from='osd.1 ' entity='osd.1' 2024-09-10T10:39:11.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:39:11.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T10:39:11.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:11.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:11.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi103 2024-09-10T10:39:11.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:11.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-10T10:39:11.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.550 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:11.550 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:11.550 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.550 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:11.550 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:11.550 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:11.550 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:11 smithi133 ceph-mon[35282]: from='osd.1 ' entity='osd.1' 2024-09-10T10:39:11.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:39:11.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T10:39:11.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:11.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:11.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi103 2024-09-10T10:39:11.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi103 to 17088M 2024-09-10T10:39:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:11 smithi007 ceph-mon[30873]: from='osd.1 ' entity='osd.1' 2024-09-10T10:39:12.638 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:12 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:39:12.638 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:12 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:39:12.638 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:12 smithi133 ceph-mon[35282]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:12.639 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:12 smithi133 ceph-mon[35282]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:12.639 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:12.639 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:12 smithi133 ceph-mon[35282]: osd.1 [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] boot 2024-09-10T10:39:12.639 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:12 smithi133 ceph-mon[35282]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T10:39:12.639 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:12.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:12 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:39:12.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:12 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:39:12.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:12 smithi103 ceph-mon[35160]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:12.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:12 smithi103 ceph-mon[35160]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:12.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:12.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:12 smithi103 ceph-mon[35160]: osd.1 [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] boot 2024-09-10T10:39:12.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:12 smithi103 ceph-mon[35160]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T10:39:12.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:12.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:12 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:39:12.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:12 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:39:12.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:12 smithi007 ceph-mon[30873]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T10:39:12.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:12 smithi007 ceph-mon[30873]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:12.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:12.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:12 smithi007 ceph-mon[30873]: osd.1 [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] boot 2024-09-10T10:39:12.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:12 smithi007 ceph-mon[30873]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T10:39:12.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:39:14.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:14 smithi103 ceph-mon[35160]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:14.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:14 smithi103 ceph-mon[35160]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T10:39:14.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:14 smithi103 ceph-mon[35160]: from='client.? 172.21.15.133:0/708947303' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ecb070a-a2f1-499a-99e7-f1967ebb52ff"}]: dispatch 2024-09-10T10:39:14.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:14 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ecb070a-a2f1-499a-99e7-f1967ebb52ff"}]: dispatch 2024-09-10T10:39:14.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:14 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ecb070a-a2f1-499a-99e7-f1967ebb52ff"}]': finished 2024-09-10T10:39:14.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:14 smithi103 ceph-mon[35160]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T10:39:14.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:14.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:14 smithi103 ceph-mon[35160]: from='client.? 172.21.15.133:0/2269612679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:14.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:14 smithi007 ceph-mon[30873]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:14.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:14 smithi007 ceph-mon[30873]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T10:39:14.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:14 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/708947303' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ecb070a-a2f1-499a-99e7-f1967ebb52ff"}]: dispatch 2024-09-10T10:39:14.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:14 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ecb070a-a2f1-499a-99e7-f1967ebb52ff"}]: dispatch 2024-09-10T10:39:14.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:14 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ecb070a-a2f1-499a-99e7-f1967ebb52ff"}]': finished 2024-09-10T10:39:14.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:14 smithi007 ceph-mon[30873]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T10:39:14.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:14.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:14 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/2269612679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:14.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:14 smithi133 ceph-mon[35282]: pgmap v59: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:14.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:14 smithi133 ceph-mon[35282]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T10:39:14.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:14 smithi133 ceph-mon[35282]: from='client.? 172.21.15.133:0/708947303' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ecb070a-a2f1-499a-99e7-f1967ebb52ff"}]: dispatch 2024-09-10T10:39:14.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:14 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ecb070a-a2f1-499a-99e7-f1967ebb52ff"}]: dispatch 2024-09-10T10:39:14.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:14 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ecb070a-a2f1-499a-99e7-f1967ebb52ff"}]': finished 2024-09-10T10:39:14.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:14 smithi133 ceph-mon[35282]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T10:39:14.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:14.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:14 smithi133 ceph-mon[35282]: from='client.? 172.21.15.133:0/2269612679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:16.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:16 smithi103 ceph-mon[35160]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:16 smithi007 ceph-mon[30873]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:16.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:16 smithi133 ceph-mon[35282]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:18.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:18 smithi103 ceph-mon[35160]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:18.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:18 smithi007 ceph-mon[30873]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:18.694 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:18 smithi133 ceph-mon[35282]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:20.632 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:20 smithi133 ceph-mon[35282]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:20.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:20 smithi103 ceph-mon[35160]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:20.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:20 smithi007 ceph-mon[30873]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:22.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:22 smithi133 ceph-mon[35282]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:22.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T10:39:22.549 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:22.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:22 smithi103 ceph-mon[35160]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:22.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T10:39:22.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:22.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:22 smithi007 ceph-mon[30873]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:22.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T10:39:22.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:23.619 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:23 smithi133 ceph-mon[35282]: Deploying daemon osd.2 on smithi133 2024-09-10T10:39:23.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:23 smithi103 ceph-mon[35160]: Deploying daemon osd.2 on smithi133 2024-09-10T10:39:23.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:23 smithi007 ceph-mon[30873]: Deploying daemon osd.2 on smithi133 2024-09-10T10:39:24.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:24 smithi133 ceph-mon[35282]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:24.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:24 smithi103 ceph-mon[35160]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:24.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:24 smithi007 ceph-mon[30873]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:26.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:26 smithi103 ceph-mon[35160]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:26.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:26 smithi007 ceph-mon[30873]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:26.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:26 smithi133 ceph-mon[35282]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:28.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:28 smithi133 ceph-mon[35282]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:28.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:28.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:28.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:28.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:28 smithi103 ceph-mon[35160]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:28.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:28.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:28.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:28.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:28 smithi007 ceph-mon[30873]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:28.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:28.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:28.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:30.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:30 smithi103 ceph-mon[35160]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:30 smithi007 ceph-mon[30873]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:30.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:30 smithi133 ceph-mon[35282]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:31.757 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:31 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:31.758 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:31 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:31.758 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:31 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:31.758 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:31 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:31.758 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:31 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:31 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:31 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:31 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:31 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:31.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:31 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:31 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:31.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:31 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:31.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:31 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:31.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:31 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:31.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:31 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:32.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.168+0000 7fd9fb7fe640 1 -- 172.21.15.133:0/1427484295 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fda04062630 con 0x7fd9f00a2af0 2024-09-10T10:39:32.170 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 2 on host 'smithi133' 2024-09-10T10:39:32.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 -- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd9f00a2af0 msgr2=0x7fd9f00a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:32.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd9f00a2af0 0x7fd9f00a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda00005fd0 tx=0x7fda0000b480 comp rx=0 tx=0).stop 2024-09-10T10:39:32.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 -- 172.21.15.133:0/1427484295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04105050 msgr2=0x7fda041006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:32.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04105050 0x7fda041006d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009230 tx=0x7fd9f4032ce0 comp rx=0 tx=0).stop 2024-09-10T10:39:32.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 -- 172.21.15.133:0/1427484295 shutdown_connections 2024-09-10T10:39:32.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd9f00a2af0 0x7fd9f00a4fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:32.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04107980 0x7fda04100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:32.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04105050 0x7fda041006d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:32.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 --2- 172.21.15.133:0/1427484295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda04102720 0x7fda04102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:32.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 -- 172.21.15.133:0/1427484295 >> 172.21.15.133:0/1427484295 conn(0x7fda040fc280 msgr2=0x7fda04103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:39:32.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 -- 172.21.15.133:0/1427484295 shutdown_connections 2024-09-10T10:39:32.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:39:32.171+0000 7fda0c348640 1 -- 172.21.15.133:0/1427484295 wait complete. 2024-09-10T10:39:32.788 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:32 smithi133 ceph-mon[35282]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:32.788 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:32.788 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:32.788 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:32.814 DEBUG:teuthology.orchestra.run.smithi133:osd.2> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T10:39:32.817 INFO:tasks.cephadm:Deploying osd.3 on smithi007 with /dev/vg_nvme/lv_3... 2024-09-10T10:39:32.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T10:39:32.842 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:32 smithi007 ceph-mon[30873]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:32.842 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:32.842 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:32.842 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:32.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:32 smithi103 ceph-mon[35160]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:32.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:32.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:32.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:33.068 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:39:34.380 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:34 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2[40258]: 2024-09-10T10:39:34.289+0000 7f2c4ccf8640 -1 osd.2 0 log_to_monitors true 2024-09-10T10:39:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:34 smithi133 ceph-mon[35282]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:34.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:34 smithi133 ceph-mon[35282]: from='osd.2 [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:39:34.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:34 smithi133 ceph-mon[35282]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:39:34.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:34 smithi007 ceph-mon[30873]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:34.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:34 smithi007 ceph-mon[30873]: from='osd.2 [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:39:34.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:34 smithi007 ceph-mon[30873]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:39:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:34 smithi103 ceph-mon[35160]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:34 smithi103 ceph-mon[35160]: from='osd.2 [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:39:34.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:34 smithi103 ceph-mon[35160]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T10:39:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:39:35.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_3 2024-09-10T10:39:35.594 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:35 smithi133 ceph-mon[35282]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T10:39:35.594 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:35 smithi133 ceph-mon[35282]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T10:39:35.594 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:35 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:35.595 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:35 smithi133 ceph-mon[35282]: from='osd.2 [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:39:35.595 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:35 smithi133 ceph-mon[35282]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:39:35.595 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:35 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2[40258]: 2024-09-10T10:39:35.545+0000 7f2c48458640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T10:39:35.595 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:35 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2[40258]: 2024-09-10T10:39:35.549+0000 7f2c44a92640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:39:35.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:35 smithi007 ceph-mon[30873]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T10:39:35.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:35 smithi007 ceph-mon[30873]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T10:39:35.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:35 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:35.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:35 smithi007 ceph-mon[30873]: from='osd.2 [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:39:35.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:35 smithi007 ceph-mon[30873]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:39:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:39:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:35 smithi103 ceph-mon[35160]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T10:39:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:35 smithi103 ceph-mon[35160]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T10:39:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:35 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:35 smithi103 ceph-mon[35160]: from='osd.2 [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:39:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:35 smithi103 ceph-mon[35160]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:39:36.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.281+0000 7fa749187640 1 -- 172.21.15.7:0/2191944018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa744102fb0 msgr2=0x7fa744103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.281+0000 7fa749187640 1 --2- 172.21.15.7:0/2191944018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa744102fb0 0x7fa744103430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa738009930 tx=0x7fa73802a420 comp rx=0 tx=0).stop 2024-09-10T10:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.281+0000 7fa749187640 1 -- 172.21.15.7:0/2191944018 shutdown_connections 2024-09-10T10:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.281+0000 7fa749187640 1 --2- 172.21.15.7:0/2191944018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa744103970 0x7fa74410a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.282+0000 7fa749187640 1 --2- 172.21.15.7:0/2191944018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa744102fb0 0x7fa744103430 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.282+0000 7fa749187640 1 --2- 172.21.15.7:0/2191944018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7440fff30 0x7fa744100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.282+0000 7fa749187640 1 -- 172.21.15.7:0/2191944018 >> 172.21.15.7:0/2191944018 conn(0x7fa7440fba80 msgr2=0x7fa7440fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.282+0000 7fa749187640 1 -- 172.21.15.7:0/2191944018 shutdown_connections 2024-09-10T10:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.282+0000 7fa749187640 1 -- 172.21.15.7:0/2191944018 wait complete. 2024-09-10T10:39:36.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.282+0000 7fa749187640 1 Processor -- start 2024-09-10T10:39:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa749187640 1 -- start start 2024-09-10T10:39:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa749187640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7440fff30 0x7fa744197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa749187640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa744102fb0 0x7fa7441984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa749187640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa744103970 0x7fa74419f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa749187640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa74410d880 con 0x7fa744103970 2024-09-10T10:39:36.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa749187640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa74410d700 con 0x7fa7440fff30 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa749187640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa74410da00 con 0x7fa744102fb0 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa74359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa744103970 0x7fa74419f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa74359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa744103970 0x7fa74419f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33492/0 (socket says 172.21.15.7:33492) 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa74359e640 1 -- 172.21.15.7:0/1476501839 learned_addr learned my addr 172.21.15.7:0/1476501839 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa742d9d640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7440fff30 0x7fa744197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.283+0000 7fa74259c640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa744102fb0 0x7fa7441984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa74359e640 1 -- 172.21.15.7:0/1476501839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa744102fb0 msgr2=0x7fa7441984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa74359e640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa744102fb0 0x7fa7441984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa74359e640 1 -- 172.21.15.7:0/1476501839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7440fff30 msgr2=0x7fa744197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa74359e640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7440fff30 0x7fa744197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa74359e640 1 -- 172.21.15.7:0/1476501839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74419fba0 con 0x7fa744103970 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa74359e640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa744103970 0x7fa74419f530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00cad0 tx=0x7fa73c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:39:36.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa72bfff640 1 -- 172.21.15.7:0/1476501839 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00ae20 con 0x7fa744103970 2024-09-10T10:39:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa72bfff640 1 -- 172.21.15.7:0/1476501839 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73c005020 con 0x7fa744103970 2024-09-10T10:39:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa749187640 1 -- 172.21.15.7:0/1476501839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74419fe90 con 0x7fa744103970 2024-09-10T10:39:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa72bfff640 1 -- 172.21.15.7:0/1476501839 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00f6b0 con 0x7fa744103970 2024-09-10T10:39:36.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.284+0000 7fa749187640 1 -- 172.21.15.7:0/1476501839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7441a0340 con 0x7fa744103970 2024-09-10T10:39:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.285+0000 7fa749187640 1 -- 172.21.15.7:0/1476501839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa744109250 con 0x7fa744103970 2024-09-10T10:39:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.286+0000 7fa72bfff640 1 -- 172.21.15.7:0/1476501839 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fa73c0051d0 con 0x7fa744103970 2024-09-10T10:39:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.287+0000 7fa72bfff640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa72c0a2b20 0x7fa72c0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:39:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.288+0000 7fa742d9d640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa72c0a2b20 0x7fa72c0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:39:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.288+0000 7fa72bfff640 1 -- 172.21.15.7:0/1476501839 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7fa73c0cdc50 con 0x7fa744103970 2024-09-10T10:39:36.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.288+0000 7fa742d9d640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa72c0a2b20 0x7fa72c0a4fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa744069cb0 tx=0x7fa73400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:39:36.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.292+0000 7fa72bfff640 1 -- 172.21.15.7:0/1476501839 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa73c014030 con 0x7fa744103970 2024-09-10T10:39:36.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:36.480+0000 7fa749187640 1 -- 172.21.15.7:0/1476501839 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa744062630 con 0x7fa72c0a2b20 2024-09-10T10:39:36.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:36.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:39:36.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T10:39:36.852 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:36.853 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:36.853 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.853 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.853 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:36.853 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.853 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:36.853 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:36.853 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.854 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:36.854 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:36.854 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:39:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T10:39:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:36.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:36.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:36.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:39:36.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T10:39:36.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:36.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:36.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:39:36.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:36.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:36.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:36.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:39:36.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:39:36.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:37 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:39:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:37 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:39:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:37 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi133 2024-09-10T10:39:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:37 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi133 to 17088M 2024-09-10T10:39:37.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:37 smithi103 ceph-mon[35160]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:37.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:37 smithi103 ceph-mon[35160]: osd.2 [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] boot 2024-09-10T10:39:37.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:37 smithi103 ceph-mon[35160]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T10:39:37.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:37.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:37 smithi103 ceph-mon[35160]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:37.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:39:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:37 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:39:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:37 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:39:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:37 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi133 2024-09-10T10:39:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:37 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi133 to 17088M 2024-09-10T10:39:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:37 smithi007 ceph-mon[30873]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:37 smithi007 ceph-mon[30873]: osd.2 [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] boot 2024-09-10T10:39:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:37 smithi007 ceph-mon[30873]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T10:39:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:37 smithi007 ceph-mon[30873]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:39:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:37 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:39:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:37 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:39:37.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:37 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi133 2024-09-10T10:39:37.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:37 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi133 to 17088M 2024-09-10T10:39:37.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:37 smithi133 ceph-mon[35282]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:39:37.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:37 smithi133 ceph-mon[35282]: osd.2 [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] boot 2024-09-10T10:39:37.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:37 smithi133 ceph-mon[35282]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T10:39:37.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:39:37.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:37 smithi133 ceph-mon[35282]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-10T10:39:37.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:39:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:38 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T10:39:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:38 smithi103 ceph-mon[35160]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T10:39:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:38 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:39:38.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:38 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T10:39:38.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:38 smithi007 ceph-mon[30873]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T10:39:38.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:38 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:39:38.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:38 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T10:39:38.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:38 smithi133 ceph-mon[35282]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T10:39:38.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:38 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:39:39.358 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:39:39 smithi007 sudo[49633]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:39:39.358 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:39:39 smithi007 sudo[49633]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:39.358 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:39:39 smithi007 sudo[49633]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:39.358 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:39:39 smithi007 sudo[49633]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:39.619 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:39:39 smithi007 sudo[49661]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:39:39.619 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:39:39 smithi007 sudo[49661]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:39.619 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:39:39 smithi007 sudo[49661]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:39.619 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 10:39:39 smithi007 sudo[49661]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:39.619 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:39 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:39:39.619 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:39 smithi007 ceph-mon[30873]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T10:39:39.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:39 smithi007 ceph-mon[30873]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:39.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:39 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1684567923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e53d79e-038f-4a7f-9ac9-16b60f47bb4e"}]: dispatch 2024-09-10T10:39:39.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:39 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1684567923' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e53d79e-038f-4a7f-9ac9-16b60f47bb4e"}]': finished 2024-09-10T10:39:39.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:39 smithi007 ceph-mon[30873]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T10:39:39.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:39 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:39:39.822 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:39 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:39:39.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:39 smithi103 ceph-mon[35160]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T10:39:39.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:39 smithi103 ceph-mon[35160]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:39.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:39 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1684567923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e53d79e-038f-4a7f-9ac9-16b60f47bb4e"}]: dispatch 2024-09-10T10:39:39.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:39 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1684567923' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e53d79e-038f-4a7f-9ac9-16b60f47bb4e"}]': finished 2024-09-10T10:39:39.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:39 smithi103 ceph-mon[35160]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T10:39:39.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:39 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:39:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:39 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:39:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:39 smithi133 ceph-mon[35282]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T10:39:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:39 smithi133 ceph-mon[35282]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:39 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1684567923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e53d79e-038f-4a7f-9ac9-16b60f47bb4e"}]: dispatch 2024-09-10T10:39:39.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:39 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1684567923' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e53d79e-038f-4a7f-9ac9-16b60f47bb4e"}]': finished 2024-09-10T10:39:39.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:39 smithi133 ceph-mon[35282]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T10:39:39.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:39 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:39:40.078 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:39 smithi103 sudo[43800]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:39:40.078 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:39 smithi103 sudo[43800]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:40.078 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:39 smithi103 sudo[43800]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:40.078 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:39 smithi103 sudo[43800]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:40.422 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:40 smithi103 sudo[43803]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:39:40.422 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:40 smithi103 sudo[43803]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:40.422 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:40 smithi103 sudo[43803]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:40.422 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 10:39:40 smithi103 sudo[43803]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:40.606 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:40 smithi133 sudo[44357]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:39:40.606 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:40 smithi133 sudo[44357]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:40.606 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:40 smithi133 sudo[44357]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:40.606 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:40 smithi133 sudo[44357]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:40.606 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:40.606 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:40 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4094961342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:40.606 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:40 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4094961342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:40.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:40.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:40 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4094961342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:39:40.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:40.975 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:40 smithi133 sudo[44360]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:39:40.975 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:40 smithi133 sudo[44360]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:40.975 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:40 smithi133 sudo[44360]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:40.975 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 10:39:40 smithi133 sudo[44360]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:41.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:41 smithi007 sudo[50088]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:39:41.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:41 smithi007 sudo[50088]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:41.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:41 smithi007 sudo[50088]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:41.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:41 smithi007 sudo[50088]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:41.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:41 smithi007 sudo[50095]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:39:41.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:41 smithi007 sudo[50095]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:41.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:41 smithi007 sudo[50095]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:41.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:41 smithi007 sudo[50095]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:42.035 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:41 smithi133 sudo[44364]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:39:42.036 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:41 smithi133 sudo[44364]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:42.036 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:41 smithi133 sudo[44364]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:42.036 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:41 smithi133 sudo[44364]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:42.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 sudo[44367]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:39:42.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 sudo[44367]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:42.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 sudo[44367]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:42.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 sudo[44367]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:42.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 sudo[43806]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T10:39:42.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 sudo[43806]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:42.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 sudo[43806]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:42.536 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 sudo[43806]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:42.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:42.548+0000 7fa72bfff640 1 -- 172.21.15.7:0/1476501839 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa73c0981d0 con 0x7fa744103970 2024-09-10T10:39:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 sudo[43809]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T10:39:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 sudo[43809]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T10:39:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 sudo[43809]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T10:39:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 sudo[43809]: pam_unix(sudo:session): session closed for user root 2024-09-10T10:39:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 ceph-mon[35160]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:42.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 ceph-mon[35160]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:39:42.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 ceph-mon[35160]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:39:42.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:39:42.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:39:42.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:39:42.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:39:42.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:39:42.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:39:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:42 smithi007 ceph-mon[30873]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:42 smithi007 ceph-mon[30873]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:39:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:42 smithi007 ceph-mon[30873]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:39:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:39:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:39:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:39:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:39:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:39:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:39:42.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 ceph-mon[35282]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:42.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:42.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 ceph-mon[35282]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:39:42.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 ceph-mon[35282]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:39:42.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:39:42.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:39:42.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:39:42.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:39:42.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:39:42.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:39:43.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:43 smithi007 ceph-mon[30873]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:39:43.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:43 smithi007 ceph-mon[30873]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:39:43.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:43 smithi007 ceph-mon[30873]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:39:43.878 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:39:43.878 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:39:43.878 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:39:43.878 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:43 smithi007 ceph-mon[30873]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:39:43.878 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:43 smithi007 ceph-mon[30873]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T10:39:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:43 smithi103 ceph-mon[35160]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:39:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:43 smithi103 ceph-mon[35160]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:39:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:43 smithi103 ceph-mon[35160]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:39:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:43 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:39:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:43 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:39:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:43 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:39:43.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:43 smithi103 ceph-mon[35160]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:39:43.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:43 smithi103 ceph-mon[35160]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T10:39:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:43 smithi133 ceph-mon[35282]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:39:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:43 smithi133 ceph-mon[35282]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:39:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:43 smithi133 ceph-mon[35282]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T10:39:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:43 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:39:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:43 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:39:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:43 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:39:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:43 smithi133 ceph-mon[35282]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T10:39:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:43 smithi133 ceph-mon[35282]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T10:39:44.820 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:44 smithi007 ceph-mon[30873]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:44.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:44 smithi103 ceph-mon[35160]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:44.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:44 smithi133 ceph-mon[35282]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:45 smithi103 ceph-mon[35160]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:45.926 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:45 smithi007 ceph-mon[30873]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:45 smithi133 ceph-mon[35282]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:47 smithi103 ceph-mon[35160]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:47 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T10:39:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:47 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:47 smithi103 ceph-mon[35160]: Deploying daemon osd.3 on smithi007 2024-09-10T10:39:47.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:47 smithi007 ceph-mon[30873]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:47.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:47 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T10:39:47.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:47 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:47.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:47 smithi007 ceph-mon[30873]: Deploying daemon osd.3 on smithi007 2024-09-10T10:39:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:47 smithi133 ceph-mon[35282]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:39:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:47 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T10:39:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:47 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:47 smithi133 ceph-mon[35282]: Deploying daemon osd.3 on smithi007 2024-09-10T10:39:49.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:49 smithi007 ceph-mon[30873]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-10T10:39:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:49 smithi133 ceph-mon[35282]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-10T10:39:50.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:49 smithi103 ceph-mon[35160]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-10T10:39:51.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:51 smithi007 ceph-mon[30873]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-10T10:39:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:51 smithi133 ceph-mon[35282]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-10T10:39:52.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:51 smithi103 ceph-mon[35160]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-10T10:39:53.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:53 smithi007 ceph-mon[30873]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T10:39:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:53 smithi133 ceph-mon[35282]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T10:39:54.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:53 smithi103 ceph-mon[35160]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T10:39:54.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:54 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:54.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:54 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:54.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:54 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:54.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:54 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:54.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:54 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:54.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:54 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:55.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:54 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:55.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:54 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:55.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:54 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:56.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:55 smithi007 ceph-mon[30873]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T10:39:56.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:55 smithi103 ceph-mon[35160]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T10:39:56.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:55 smithi133 ceph-mon[35282]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T10:39:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.268+0000 7fa72bfff640 1 -- 172.21.15.7:0/1476501839 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa744062630 con 0x7fa72c0a2b20 2024-09-10T10:39:57.270 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 3 on host 'smithi007' 2024-09-10T10:39:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.271+0000 7fa749187640 1 -- 172.21.15.7:0/1476501839 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa72c0a2b20 msgr2=0x7fa72c0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.271+0000 7fa749187640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa72c0a2b20 0x7fa72c0a4fe0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa744069cb0 tx=0x7fa73400b390 comp rx=0 tx=0).stop 2024-09-10T10:39:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.271+0000 7fa749187640 1 -- 172.21.15.7:0/1476501839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa744103970 msgr2=0x7fa74419f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:39:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.271+0000 7fa749187640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa744103970 0x7fa74419f530 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00cad0 tx=0x7fa73c00cf90 comp rx=0 tx=0).stop 2024-09-10T10:39:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.272+0000 7fa749187640 1 -- 172.21.15.7:0/1476501839 shutdown_connections 2024-09-10T10:39:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.272+0000 7fa749187640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa72c0a2b20 0x7fa72c0a4fe0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.272+0000 7fa749187640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa744103970 0x7fa74419f530 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.272+0000 7fa749187640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa744102fb0 0x7fa7441984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.272+0000 7fa749187640 1 --2- 172.21.15.7:0/1476501839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7440fff30 0x7fa744197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:39:57.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.272+0000 7fa749187640 1 -- 172.21.15.7:0/1476501839 >> 172.21.15.7:0/1476501839 conn(0x7fa7440fba80 msgr2=0x7fa744106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:39:57.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.272+0000 7fa749187640 1 -- 172.21.15.7:0/1476501839 shutdown_connections 2024-09-10T10:39:57.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:39:57.272+0000 7fa749187640 1 -- 172.21.15.7:0/1476501839 wait complete. 2024-09-10T10:39:57.582 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.582 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.582 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:57.582 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:57.582 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.582 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:57.582 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.582 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.904 DEBUG:teuthology.orchestra.run.smithi007:osd.3> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T10:39:57.906 INFO:tasks.cephadm:Deploying osd.4 on smithi103 with /dev/vg_nvme/lv_3... 2024-09-10T10:39:57.907 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T10:39:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:39:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:39:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:39:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:39:58.164 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:39:58.654 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:39:58 smithi103 ceph-mon[35160]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T10:39:58.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:39:58 smithi133 ceph-mon[35282]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T10:39:58.789 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:39:58 smithi007 ceph-mon[30873]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T10:40:00.434 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 10:40:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3[51694]: 2024-09-10T10:40:00.103+0000 7f458c229640 -1 osd.3 0 log_to_monitors true 2024-09-10T10:40:00.631 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:00 smithi103 ceph-mon[35160]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T10:40:00.632 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:00 smithi103 ceph-mon[35160]: overall HEALTH_OK 2024-09-10T10:40:00.632 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:00 smithi103 ceph-mon[35160]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T10:40:00.634 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:40:00.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-09-10T10:40:00.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:00 smithi133 ceph-mon[35282]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T10:40:00.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:00 smithi133 ceph-mon[35282]: overall HEALTH_OK 2024-09-10T10:40:00.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:00 smithi133 ceph-mon[35282]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T10:40:00.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:00 smithi007 ceph-mon[30873]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T10:40:00.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:00 smithi007 ceph-mon[30873]: overall HEALTH_OK 2024-09-10T10:40:00.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:00 smithi007 ceph-mon[30873]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T10:40:00.959 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:40:01.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.423+0000 7fb389d51640 1 -- 172.21.15.103:0/4083753439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb384107900 msgr2=0x7fb384109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:01.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.423+0000 7fb389d51640 1 --2- 172.21.15.103:0/4083753439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb384107900 0x7fb384109d50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009ca0 tx=0x7fb37c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.423+0000 7fb389d51640 1 -- 172.21.15.103:0/4083753439 shutdown_connections 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.423+0000 7fb389d51640 1 --2- 172.21.15.103:0/4083753439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb384107900 0x7fb384109d50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.423+0000 7fb389d51640 1 --2- 172.21.15.103:0/4083753439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb384104fd0 0x7fb3841073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.423+0000 7fb389d51640 1 --2- 172.21.15.103:0/4083753439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841026a0 0x7fb384104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.423+0000 7fb389d51640 1 -- 172.21.15.103:0/4083753439 >> 172.21.15.103:0/4083753439 conn(0x7fb3840fc2a0 msgr2=0x7fb3840fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.423+0000 7fb389d51640 1 -- 172.21.15.103:0/4083753439 shutdown_connections 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.424+0000 7fb389d51640 1 -- 172.21.15.103:0/4083753439 wait complete. 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.424+0000 7fb389d51640 1 Processor -- start 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.424+0000 7fb389d51640 1 -- start start 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.424+0000 7fb389d51640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841026a0 0x7fb384197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:01.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.424+0000 7fb389d51640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb384104fd0 0x7fb384198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.424+0000 7fb389d51640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb384107900 0x7fb38419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.424+0000 7fb389d51640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb38410d8c0 con 0x7fb384104fd0 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.424+0000 7fb389d51640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb38410d740 con 0x7fb384107900 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb389d51640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb38410da40 con 0x7fb3841026a0 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb383fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb384107900 0x7fb38419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb3837fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841026a0 0x7fb384197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb383fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb384107900 0x7fb38419f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41520/0 (socket says 172.21.15.103:41520) 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb383fff640 1 -- 172.21.15.103:0/2090373692 learned_addr learned my addr 172.21.15.103:0/2090373692 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb382ffd640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb384104fd0 0x7fb384198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb383fff640 1 -- 172.21.15.103:0/2090373692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841026a0 msgr2=0x7fb384197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:01.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb383fff640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841026a0 0x7fb384197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb383fff640 1 -- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb384104fd0 msgr2=0x7fb384198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb383fff640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb384104fd0 0x7fb384198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb383fff640 1 -- 172.21.15.103:0/2090373692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38419fb20 con 0x7fb384107900 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb3837fe640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841026a0 0x7fb384197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb382ffd640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb384104fd0 0x7fb384198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb383fff640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb384107900 0x7fb38419f4b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb37c031ae0 tx=0x7fb37c031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb380ff9640 1 -- 172.21.15.103:0/2090373692 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c003c30 con 0x7fb384107900 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb389d51640 1 -- 172.21.15.103:0/2090373692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38419fdb0 con 0x7fb384107900 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb380ff9640 1 -- 172.21.15.103:0/2090373692 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb37c003dd0 con 0x7fb384107900 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb380ff9640 1 -- 172.21.15.103:0/2090373692 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37c02fcb0 con 0x7fb384107900 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.425+0000 7fb389d51640 1 -- 172.21.15.103:0/2090373692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3841a0260 con 0x7fb384107900 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.426+0000 7fb389d51640 1 -- 172.21.15.103:0/2090373692 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3840fdd90 con 0x7fb384107900 2024-09-10T10:40:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.427+0000 7fb380ff9640 1 -- 172.21.15.103:0/2090373692 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb37c004050 con 0x7fb384107900 2024-09-10T10:40:01.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.428+0000 7fb380ff9640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb36c0a2bb0 0x7fb36c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:01.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.428+0000 7fb380ff9640 1 -- 172.21.15.103:0/2090373692 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3373+0+0 (secure 0 0 0) 0x7fb37c0ef140 con 0x7fb384107900 2024-09-10T10:40:01.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.428+0000 7fb3837fe640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb36c0a2bb0 0x7fb36c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:01.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.429+0000 7fb3837fe640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb36c0a2bb0 0x7fb36c0a5070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb374009fc0 tx=0x7fb374005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:40:01.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.432+0000 7fb380ff9640 1 -- 172.21.15.103:0/2090373692 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb37c0bd3f0 con 0x7fb384107900 2024-09-10T10:40:01.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:01.608+0000 7fb389d51640 1 -- 172.21.15.103:0/2090373692 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb384062630 con 0x7fb36c0a2bb0 2024-09-10T10:40:01.610 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T10:40:01.610 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T10:40:01.610 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:40:01.610 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:01.610 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.610 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.610 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:01.610 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:01.610 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.611 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:01.611 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:01.611 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.611 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:01 smithi103 ceph-mon[35160]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:40:01.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T10:40:01.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T10:40:01.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:40:01.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:01.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:01.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:01.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:01.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:01.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:01 smithi133 ceph-mon[35282]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:01.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:01.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:01 smithi007 ceph-mon[30873]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:40:02.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi007 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi007 to 8546M 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] boot 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T10:40:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:02.683 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3[51694]: 2024-09-10T10:40:02.220+0000 7f4587989640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T10:40:02.684 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3[51694]: 2024-09-10T10:40:02.226+0000 7f457ffbb640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:40:02.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:40:02.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi007 2024-09-10T10:40:02.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi007 to 8546M 2024-09-10T10:40:02.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T10:40:02.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:02.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:02.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:02.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:02.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:02.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' 2024-09-10T10:40:02.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] boot 2024-09-10T10:40:02.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T10:40:02.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:02.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-09-10T10:40:02.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi007 2024-09-10T10:40:02.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi007 to 8546M 2024-09-10T10:40:02.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T10:40:02.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:02.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:02.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:02.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:02.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:02.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: from='osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144]' entity='osd.3' 2024-09-10T10:40:02.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: osd.3 [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] boot 2024-09-10T10:40:02.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T10:40:02.727 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:40:03.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:03 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:40:03.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:03 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:40:03.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:03 smithi133 ceph-mon[35282]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:03.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:03 smithi133 ceph-mon[35282]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T10:40:03.875 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:03 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:40:03.876 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:03 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:40:03.876 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:03 smithi007 ceph-mon[30873]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:03.876 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:03 smithi007 ceph-mon[30873]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T10:40:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:03 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:40:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:03 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:40:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:03 smithi103 ceph-mon[35160]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:03 smithi103 ceph-mon[35160]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T10:40:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:04 smithi133 ceph-mon[35282]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:04 smithi133 ceph-mon[35282]: from='client.? 172.21.15.103:0/4014673777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f04c13-594c-43dd-9a98-d2138061b1a0"}]: dispatch 2024-09-10T10:40:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:04 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f04c13-594c-43dd-9a98-d2138061b1a0"}]: dispatch 2024-09-10T10:40:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:04 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4f04c13-594c-43dd-9a98-d2138061b1a0"}]': finished 2024-09-10T10:40:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:04 smithi133 ceph-mon[35282]: osdmap e26: 5 total, 4 up, 5 in 2024-09-10T10:40:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:04 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:04.881 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:04 smithi007 ceph-mon[30873]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:04.881 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:04 smithi007 ceph-mon[30873]: from='client.? 172.21.15.103:0/4014673777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f04c13-594c-43dd-9a98-d2138061b1a0"}]: dispatch 2024-09-10T10:40:04.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:04 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f04c13-594c-43dd-9a98-d2138061b1a0"}]: dispatch 2024-09-10T10:40:04.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:04 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4f04c13-594c-43dd-9a98-d2138061b1a0"}]': finished 2024-09-10T10:40:04.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:04 smithi007 ceph-mon[30873]: osdmap e26: 5 total, 4 up, 5 in 2024-09-10T10:40:04.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:04 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:04.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:04 smithi103 ceph-mon[35160]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:04.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:04 smithi103 ceph-mon[35160]: from='client.? 172.21.15.103:0/4014673777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f04c13-594c-43dd-9a98-d2138061b1a0"}]: dispatch 2024-09-10T10:40:04.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:04 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4f04c13-594c-43dd-9a98-d2138061b1a0"}]: dispatch 2024-09-10T10:40:04.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:04 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4f04c13-594c-43dd-9a98-d2138061b1a0"}]': finished 2024-09-10T10:40:04.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:04 smithi103 ceph-mon[35160]: osdmap e26: 5 total, 4 up, 5 in 2024-09-10T10:40:04.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:04 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:05.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:05 smithi133 ceph-mon[35282]: from='client.? 172.21.15.103:0/3260663464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:05.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:05 smithi133 ceph-mon[35282]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T10:40:05.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:05 smithi103 ceph-mon[35160]: from='client.? 172.21.15.103:0/3260663464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:05 smithi103 ceph-mon[35160]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T10:40:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:05 smithi007 ceph-mon[30873]: from='client.? 172.21.15.103:0/3260663464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:05 smithi007 ceph-mon[30873]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T10:40:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:06.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:06 smithi133 ceph-mon[35282]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:06.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:06 smithi133 ceph-mon[35282]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T10:40:06.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:06 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:06 smithi103 ceph-mon[35160]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:06 smithi103 ceph-mon[35160]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T10:40:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:06 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:06.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:06 smithi007 ceph-mon[30873]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:06 smithi007 ceph-mon[30873]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T10:40:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:06 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:08.728 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:08 smithi133 ceph-mon[35282]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:08.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:08 smithi103 ceph-mon[35160]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:08.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:08 smithi007 ceph-mon[30873]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 528 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:10.661 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:10 smithi103 ceph-mon[35160]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-10T10:40:10.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:10 smithi133 ceph-mon[35282]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-10T10:40:10.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:10 smithi007 ceph-mon[30873]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-10T10:40:12.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:12 smithi133 ceph-mon[35282]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-10T10:40:12.897 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:12 smithi103 ceph-mon[35160]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-10T10:40:12.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:12 smithi007 ceph-mon[30873]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-10T10:40:13.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:13 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T10:40:13.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:13 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:13.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:13 smithi103 ceph-mon[35160]: Deploying daemon osd.4 on smithi103 2024-09-10T10:40:13.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:13 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T10:40:13.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:13 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:13.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:13 smithi007 ceph-mon[30873]: Deploying daemon osd.4 on smithi103 2024-09-10T10:40:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:13 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T10:40:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:13 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:13 smithi133 ceph-mon[35282]: Deploying daemon osd.4 on smithi103 2024-09-10T10:40:14.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:14 smithi103 ceph-mon[35160]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:40:14.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:14 smithi007 ceph-mon[30873]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:40:14.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:14 smithi133 ceph-mon[35282]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:40:16.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:16 smithi103 ceph-mon[35160]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T10:40:16.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:16 smithi007 ceph-mon[30873]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T10:40:16.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:16 smithi133 ceph-mon[35282]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T10:40:18.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:18 smithi103 ceph-mon[35160]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T10:40:18.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:18 smithi007 ceph-mon[30873]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T10:40:18.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:18 smithi133 ceph-mon[35282]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T10:40:19.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:19 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:19.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:19 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:19.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:19 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:19.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:19 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:19.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:19 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:19.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:19 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:19 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:19 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:19 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:20.821 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:20 smithi103 ceph-mon[35160]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:40:20.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:20 smithi007 ceph-mon[30873]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:40:20.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:20 smithi133 ceph-mon[35282]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:40:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:22 smithi103 ceph-mon[35160]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:22.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:22 smithi007 ceph-mon[30873]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:22.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:22.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:22.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:22.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:22.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:22.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:22 smithi133 ceph-mon[35282]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:22.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:22.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:22.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:22.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:22.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:22.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.480+0000 7fb380ff9640 1 -- 172.21.15.103:0/2090373692 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb384062630 con 0x7fb36c0a2bb0 2024-09-10T10:40:22.482 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 4 on host 'smithi103' 2024-09-10T10:40:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.482+0000 7fb389d51640 1 -- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb36c0a2bb0 msgr2=0x7fb36c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.482+0000 7fb389d51640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb36c0a2bb0 0x7fb36c0a5070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb374009fc0 tx=0x7fb374005f90 comp rx=0 tx=0).stop 2024-09-10T10:40:22.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.482+0000 7fb389d51640 1 -- 172.21.15.103:0/2090373692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb384107900 msgr2=0x7fb38419f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.482+0000 7fb389d51640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb384107900 0x7fb38419f4b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb37c031ae0 tx=0x7fb37c031ee0 comp rx=0 tx=0).stop 2024-09-10T10:40:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.482+0000 7fb389d51640 1 -- 172.21.15.103:0/2090373692 shutdown_connections 2024-09-10T10:40:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.482+0000 7fb389d51640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb36c0a2bb0 0x7fb36c0a5070 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.482+0000 7fb389d51640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb384107900 0x7fb38419f4b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.482+0000 7fb389d51640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb384104fd0 0x7fb384198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.482+0000 7fb389d51640 1 --2- 172.21.15.103:0/2090373692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3841026a0 0x7fb384197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.482+0000 7fb389d51640 1 -- 172.21.15.103:0/2090373692 >> 172.21.15.103:0/2090373692 conn(0x7fb3840fc2a0 msgr2=0x7fb384102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:40:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.483+0000 7fb389d51640 1 -- 172.21.15.103:0/2090373692 shutdown_connections 2024-09-10T10:40:22.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:40:22.483+0000 7fb389d51640 1 -- 172.21.15.103:0/2090373692 wait complete. 2024-09-10T10:40:23.170 DEBUG:teuthology.orchestra.run.smithi103:osd.4> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T10:40:23.172 INFO:tasks.cephadm:Deploying osd.5 on smithi133 with /dev/vg_nvme/lv_3... 2024-09-10T10:40:23.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T10:40:23.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:23 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:23.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:23 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:23.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:23 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:23.428 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:40:23.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:23 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:23.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:23 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:23.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:23 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:23.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:23 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:23.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:23 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:23.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:23 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:24.755 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:24 smithi133 ceph-mon[35282]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:24.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:24 smithi103 ceph-mon[35160]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:24.922 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 10:40:24 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4[46574]: 2024-09-10T10:40:24.615+0000 7fb57af55640 -1 osd.4 0 log_to_monitors true 2024-09-10T10:40:24.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:24 smithi007 ceph-mon[30873]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:25.565 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:25 smithi103 ceph-mon[35160]: from='osd.4 [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:40:25.566 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:25 smithi103 ceph-mon[35160]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:40:25.688 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:25 smithi133 ceph-mon[35282]: from='osd.4 [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:40:25.688 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:25 smithi133 ceph-mon[35282]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:40:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:25 smithi007 ceph-mon[30873]: from='osd.4 [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:40:25.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:25 smithi007 ceph-mon[30873]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T10:40:25.947 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:40:25.975 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2024-09-10T10:40:26.231 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:40:26.590 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:26.590 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T10:40:26.590 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T10:40:26.590 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:26.590 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='osd.4 [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:40:26.590 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:40:26.590 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.591 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.591 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:26.591 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:26.591 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.591 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:26.591 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:26.591 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:26 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.672+0000 7f06c73a9640 1 -- 172.21.15.133:0/538869390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c0101cd0 msgr2=0x7f06c0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.672+0000 7f06c73a9640 1 --2- 172.21.15.133:0/538869390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c0101cd0 0x7f06c0102150 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009900 tx=0x7f06b002a410 comp rx=0 tx=0).stop 2024-09-10T10:40:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.672+0000 7f06c73a9640 1 -- 172.21.15.133:0/538869390 shutdown_connections 2024-09-10T10:40:26.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.672+0000 7f06c73a9640 1 --2- 172.21.15.133:0/538869390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c0102690 0x7f06c0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.672+0000 7f06c73a9640 1 --2- 172.21.15.133:0/538869390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c0101cd0 0x7f06c0102150 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.672+0000 7f06c73a9640 1 --2- 172.21.15.133:0/538869390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0100ad0 0x7f06c0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.672+0000 7f06c73a9640 1 -- 172.21.15.133:0/538869390 >> 172.21.15.133:0/538869390 conn(0x7f06c00fc280 msgr2=0x7f06c00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.672+0000 7f06c73a9640 1 -- 172.21.15.133:0/538869390 shutdown_connections 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.672+0000 7f06c73a9640 1 -- 172.21.15.133:0/538869390 wait complete. 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.672+0000 7f06c73a9640 1 Processor -- start 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c73a9640 1 -- start start 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c73a9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c0100ad0 0x7f06c0196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c73a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0101cd0 0x7f06c0197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c73a9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c0102690 0x7f06c019e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:26.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c73a9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06c010c730 con 0x7f06c0101cd0 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c73a9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06c010c5b0 con 0x7f06c0102690 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c73a9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f06c010c8b0 con 0x7f06c0100ad0 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5145640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c0100ad0 0x7f06c0196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5145640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c0100ad0 0x7f06c0196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33850/0 (socket says 172.21.15.133:33850) 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5145640 1 -- 172.21.15.133:0/2735817729 learned_addr learned my addr 172.21.15.133:0/2735817729 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5946640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c0102690 0x7f06c019e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5145640 1 -- 172.21.15.133:0/2735817729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c0102690 msgr2=0x7f06c019e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5145640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c0102690 0x7f06c019e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5145640 1 -- 172.21.15.133:0/2735817729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0101cd0 msgr2=0x7f06c0197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5145640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0101cd0 0x7f06c0197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5145640 1 -- 172.21.15.133:0/2735817729 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c019e960 con 0x7f06c0100ad0 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5946640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c0102690 0x7f06c019e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:40:26.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06c5145640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c0100ad0 0x7f06c0196d40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00ccb0 tx=0x7f06bc00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:40:26.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.673+0000 7f06b67fc640 1 -- 172.21.15.133:0/2735817729 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc003b70 con 0x7f06c0100ad0 2024-09-10T10:40:26.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.674+0000 7f06b67fc640 1 -- 172.21.15.133:0/2735817729 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06bc00ae50 con 0x7f06c0100ad0 2024-09-10T10:40:26.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.674+0000 7f06b67fc640 1 -- 172.21.15.133:0/2735817729 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc00f810 con 0x7f06c0100ad0 2024-09-10T10:40:26.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.674+0000 7f06c73a9640 1 -- 172.21.15.133:0/2735817729 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c019ec50 con 0x7f06c0100ad0 2024-09-10T10:40:26.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.674+0000 7f06c73a9640 1 -- 172.21.15.133:0/2735817729 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c019f140 con 0x7f06c0100ad0 2024-09-10T10:40:26.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.675+0000 7f06c73a9640 1 -- 172.21.15.133:0/2735817729 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c0107e80 con 0x7f06c0100ad0 2024-09-10T10:40:26.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.676+0000 7f06b67fc640 1 -- 172.21.15.133:0/2735817729 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f06bc00f9b0 con 0x7f06c0100ad0 2024-09-10T10:40:26.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.676+0000 7f06b67fc640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f06ac0a2c00 0x7f06ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:26.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.677+0000 7f06b67fc640 1 -- 172.21.15.133:0/2735817729 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7f06bc0ce430 con 0x7f06c0100ad0 2024-09-10T10:40:26.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.677+0000 7f06c4944640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f06ac0a2c00 0x7f06ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:26.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.677+0000 7f06c4944640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f06ac0a2c00 0x7f06ac0a50c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06b00096c0 tx=0x7f06b0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:40:26.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.680+0000 7f06b67fc640 1 -- 172.21.15.133:0/2735817729 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f06bc014070 con 0x7f06c0100ad0 2024-09-10T10:40:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:26.857+0000 7f06c73a9640 1 -- 172.21.15.133:0/2735817729 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f06c0062630 con 0x7f06ac0a2c00 2024-09-10T10:40:26.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:26.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T10:40:26.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T10:40:26.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:26.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='osd.4 [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:40:26.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:40:26.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:26.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:26.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:26.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:26.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:26 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='osd.4 [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:26.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:26.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:26.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:26.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:26 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:27.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:27 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi103 2024-09-10T10:40:27.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:27 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-10T10:40:27.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:27 smithi133 ceph-mon[35282]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:40:27.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:27 smithi133 ceph-mon[35282]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T10:40:27.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:27 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:27.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:27 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:27.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:27 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:27.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:27 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:27.569 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:27 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:27.569 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:27 smithi133 ceph-mon[35282]: from='osd.4 ' entity='osd.4' 2024-09-10T10:40:27.672 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4[46574]: 2024-09-10T10:40:27.258+0000 7fb5766b5640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T10:40:27.672 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4[46574]: 2024-09-10T10:40:27.268+0000 7fb56ece7640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:40:27.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi103 2024-09-10T10:40:27.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-10T10:40:27.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-mon[35160]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:40:27.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-mon[35160]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T10:40:27.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:27.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:27.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:27.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:27.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:27.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:27 smithi103 ceph-mon[35160]: from='osd.4 ' entity='osd.4' 2024-09-10T10:40:27.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:27 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi103 2024-09-10T10:40:27.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:27 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-10T10:40:27.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:27 smithi007 ceph-mon[30873]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:40:27.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:27 smithi007 ceph-mon[30873]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T10:40:27.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:27 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:27.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:27 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:27.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:27 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:27.937 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:27 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:27.937 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:27 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:27.937 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:27 smithi007 ceph-mon[30873]: from='osd.4 ' entity='osd.4' 2024-09-10T10:40:28.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:28 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:40:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:28 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:40:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:28 smithi133 ceph-mon[35282]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:28 smithi133 ceph-mon[35282]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:28 smithi133 ceph-mon[35282]: osd.4 [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] boot 2024-09-10T10:40:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:28 smithi133 ceph-mon[35282]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T10:40:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:28.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:28 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:40:28.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:28 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:40:28.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:28 smithi103 ceph-mon[35160]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:28.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:28 smithi103 ceph-mon[35160]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:28.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:28.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:28 smithi103 ceph-mon[35160]: osd.4 [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] boot 2024-09-10T10:40:28.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:28 smithi103 ceph-mon[35160]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T10:40:28.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:28.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:28 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:40:28.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:28 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:40:28.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:28 smithi007 ceph-mon[30873]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-10T10:40:28.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:28 smithi007 ceph-mon[30873]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:28.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:28.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:28 smithi007 ceph-mon[30873]: osd.4 [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] boot 2024-09-10T10:40:28.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:28 smithi007 ceph-mon[30873]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T10:40:28.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:40:29.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:29 smithi133 ceph-mon[35282]: from='client.? 172.21.15.133:0/2960891731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73143170-5816-407d-ae03-dcf896057681"}]: dispatch 2024-09-10T10:40:29.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:29 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73143170-5816-407d-ae03-dcf896057681"}]: dispatch 2024-09-10T10:40:29.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:29 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73143170-5816-407d-ae03-dcf896057681"}]': finished 2024-09-10T10:40:29.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:29 smithi133 ceph-mon[35282]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T10:40:29.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:29 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:29 smithi103 ceph-mon[35160]: from='client.? 172.21.15.133:0/2960891731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73143170-5816-407d-ae03-dcf896057681"}]: dispatch 2024-09-10T10:40:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:29 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73143170-5816-407d-ae03-dcf896057681"}]: dispatch 2024-09-10T10:40:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:29 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73143170-5816-407d-ae03-dcf896057681"}]': finished 2024-09-10T10:40:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:29 smithi103 ceph-mon[35160]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T10:40:29.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:29 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:29 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/2960891731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73143170-5816-407d-ae03-dcf896057681"}]: dispatch 2024-09-10T10:40:29.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:29 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73143170-5816-407d-ae03-dcf896057681"}]: dispatch 2024-09-10T10:40:29.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:29 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73143170-5816-407d-ae03-dcf896057681"}]': finished 2024-09-10T10:40:29.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:29 smithi007 ceph-mon[30873]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T10:40:29.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:29 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:30.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:30 smithi103 ceph-mon[35160]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:30.824 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:30 smithi103 ceph-mon[35160]: from='client.? 172.21.15.133:0/1411335455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:30.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:30 smithi007 ceph-mon[30873]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:30.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:30 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/1411335455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:30.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:30 smithi133 ceph-mon[35282]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:30.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:30 smithi133 ceph-mon[35282]: from='client.? 172.21.15.133:0/1411335455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:32.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:32 smithi103 ceph-mon[35160]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:32.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:32 smithi007 ceph-mon[30873]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:32.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:32 smithi133 ceph-mon[35282]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:34.904 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:34 smithi133 ceph-mon[35282]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:34 smithi103 ceph-mon[35160]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:34.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:34 smithi007 ceph-mon[30873]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:36.706 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:36 smithi133 ceph-mon[35282]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:36 smithi103 ceph-mon[35160]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:36.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:36 smithi007 ceph-mon[30873]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:38.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:38 smithi133 ceph-mon[35282]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:38.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:38 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T10:40:38.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:38 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:38 smithi103 ceph-mon[35160]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:38 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T10:40:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:38 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:38.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:38 smithi007 ceph-mon[30873]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:38.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:38 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T10:40:38.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:38 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:39.607 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:39 smithi133 ceph-mon[35282]: Deploying daemon osd.5 on smithi133 2024-09-10T10:40:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:39 smithi103 ceph-mon[35160]: Deploying daemon osd.5 on smithi133 2024-09-10T10:40:39.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:39 smithi007 ceph-mon[30873]: Deploying daemon osd.5 on smithi133 2024-09-10T10:40:40.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:40 smithi133 ceph-mon[35282]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:40 smithi103 ceph-mon[35160]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:40.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:40 smithi007 ceph-mon[30873]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:42.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:42 smithi133 ceph-mon[35282]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:42 smithi103 ceph-mon[35160]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:42.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:42 smithi007 ceph-mon[30873]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:44.720 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:44 smithi133 ceph-mon[35282]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:44.721 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:44.721 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:44.721 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:44.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:44 smithi103 ceph-mon[35160]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:44.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:44.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:44.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:44.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:44 smithi007 ceph-mon[30873]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:44.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:44.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:44.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:46.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:46 smithi103 ceph-mon[35160]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:46.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:46.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:46 smithi007 ceph-mon[30873]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:46.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:46 smithi133 ceph-mon[35282]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:46.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:46.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:46.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:46.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:46.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:47.387 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 5 on host 'smithi133' 2024-09-10T10:40:47.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.385+0000 7f06b67fc640 1 -- 172.21.15.133:0/2735817729 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f06c0062630 con 0x7f06ac0a2c00 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 -- 172.21.15.133:0/2735817729 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f06ac0a2c00 msgr2=0x7f06ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f06ac0a2c00 0x7f06ac0a50c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06b00096c0 tx=0x7f06b0031400 comp rx=0 tx=0).stop 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 -- 172.21.15.133:0/2735817729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c0100ad0 msgr2=0x7f06c0196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c0100ad0 0x7f06c0196d40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00ccb0 tx=0x7f06bc00a620 comp rx=0 tx=0).stop 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 -- 172.21.15.133:0/2735817729 shutdown_connections 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f06ac0a2c00 0x7f06ac0a50c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c0102690 0x7f06c019e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0101cd0 0x7f06c0197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 --2- 172.21.15.133:0/2735817729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c0100ad0 0x7f06c0196d40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 -- 172.21.15.133:0/2735817729 >> 172.21.15.133:0/2735817729 conn(0x7f06c00fc280 msgr2=0x7f06c0105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:40:47.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 -- 172.21.15.133:0/2735817729 shutdown_connections 2024-09-10T10:40:47.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:40:47.388+0000 7f06c73a9640 1 -- 172.21.15.133:0/2735817729 wait complete. 2024-09-10T10:40:47.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:47 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:47.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:47 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:47.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:47 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:47 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:47 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:47 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:47 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:40:47.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:47 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:47.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:47 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:48.078 DEBUG:teuthology.orchestra.run.smithi133:osd.5> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T10:40:48.080 INFO:tasks.cephadm:Deploying osd.6 on smithi007 with /dev/vg_nvme/lv_2... 2024-09-10T10:40:48.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T10:40:48.347 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:40:48.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:48 smithi133 ceph-mon[35282]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:48.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:48 smithi007 ceph-mon[30873]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:48.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:48 smithi103 ceph-mon[35160]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:50.226 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 10:40:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T10:40:49.905+0000 7fc69122d640 -1 osd.5 0 log_to_monitors true 2024-09-10T10:40:50.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:50 smithi007 ceph-mon[30873]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:50.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:50 smithi007 ceph-mon[30873]: from='osd.5 [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:40:50.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:50 smithi007 ceph-mon[30873]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:40:50.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:50 smithi133 ceph-mon[35282]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:50.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:50 smithi133 ceph-mon[35282]: from='osd.5 [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:40:50.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:50 smithi133 ceph-mon[35282]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:40:50.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:50 smithi103 ceph-mon[35160]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:50.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:50 smithi103 ceph-mon[35160]: from='osd.5 [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:40:50.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:50 smithi103 ceph-mon[35160]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T10:40:51.087 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:40:51.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_2 2024-09-10T10:40:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:40:51.815 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T10:40:51.815 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T10:40:51.815 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:51.815 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='osd.5 [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:40:51.816 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:40:51.816 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.816 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.816 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:51.816 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:51.816 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.816 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:51.816 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:51.817 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.824+0000 7f983e53d640 1 -- 172.21.15.7:0/1286239847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98381026b0 msgr2=0x7f9838108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:51.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.824+0000 7f983e53d640 1 --2- 172.21.15.7:0/1286239847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98381026b0 0x7f9838108f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9830009ca0 tx=0x7f983002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:40:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.824+0000 7f983e53d640 1 -- 172.21.15.7:0/1286239847 shutdown_connections 2024-09-10T10:40:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.824+0000 7f983e53d640 1 --2- 172.21.15.7:0/1286239847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98381026b0 0x7f9838108f40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.824+0000 7f983e53d640 1 --2- 172.21.15.7:0/1286239847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9838101cf0 0x7f9838102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.824+0000 7f983e53d640 1 --2- 172.21.15.7:0/1286239847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9838100af0 0x7f9838100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.824+0000 7f983e53d640 1 -- 172.21.15.7:0/1286239847 >> 172.21.15.7:0/1286239847 conn(0x7f98380fc2a0 msgr2=0x7f98380fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:40:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.824+0000 7f983e53d640 1 -- 172.21.15.7:0/1286239847 shutdown_connections 2024-09-10T10:40:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.824+0000 7f983e53d640 1 -- 172.21.15.7:0/1286239847 wait complete. 2024-09-10T10:40:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983e53d640 1 Processor -- start 2024-09-10T10:40:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983e53d640 1 -- start start 2024-09-10T10:40:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983e53d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9838100af0 0x7f983810b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983e53d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9838101cf0 0x7f9838109ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:51.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983e53d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98381026b0 0x7f983810a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:51.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983e53d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f983810d8a0 con 0x7f98381026b0 2024-09-10T10:40:51.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983e53d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f983810d720 con 0x7f9838100af0 2024-09-10T10:40:51.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983e53d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f983810da20 con 0x7f9838101cf0 2024-09-10T10:40:51.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983cada640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98381026b0 0x7f983810a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983cada640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98381026b0 0x7f983810a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34360/0 (socket says 172.21.15.7:34360) 2024-09-10T10:40:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f983cada640 1 -- 172.21.15.7:0/3175899204 learned_addr learned my addr 172.21.15.7:0/3175899204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:40:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f9837fff640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9838100af0 0x7f983810b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.825+0000 7f98377fe640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9838101cf0 0x7f9838109ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f983cada640 1 -- 172.21.15.7:0/3175899204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9838101cf0 msgr2=0x7f9838109ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f983cada640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9838101cf0 0x7f9838109ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f983cada640 1 -- 172.21.15.7:0/3175899204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9838100af0 msgr2=0x7f983810b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:40:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f983cada640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9838100af0 0x7f983810b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:40:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f983cada640 1 -- 172.21.15.7:0/3175899204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983810a8c0 con 0x7f98381026b0 2024-09-10T10:40:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f983cada640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98381026b0 0x7f983810a000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9830005e30 tx=0x7f9830031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:40:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f98357fa640 1 -- 172.21.15.7:0/3175899204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830003c30 con 0x7f98381026b0 2024-09-10T10:40:51.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f98357fa640 1 -- 172.21.15.7:0/3175899204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9830003dd0 con 0x7f98381026b0 2024-09-10T10:40:51.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f98357fa640 1 -- 172.21.15.7:0/3175899204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983002fde0 con 0x7f98381026b0 2024-09-10T10:40:51.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f983e53d640 1 -- 172.21.15.7:0/3175899204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98381a1fd0 con 0x7f98381026b0 2024-09-10T10:40:51.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.826+0000 7f983e53d640 1 -- 172.21.15.7:0/3175899204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98381a2430 con 0x7f98381026b0 2024-09-10T10:40:51.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.827+0000 7f983e53d640 1 -- 172.21.15.7:0/3175899204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9838107f40 con 0x7f98381026b0 2024-09-10T10:40:51.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.828+0000 7f98357fa640 1 -- 172.21.15.7:0/3175899204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9830004050 con 0x7f98381026b0 2024-09-10T10:40:51.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.829+0000 7f98357fa640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f98200a2b80 0x7f98200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:40:51.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.829+0000 7f98357fa640 1 -- 172.21.15.7:0/3175899204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7f98300ef5b0 con 0x7f98381026b0 2024-09-10T10:40:51.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.829+0000 7f9837fff640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f98200a2b80 0x7f98200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:40:51.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.830+0000 7f9837fff640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f98200a2b80 0x7f98200a5040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98280058c0 tx=0x7f982800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:40:51.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:51.832+0000 7f98357fa640 1 -- 172.21.15.7:0/3175899204 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98300bd610 con 0x7f98381026b0 2024-09-10T10:40:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T10:40:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T10:40:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='osd.5 [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:40:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:40:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:51 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T10:40:51.561+0000 7fc68c98d640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T10:40:51.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T10:40:51.568+0000 7fc688fc7640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='osd.5 [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:51.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:51.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:40:51.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:51 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:40:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:40:52.017+0000 7f983e53d640 1 -- 172.21.15.7:0/3175899204 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9838062630 con 0x7f98200a2b80 2024-09-10T10:40:52.775 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:52 smithi007 ceph-mon[30873]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:52.775 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:52 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi133 2024-09-10T10:40:52.775 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:52 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi133 to 8544M 2024-09-10T10:40:52.775 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:52 smithi007 ceph-mon[30873]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:40:52.775 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:52 smithi007 ceph-mon[30873]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T10:40:52.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:52 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:52.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:52 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:52.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:52 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:52.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:52 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:52.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:52 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:52 smithi103 ceph-mon[35160]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:52 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi133 2024-09-10T10:40:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:52 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi133 to 8544M 2024-09-10T10:40:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:52 smithi103 ceph-mon[35160]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:40:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:52 smithi103 ceph-mon[35160]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T10:40:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:52 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:52 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:52 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:52 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:52.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:52 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:52.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:52 smithi133 ceph-mon[35282]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:52.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:52 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi133 2024-09-10T10:40:52.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:52 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi133 to 8544M 2024-09-10T10:40:52.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:52 smithi133 ceph-mon[35282]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:40:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:52 smithi133 ceph-mon[35282]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T10:40:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:52 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:52 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:52 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:40:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:52 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:40:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:52 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:40:53.849 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:53 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:40:53.849 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:53 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:40:53.849 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:53 smithi007 ceph-mon[30873]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:53.849 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:53.849 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:53 smithi007 ceph-mon[30873]: osd.5 [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] boot 2024-09-10T10:40:53.849 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:53 smithi007 ceph-mon[30873]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T10:40:53.850 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:53.850 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:53 smithi007 ceph-mon[30873]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:53.850 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:53 smithi007 ceph-mon[30873]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T10:40:53.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:53 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:40:53.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:53 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:40:53.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:53 smithi103 ceph-mon[35160]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:53.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:53.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:53 smithi103 ceph-mon[35160]: osd.5 [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] boot 2024-09-10T10:40:53.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:53 smithi103 ceph-mon[35160]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T10:40:53.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:53.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:53 smithi103 ceph-mon[35160]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:53.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:53 smithi103 ceph-mon[35160]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T10:40:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:53 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:40:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:53 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:40:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:53 smithi133 ceph-mon[35282]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:40:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:53 smithi133 ceph-mon[35282]: osd.5 [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] boot 2024-09-10T10:40:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:53 smithi133 ceph-mon[35282]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T10:40:53.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:40:53.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:53 smithi133 ceph-mon[35282]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-09-10T10:40:53.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:53 smithi133 ceph-mon[35282]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T10:40:54.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:54 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1851242690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a5904d5-36de-47b0-97fa-5a10b24d0c17"}]: dispatch 2024-09-10T10:40:54.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:54 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1851242690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a5904d5-36de-47b0-97fa-5a10b24d0c17"}]': finished 2024-09-10T10:40:54.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:54 smithi103 ceph-mon[35160]: osdmap e37: 7 total, 6 up, 7 in 2024-09-10T10:40:54.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:54 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:40:54.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:54 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1851242690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a5904d5-36de-47b0-97fa-5a10b24d0c17"}]: dispatch 2024-09-10T10:40:54.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:54 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1851242690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a5904d5-36de-47b0-97fa-5a10b24d0c17"}]': finished 2024-09-10T10:40:54.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:54 smithi007 ceph-mon[30873]: osdmap e37: 7 total, 6 up, 7 in 2024-09-10T10:40:54.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:54 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:40:54.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:54 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1851242690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a5904d5-36de-47b0-97fa-5a10b24d0c17"}]: dispatch 2024-09-10T10:40:54.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:54 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1851242690' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a5904d5-36de-47b0-97fa-5a10b24d0c17"}]': finished 2024-09-10T10:40:54.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:54 smithi133 ceph-mon[35282]: osdmap e37: 7 total, 6 up, 7 in 2024-09-10T10:40:54.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:54 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:40:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:55 smithi103 ceph-mon[35160]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:40:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:55 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1446006060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:55 smithi103 ceph-mon[35160]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T10:40:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:55 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:40:55.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:55 smithi007 ceph-mon[30873]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:40:55.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:55 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1446006060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:55.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:55 smithi007 ceph-mon[30873]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T10:40:55.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:55 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:40:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:55 smithi133 ceph-mon[35282]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:40:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:55 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1446006060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:40:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:55 smithi133 ceph-mon[35282]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T10:40:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:55 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:40:57.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:57 smithi103 ceph-mon[35160]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T10:40:57.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:40:57.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:57 smithi007 ceph-mon[30873]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T10:40:57.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:40:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:57 smithi133 ceph-mon[35282]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T10:40:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:40:58.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:40:58 smithi103 ceph-mon[35160]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:40:58.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:40:58 smithi007 ceph-mon[30873]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:40:58.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:40:58 smithi133 ceph-mon[35282]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:00.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:00 smithi007 ceph-mon[30873]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-10T10:41:00.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:00 smithi103 ceph-mon[35160]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-10T10:41:00.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:00 smithi133 ceph-mon[35282]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-10T10:41:02.632 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:02 smithi007 ceph-mon[30873]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T10:41:02.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:02 smithi103 ceph-mon[35160]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T10:41:02.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:02 smithi133 ceph-mon[35282]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-10T10:41:04.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:04 smithi007 ceph-mon[30873]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:41:04.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:04 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T10:41:04.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:04 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:04.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:04 smithi007 ceph-mon[30873]: Deploying daemon osd.6 on smithi007 2024-09-10T10:41:04.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:04 smithi103 ceph-mon[35160]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:41:04.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:04 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T10:41:04.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:04 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:04.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:04 smithi103 ceph-mon[35160]: Deploying daemon osd.6 on smithi007 2024-09-10T10:41:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:04 smithi133 ceph-mon[35282]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:41:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:04 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T10:41:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:04 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:04.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:04 smithi133 ceph-mon[35282]: Deploying daemon osd.6 on smithi007 2024-09-10T10:41:06.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:06 smithi103 ceph-mon[35160]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T10:41:06.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:06 smithi007 ceph-mon[30873]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T10:41:06.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:06 smithi133 ceph-mon[35282]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-10T10:41:08.519 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:08 smithi007 ceph-mon[30873]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T10:41:08.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:08 smithi103 ceph-mon[35160]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T10:41:08.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:08 smithi133 ceph-mon[35282]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T10:41:10.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:10 smithi007 ceph-mon[30873]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:41:10.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:10 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:10.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:10 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:10.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:10 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:10 smithi103 ceph-mon[35160]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:41:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:10.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:10 smithi133 ceph-mon[35282]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:41:10.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:10 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:10.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:10 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:10.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:10 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:12.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:12 smithi007 ceph-mon[30873]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:12.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:12.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:12.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:12.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:12.599 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:12 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:12.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:12 smithi103 ceph-mon[35160]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:12.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:12.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:12.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:12.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:12.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:12 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:12.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:12 smithi133 ceph-mon[35282]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:12.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:12.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:12.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:12.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:12.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:12 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:12.912 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 6 on host 'smithi007' 2024-09-10T10:41:12.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.911+0000 7f98357fa640 1 -- 172.21.15.7:0/3175899204 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9838062630 con 0x7f98200a2b80 2024-09-10T10:41:12.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.913+0000 7f983e53d640 1 -- 172.21.15.7:0/3175899204 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f98200a2b80 msgr2=0x7f98200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.913+0000 7f983e53d640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f98200a2b80 0x7f98200a5040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98280058c0 tx=0x7f982800a2d0 comp rx=0 tx=0).stop 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.913+0000 7f983e53d640 1 -- 172.21.15.7:0/3175899204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98381026b0 msgr2=0x7f983810a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.913+0000 7f983e53d640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98381026b0 0x7f983810a000 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9830005e30 tx=0x7f9830031ee0 comp rx=0 tx=0).stop 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.913+0000 7f983e53d640 1 -- 172.21.15.7:0/3175899204 shutdown_connections 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.913+0000 7f983e53d640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f98200a2b80 0x7f98200a5040 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.913+0000 7f983e53d640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98381026b0 0x7f983810a000 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.913+0000 7f983e53d640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9838101cf0 0x7f9838109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.913+0000 7f983e53d640 1 --2- 172.21.15.7:0/3175899204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9838100af0 0x7f983810b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.913+0000 7f983e53d640 1 -- 172.21.15.7:0/3175899204 >> 172.21.15.7:0/3175899204 conn(0x7f98380fc2a0 msgr2=0x7f9838105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.914+0000 7f983e53d640 1 -- 172.21.15.7:0/3175899204 shutdown_connections 2024-09-10T10:41:12.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:41:12.914+0000 7f983e53d640 1 -- 172.21.15.7:0/3175899204 wait complete. 2024-09-10T10:41:13.538 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:13 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:13.538 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:13 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:13.538 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:13 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:13.540 DEBUG:teuthology.orchestra.run.smithi007:osd.6> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T10:41:13.543 INFO:tasks.cephadm:Deploying osd.7 on smithi103 with /dev/vg_nvme/lv_2... 2024-09-10T10:41:13.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T10:41:13.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:13 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:13.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:13 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:13.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:13 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:13.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:13 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:13.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:13 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:13.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:13 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:13.798 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:41:14.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:14 smithi103 ceph-mon[35160]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:14.680 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:14 smithi007 ceph-mon[30873]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:14.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:14 smithi133 ceph-mon[35282]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:15.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 10:41:15 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T10:41:15.486+0000 7efcc3206640 -1 osd.6 0 log_to_monitors true 2024-09-10T10:41:16.249 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:41:16.276 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-09-10T10:41:16.531 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:41:16.569 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:16.569 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T10:41:16.569 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.569 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.569 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:16.569 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:16.569 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:16.569 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.570 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:16.570 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:16.570 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:16 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T10:41:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:16.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:16.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:16 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:16.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T10:41:16.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:16.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:16.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:16.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:16.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:16.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:16.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:16 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:17.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.035+0000 7f7043d85640 1 -- 172.21.15.103:0/3470422119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f703c104ff0 msgr2=0x7f703c1073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.035+0000 7f7043d85640 1 --2- 172.21.15.103:0/3470422119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f703c104ff0 0x7f703c1073e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f702c009900 tx=0x7f702c02a410 comp rx=0 tx=0).stop 2024-09-10T10:41:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.035+0000 7f7043d85640 1 -- 172.21.15.103:0/3470422119 shutdown_connections 2024-09-10T10:41:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.035+0000 7f7043d85640 1 --2- 172.21.15.103:0/3470422119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c107920 0x7f703c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.035+0000 7f7043d85640 1 --2- 172.21.15.103:0/3470422119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f703c104ff0 0x7f703c1073e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.035+0000 7f7043d85640 1 --2- 172.21.15.103:0/3470422119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f703c1026c0 0x7f703c104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 -- 172.21.15.103:0/3470422119 >> 172.21.15.103:0/3470422119 conn(0x7f703c0fc280 msgr2=0x7f703c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 -- 172.21.15.103:0/3470422119 shutdown_connections 2024-09-10T10:41:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 -- 172.21.15.103:0/3470422119 wait complete. 2024-09-10T10:41:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 Processor -- start 2024-09-10T10:41:17.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 -- start start 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f703c1026c0 0x7f703c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c104ff0 0x7f703c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f703c107920 0x7f703c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f703c10d850 con 0x7f703c104ff0 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f703c10d6d0 con 0x7f703c1026c0 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.036+0000 7f7043d85640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f703c10d9d0 con 0x7f703c107920 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041b21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f703c1026c0 0x7f703c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041320640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c104ff0 0x7f703c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041b21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f703c1026c0 0x7f703c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39096/0 (socket says 172.21.15.103:39096) 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041b21640 1 -- 172.21.15.103:0/194406732 learned_addr learned my addr 172.21.15.103:0/194406732 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7042322640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f703c107920 0x7f703c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:17.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041b21640 1 -- 172.21.15.103:0/194406732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f703c107920 msgr2=0x7f703c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041b21640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f703c107920 0x7f703c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041b21640 1 -- 172.21.15.103:0/194406732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c104ff0 msgr2=0x7f703c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041b21640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c104ff0 0x7f703c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041b21640 1 -- 172.21.15.103:0/194406732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c100530 con 0x7f703c1026c0 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041320640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c104ff0 0x7f703c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7041b21640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f703c1026c0 0x7f703c197f60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f703800e9f0 tx=0x7f703800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7032ffd640 1 -- 172.21.15.103:0/194406732 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703800cd80 con 0x7f703c1026c0 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7043d85640 1 -- 172.21.15.103:0/194406732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c100820 con 0x7f703c1026c0 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7032ffd640 1 -- 172.21.15.103:0/194406732 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7038005500 con 0x7f703c1026c0 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7032ffd640 1 -- 172.21.15.103:0/194406732 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038010690 con 0x7f703c1026c0 2024-09-10T10:41:17.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.037+0000 7f7043d85640 1 -- 172.21.15.103:0/194406732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c100ce0 con 0x7f703c1026c0 2024-09-10T10:41:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.038+0000 7f7043d85640 1 -- 172.21.15.103:0/194406732 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c02d7c0 con 0x7f703c1026c0 2024-09-10T10:41:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.039+0000 7f7032ffd640 1 -- 172.21.15.103:0/194406732 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f70380046c0 con 0x7f703c1026c0 2024-09-10T10:41:17.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.040+0000 7f7032ffd640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70280a2bb0 0x7f70280a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.040+0000 7f7032ffd640 1 -- 172.21.15.103:0/194406732 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4669+0+0 (secure 0 0 0) 0x7f70380ce5e0 con 0x7f703c1026c0 2024-09-10T10:41:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.040+0000 7f7041320640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70280a2bb0 0x7f70280a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:17.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.041+0000 7f7041320640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70280a2bb0 0x7f70280a5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f702c005ab0 tx=0x7f702c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:41:17.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.044+0000 7f7032ffd640 1 -- 172.21.15.103:0/194406732 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f703809c380 con 0x7f703c1026c0 2024-09-10T10:41:17.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:17.221+0000 7f7043d85640 1 -- 172.21.15.103:0/194406732 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f703c0fd950 con 0x7f70280a2bb0 2024-09-10T10:41:17.555 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:17 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi007 2024-09-10T10:41:17.555 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:17 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi007 to 5697M 2024-09-10T10:41:17.555 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:17 smithi103 ceph-mon[35160]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T10:41:17.555 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:17 smithi103 ceph-mon[35160]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T10:41:17.555 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:17 smithi103 ceph-mon[35160]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:41:17.555 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:17 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:17.555 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:17 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:17.555 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:17 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:17.556 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:17 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:17.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:17 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi007 2024-09-10T10:41:17.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:17 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi007 to 5697M 2024-09-10T10:41:17.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:17 smithi007 ceph-mon[30873]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T10:41:17.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:17 smithi007 ceph-mon[30873]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T10:41:17.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:17 smithi007 ceph-mon[30873]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:41:17.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:17 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:17.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:17 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:17.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:17 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:17.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:17 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:17.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:17 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi007 2024-09-10T10:41:17.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:17 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi007 to 5697M 2024-09-10T10:41:17.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:17 smithi133 ceph-mon[35282]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T10:41:17.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:17 smithi133 ceph-mon[35282]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T10:41:17.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:17 smithi133 ceph-mon[35282]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:41:17.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:17 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:17.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:17 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:17.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:17 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:17.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:17 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:18.415 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 10:41:18 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T10:41:18.111+0000 7efcbe966640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T10:41:18.415 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 10:41:18 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T10:41:18.120+0000 7efcb6f98640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:41:18.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:18 smithi103 ceph-mon[35160]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:18.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:18 smithi103 ceph-mon[35160]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:18.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:18 smithi103 ceph-mon[35160]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:41:18.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:18 smithi103 ceph-mon[35160]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T10:41:18.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:18.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:18.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:18 smithi103 ceph-mon[35160]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' 2024-09-10T10:41:18.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:18 smithi007 ceph-mon[30873]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:18.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:18 smithi007 ceph-mon[30873]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:18.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:18 smithi007 ceph-mon[30873]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:41:18.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:18 smithi007 ceph-mon[30873]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T10:41:18.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:18.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:18.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:18 smithi007 ceph-mon[30873]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' 2024-09-10T10:41:18.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:18 smithi133 ceph-mon[35282]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:18.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:18 smithi133 ceph-mon[35282]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:18.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:18 smithi133 ceph-mon[35282]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:41:18.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:18 smithi133 ceph-mon[35282]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T10:41:18.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:18.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:18.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:18 smithi133 ceph-mon[35282]: from='osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734]' entity='osd.6' 2024-09-10T10:41:19.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:19 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:41:19.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:19 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:41:19.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:19 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:19.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:19 smithi103 ceph-mon[35160]: osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] boot 2024-09-10T10:41:19.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:19 smithi103 ceph-mon[35160]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T10:41:19.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:19 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:19.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:19 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:41:19.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:19 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:41:19.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:19 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:19.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:19 smithi007 ceph-mon[30873]: osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] boot 2024-09-10T10:41:19.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:19 smithi007 ceph-mon[30873]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T10:41:19.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:19 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:19.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:19 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:41:19.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:19 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:41:19.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:19 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:19.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:19 smithi133 ceph-mon[35282]: osd.6 [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] boot 2024-09-10T10:41:19.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:19 smithi133 ceph-mon[35282]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T10:41:19.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:19 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:41:20.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:20 smithi007 ceph-mon[30873]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:20.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:20 smithi007 ceph-mon[30873]: from='client.? 172.21.15.103:0/3236474736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91e79d42-decd-4405-b9f2-9480f7edfca8"}]: dispatch 2024-09-10T10:41:20.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:20 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91e79d42-decd-4405-b9f2-9480f7edfca8"}]: dispatch 2024-09-10T10:41:20.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:20 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91e79d42-decd-4405-b9f2-9480f7edfca8"}]': finished 2024-09-10T10:41:20.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:20 smithi007 ceph-mon[30873]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T10:41:20.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:20 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:20.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:20 smithi007 ceph-mon[30873]: from='client.? 172.21.15.103:0/3451220046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:20.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:20 smithi133 ceph-mon[35282]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:20.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:20 smithi133 ceph-mon[35282]: from='client.? 172.21.15.103:0/3236474736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91e79d42-decd-4405-b9f2-9480f7edfca8"}]: dispatch 2024-09-10T10:41:20.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:20 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91e79d42-decd-4405-b9f2-9480f7edfca8"}]: dispatch 2024-09-10T10:41:20.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:20 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91e79d42-decd-4405-b9f2-9480f7edfca8"}]': finished 2024-09-10T10:41:20.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:20 smithi133 ceph-mon[35282]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T10:41:20.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:20 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:20.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:20 smithi133 ceph-mon[35282]: from='client.? 172.21.15.103:0/3451220046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:20 smithi103 ceph-mon[35160]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T10:41:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:20 smithi103 ceph-mon[35160]: from='client.? 172.21.15.103:0/3236474736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91e79d42-decd-4405-b9f2-9480f7edfca8"}]: dispatch 2024-09-10T10:41:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:20 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91e79d42-decd-4405-b9f2-9480f7edfca8"}]: dispatch 2024-09-10T10:41:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:20 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91e79d42-decd-4405-b9f2-9480f7edfca8"}]': finished 2024-09-10T10:41:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:20 smithi103 ceph-mon[35160]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T10:41:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:20 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:20 smithi103 ceph-mon[35160]: from='client.? 172.21.15.103:0/3451220046' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:22.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:22 smithi007 ceph-mon[30873]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:22.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:22 smithi133 ceph-mon[35282]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:22 smithi103 ceph-mon[35160]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:24.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:24 smithi007 ceph-mon[30873]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:24.715 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:24 smithi103 ceph-mon[35160]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:24.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:24 smithi133 ceph-mon[35282]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:26.586 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:26 smithi103 ceph-mon[35160]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:26.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:26 smithi007 ceph-mon[30873]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:26.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:26 smithi133 ceph-mon[35282]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:28.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:28 smithi103 ceph-mon[35160]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:28.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T10:41:28.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:28.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:28 smithi133 ceph-mon[35282]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:28.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T10:41:28.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:28.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:28 smithi007 ceph-mon[30873]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:28.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T10:41:28.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:29.719 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:29 smithi103 ceph-mon[35160]: Deploying daemon osd.7 on smithi103 2024-09-10T10:41:29.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:29 smithi133 ceph-mon[35282]: Deploying daemon osd.7 on smithi103 2024-09-10T10:41:29.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:29 smithi007 ceph-mon[30873]: Deploying daemon osd.7 on smithi103 2024-09-10T10:41:30.617 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:30 smithi103 ceph-mon[35160]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:30.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:30 smithi133 ceph-mon[35282]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:30.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:30 smithi007 ceph-mon[30873]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:32.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:32 smithi133 ceph-mon[35282]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:32.838 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:32 smithi103 ceph-mon[35160]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:32.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:32 smithi007 ceph-mon[30873]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:34.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:34 smithi103 ceph-mon[35160]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:34.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:34 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:34.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:34 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:34.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:34 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:34 smithi133 ceph-mon[35282]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:34 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:34 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:34 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:34.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:34 smithi007 ceph-mon[30873]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:34.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:34.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:34.736 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:36.636 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:36 smithi103 ceph-mon[35160]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:36.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:36 smithi133 ceph-mon[35282]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:36.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:36 smithi007 ceph-mon[30873]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:38.033 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:37 smithi103 ceph-mon[35160]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:38.033 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:38.034 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:38.034 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:38.034 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:38.034 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:38.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.074+0000 7f7032ffd640 1 -- 172.21.15.103:0/194406732 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f703c0fd950 con 0x7f70280a2bb0 2024-09-10T10:41:38.075 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 7 on host 'smithi103' 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 -- 172.21.15.103:0/194406732 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70280a2bb0 msgr2=0x7f70280a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70280a2bb0 0x7f70280a5070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f702c005ab0 tx=0x7f702c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 -- 172.21.15.103:0/194406732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f703c1026c0 msgr2=0x7f703c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f703c1026c0 0x7f703c197f60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f703800e9f0 tx=0x7f703800edf0 comp rx=0 tx=0).stop 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 -- 172.21.15.103:0/194406732 shutdown_connections 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70280a2bb0 0x7f70280a5070 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f703c107920 0x7f703c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f703c104ff0 0x7f703c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 --2- 172.21.15.103:0/194406732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f703c1026c0 0x7f703c197f60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 -- 172.21.15.103:0/194406732 >> 172.21.15.103:0/194406732 conn(0x7f703c0fc280 msgr2=0x7f703c103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 -- 172.21.15.103:0/194406732 shutdown_connections 2024-09-10T10:41:38.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:41:38.076+0000 7f7043d85640 1 -- 172.21.15.103:0/194406732 wait complete. 2024-09-10T10:41:38.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:37 smithi133 ceph-mon[35282]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:38.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:38.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:38.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:38.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:38.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:38.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:37 smithi007 ceph-mon[30873]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:38.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:38.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:38.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:38.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:38.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:38.713 DEBUG:teuthology.orchestra.run.smithi103:osd.7> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T10:41:38.716 INFO:tasks.cephadm:Deploying osd.8 on smithi133 with /dev/vg_nvme/lv_2... 2024-09-10T10:41:38.716 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T10:41:38.969 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:41:39.090 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:38 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:39.090 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:38 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:39.090 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:38 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:39.216 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:38 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:39.216 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:38 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:39.216 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:38 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:38 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:41:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:38 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:39.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:38 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:40.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:40 smithi103 ceph-mon[35160]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:40.342 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:40 smithi133 ceph-mon[35282]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:40.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:40 smithi007 ceph-mon[30873]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:40.875 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 10:41:40 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T10:41:40.427+0000 7fe95bb17640 -1 osd.7 0 log_to_monitors true 2024-09-10T10:41:41.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:41 smithi103 ceph-mon[35160]: from='osd.7 [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:41:41.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:41 smithi103 ceph-mon[35160]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:41:41.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:41 smithi133 ceph-mon[35282]: from='osd.7 [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:41:41.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:41 smithi133 ceph-mon[35282]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:41:41.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:41 smithi007 ceph-mon[30873]: from='osd.7 [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:41:41.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:41 smithi007 ceph-mon[30873]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T10:41:41.451 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:41:41.476 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2024-09-10T10:41:41.730 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:41:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.161+0000 7fed19e1d640 1 -- 172.21.15.133:0/2497955699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed141078c0 msgr2=0x7fed14109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.161+0000 7fed19e1d640 1 --2- 172.21.15.133:0/2497955699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed141078c0 0x7fed14109d10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009ca0 tx=0x7fed0c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:41:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.161+0000 7fed19e1d640 1 -- 172.21.15.133:0/2497955699 shutdown_connections 2024-09-10T10:41:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.161+0000 7fed19e1d640 1 --2- 172.21.15.133:0/2497955699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed141078c0 0x7fed14109d10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.161+0000 7fed19e1d640 1 --2- 172.21.15.133:0/2497955699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed14104f90 0x7fed14107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.161+0000 7fed19e1d640 1 --2- 172.21.15.133:0/2497955699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14102660 0x7fed14104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.161+0000 7fed19e1d640 1 -- 172.21.15.133:0/2497955699 >> 172.21.15.133:0/2497955699 conn(0x7fed140fc280 msgr2=0x7fed140fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:41:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.161+0000 7fed19e1d640 1 -- 172.21.15.133:0/2497955699 shutdown_connections 2024-09-10T10:41:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.161+0000 7fed19e1d640 1 -- 172.21.15.133:0/2497955699 wait complete. 2024-09-10T10:41:42.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.162+0000 7fed19e1d640 1 Processor -- start 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.162+0000 7fed19e1d640 1 -- start start 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.162+0000 7fed19e1d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed14102660 0x7fed14197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.162+0000 7fed19e1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14104f90 0x7fed14198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.162+0000 7fed19e1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed141078c0 0x7fed1419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.162+0000 7fed19e1d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed1410d8d0 con 0x7fed14104f90 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.162+0000 7fed19e1d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed1410d750 con 0x7fed141078c0 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.162+0000 7fed19e1d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed1410da50 con 0x7fed14102660 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed14102660 0x7fed14197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed137fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed14102660 0x7fed14197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45358/0 (socket says 172.21.15.133:45358) 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed137fe640 1 -- 172.21.15.133:0/1302538475 learned_addr learned my addr 172.21.15.133:0/1302538475 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:41:42.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed12ffd640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14104f90 0x7fed14198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed13fff640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed141078c0 0x7fed1419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed137fe640 1 -- 172.21.15.133:0/1302538475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed141078c0 msgr2=0x7fed1419f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed137fe640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed141078c0 0x7fed1419f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed137fe640 1 -- 172.21.15.133:0/1302538475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14104f90 msgr2=0x7fed14198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed137fe640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14104f90 0x7fed14198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed137fe640 1 -- 172.21.15.133:0/1302538475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1419fb50 con 0x7fed14102660 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed137fe640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed14102660 0x7fed14197f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00d950 tx=0x7fecfc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed10ff9640 1 -- 172.21.15.133:0/1302538475 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc017070 con 0x7fed14102660 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed10ff9640 1 -- 172.21.15.133:0/1302538475 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecfc009c30 con 0x7fed14102660 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed19e1d640 1 -- 172.21.15.133:0/1302538475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1419fe40 con 0x7fed14102660 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed10ff9640 1 -- 172.21.15.133:0/1302538475 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc00f6b0 con 0x7fed14102660 2024-09-10T10:41:42.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.163+0000 7fed19e1d640 1 -- 172.21.15.133:0/1302538475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed141a0310 con 0x7fed14102660 2024-09-10T10:41:42.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.165+0000 7fed19e1d640 1 -- 172.21.15.133:0/1302538475 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed140fdd70 con 0x7fed14102660 2024-09-10T10:41:42.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.165+0000 7fed10ff9640 1 -- 172.21.15.133:0/1302538475 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fecfc0109d0 con 0x7fed14102660 2024-09-10T10:41:42.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.166+0000 7fed10ff9640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fed000a2bb0 0x7fed000a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:41:42.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.166+0000 7fed10ff9640 1 -- 172.21.15.133:0/1302538475 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5117+0+0 (secure 0 0 0) 0x7fecfc0ce050 con 0x7fed14102660 2024-09-10T10:41:42.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.167+0000 7fed12ffd640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fed000a2bb0 0x7fed000a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:41:42.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.167+0000 7fed12ffd640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fed000a2bb0 0x7fed000a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed14199440 tx=0x7fed04008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:41:42.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.170+0000 7fed10ff9640 1 -- 172.21.15.133:0/1302538475 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fecfc09bc30 con 0x7fed14102660 2024-09-10T10:41:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:41:42.367+0000 7fed19e1d640 1 -- 172.21.15.133:0/1302538475 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fed14062630 con 0x7fed000a2bb0 2024-09-10T10:41:42.369 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:42.369 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T10:41:42.369 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T10:41:42.369 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:42.369 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='osd.7 [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:41:42.369 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:41:42.369 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi103 2024-09-10T10:41:42.370 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.370 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.370 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:42.370 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:42.370 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:42.370 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-10T10:41:42.370 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.370 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:42.370 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:42.370 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:42 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T10:41:42.084+0000 7fe957277640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T10:41:42.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T10:41:42.096+0000 7fe9538b1640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='osd.7 [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi103 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-10T10:41:42.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:42.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:42.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:42 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:42.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T10:41:42.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T10:41:42.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:42.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='osd.7 [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:41:42.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:41:42.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi103 2024-09-10T10:41:42.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:42.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:42.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:41:42.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-10T10:41:42.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:42.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:42.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:41:42.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:42 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:41:43.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:43 smithi103 ceph-mon[35160]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:41:43.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:43 smithi103 ceph-mon[35160]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T10:41:43.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:43 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:43.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:43 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:43.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:43 smithi103 ceph-mon[35160]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:43.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:43 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:43.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:43 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:43.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:43 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:43.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:43 smithi007 ceph-mon[30873]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:41:43.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:43 smithi007 ceph-mon[30873]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T10:41:43.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:43.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:43.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:43 smithi007 ceph-mon[30873]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:43.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:43.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:43.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:43 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:43.474 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:43 smithi133 ceph-mon[35282]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:41:43.474 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:43 smithi133 ceph-mon[35282]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T10:41:43.474 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:43 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:43.474 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:43 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:43.474 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:43 smithi133 ceph-mon[35282]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:41:43.474 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:43 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:41:43.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:43 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:41:43.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:43 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:44.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:44 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:41:44.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:44 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:41:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:44 smithi103 ceph-mon[35160]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:44 smithi103 ceph-mon[35160]: osd.7 [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] boot 2024-09-10T10:41:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:44 smithi103 ceph-mon[35160]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T10:41:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:44 smithi103 ceph-mon[35160]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T10:41:44.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:44 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:41:44.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:44 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:41:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:44 smithi007 ceph-mon[30873]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:44 smithi007 ceph-mon[30873]: osd.7 [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] boot 2024-09-10T10:41:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:44 smithi007 ceph-mon[30873]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T10:41:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:44 smithi007 ceph-mon[30873]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T10:41:44.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:44 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:41:44.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:44 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:41:44.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:44 smithi133 ceph-mon[35282]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-10T10:41:44.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:44.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:44 smithi133 ceph-mon[35282]: osd.7 [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] boot 2024-09-10T10:41:44.452 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:44 smithi133 ceph-mon[35282]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T10:41:44.452 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:41:44.452 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:44 smithi133 ceph-mon[35282]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T10:41:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:45 smithi103 ceph-mon[35160]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T10:41:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:45 smithi103 ceph-mon[35160]: from='client.? 172.21.15.133:0/4252724646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64373754-e81a-430c-967d-4b4f289f3473"}]: dispatch 2024-09-10T10:41:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:45 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64373754-e81a-430c-967d-4b4f289f3473"}]: dispatch 2024-09-10T10:41:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:45 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64373754-e81a-430c-967d-4b4f289f3473"}]': finished 2024-09-10T10:41:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:45 smithi103 ceph-mon[35160]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T10:41:45.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:45 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:41:45.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:45 smithi103 ceph-mon[35160]: from='client.? 172.21.15.133:0/2236287673' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:45 smithi007 ceph-mon[30873]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T10:41:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:45 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/4252724646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64373754-e81a-430c-967d-4b4f289f3473"}]: dispatch 2024-09-10T10:41:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:45 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64373754-e81a-430c-967d-4b4f289f3473"}]: dispatch 2024-09-10T10:41:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:45 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64373754-e81a-430c-967d-4b4f289f3473"}]': finished 2024-09-10T10:41:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:45 smithi007 ceph-mon[30873]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T10:41:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:45 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:41:45.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:45 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/2236287673' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:45.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:45 smithi133 ceph-mon[35282]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T10:41:45.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:45 smithi133 ceph-mon[35282]: from='client.? 172.21.15.133:0/4252724646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64373754-e81a-430c-967d-4b4f289f3473"}]: dispatch 2024-09-10T10:41:45.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:45 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64373754-e81a-430c-967d-4b4f289f3473"}]: dispatch 2024-09-10T10:41:45.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:45 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64373754-e81a-430c-967d-4b4f289f3473"}]': finished 2024-09-10T10:41:45.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:45 smithi133 ceph-mon[35282]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T10:41:45.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:45 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:41:45.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:45 smithi133 ceph-mon[35282]: from='client.? 172.21.15.133:0/2236287673' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:41:46.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:46 smithi103 ceph-mon[35160]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:46.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:46 smithi103 ceph-mon[35160]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T10:41:46.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:41:46.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:46 smithi007 ceph-mon[30873]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:46.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:46 smithi007 ceph-mon[30873]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T10:41:46.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:41:46.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:46 smithi133 ceph-mon[35282]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:46.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:46 smithi133 ceph-mon[35282]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T10:41:46.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:41:48.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:47 smithi103 ceph-mon[35160]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:48.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:47 smithi103 ceph-mon[35160]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T10:41:48.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:47 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:41:48.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:47 smithi007 ceph-mon[30873]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:48.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:47 smithi007 ceph-mon[30873]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T10:41:48.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:47 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:41:48.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:47 smithi133 ceph-mon[35282]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:48.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:47 smithi133 ceph-mon[35282]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T10:41:48.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:47 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:41:50.159 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:49 smithi133 ceph-mon[35282]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:50.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:49 smithi103 ceph-mon[35160]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:50.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:49 smithi007 ceph-mon[30873]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:51.904 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:51 smithi133 ceph-mon[35282]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:52.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:51 smithi103 ceph-mon[35160]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:52.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:51 smithi007 ceph-mon[30873]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:54.090 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:53 smithi133 ceph-mon[35282]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:54.090 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T10:41:54.090 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:54.090 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:53 smithi133 ceph-mon[35282]: Deploying daemon osd.8 on smithi133 2024-09-10T10:41:54.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:53 smithi103 ceph-mon[35160]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:54.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T10:41:54.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:54.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:53 smithi103 ceph-mon[35160]: Deploying daemon osd.8 on smithi133 2024-09-10T10:41:54.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:53 smithi007 ceph-mon[30873]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:54.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T10:41:54.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:41:54.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:53 smithi007 ceph-mon[30873]: Deploying daemon osd.8 on smithi133 2024-09-10T10:41:56.148 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:55 smithi133 ceph-mon[35282]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:56.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:55 smithi103 ceph-mon[35160]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:56.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:55 smithi007 ceph-mon[30873]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:58.135 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:57 smithi133 ceph-mon[35282]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:58.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:57 smithi103 ceph-mon[35160]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:41:58.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:57 smithi007 ceph-mon[30873]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:00.080 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:59 smithi133 ceph-mon[35282]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:00.080 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:59 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:00.080 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:59 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:00.080 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:41:59 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:00.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:59 smithi103 ceph-mon[35160]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:00.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:59 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:00.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:59 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:00.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:41:59 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:00.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:59 smithi007 ceph-mon[30873]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:00.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:59 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:00.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:59 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:00.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:41:59 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:01.899 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:01 smithi133 ceph-mon[35282]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:02.088 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:01 smithi103 ceph-mon[35160]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:02.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:01 smithi007 ceph-mon[30873]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:03.035 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:03.035 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:03.035 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:03.035 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:03.035 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:03.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.074+0000 7fed10ff9640 1 -- 172.21.15.133:0/1302538475 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fed14062630 con 0x7fed000a2bb0 2024-09-10T10:42:03.075 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 8 on host 'smithi133' 2024-09-10T10:42:03.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 -- 172.21.15.133:0/1302538475 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fed000a2bb0 msgr2=0x7fed000a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:03.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fed000a2bb0 0x7fed000a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed14199440 tx=0x7fed04008040 comp rx=0 tx=0).stop 2024-09-10T10:42:03.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 -- 172.21.15.133:0/1302538475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed14102660 msgr2=0x7fed14197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:03.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed14102660 0x7fed14197f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00d950 tx=0x7fecfc00de10 comp rx=0 tx=0).stop 2024-09-10T10:42:03.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 -- 172.21.15.133:0/1302538475 shutdown_connections 2024-09-10T10:42:03.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fed000a2bb0 0x7fed000a5070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed141078c0 0x7fed1419f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed14104f90 0x7fed14198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 --2- 172.21.15.133:0/1302538475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed14102660 0x7fed14197f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:03.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 -- 172.21.15.133:0/1302538475 >> 172.21.15.133:0/1302538475 conn(0x7fed140fc280 msgr2=0x7fed14102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:03.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 -- 172.21.15.133:0/1302538475 shutdown_connections 2024-09-10T10:42:03.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:03.076+0000 7fed19e1d640 1 -- 172.21.15.133:0/1302538475 wait complete. 2024-09-10T10:42:03.295 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:03.295 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:03.295 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:03.295 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:03.296 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:03.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:03.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:03.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:03.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:03.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:03.709 DEBUG:teuthology.orchestra.run.smithi133:osd.8> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T10:42:03.711 INFO:tasks.cephadm:Deploying osd.9 on smithi007 with /dev/vg_nvme/lv_1... 2024-09-10T10:42:03.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T10:42:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:04.270 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:04 smithi007 ceph-mon[30873]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:04.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:04 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:04.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:04 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:04.271 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:04 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:04.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:04 smithi103 ceph-mon[35160]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:04.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:04 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:04.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:04 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:04.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:04 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:04.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:04 smithi133 ceph-mon[35282]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:04.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:04 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:04.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:04 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:04.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:04 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:05.975 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 10:42:05 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T10:42:05.610+0000 7f00baee7640 -1 osd.8 0 log_to_monitors true 2024-09-10T10:42:06.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:06 smithi007 ceph-mon[30873]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:06.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:06 smithi007 ceph-mon[30873]: from='osd.8 [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:42:06.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:06 smithi007 ceph-mon[30873]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:42:06.359 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:06 smithi133 ceph-mon[35282]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:06.360 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:06 smithi133 ceph-mon[35282]: from='osd.8 [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:42:06.360 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:06 smithi133 ceph-mon[35282]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:42:06.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:06 smithi103 ceph-mon[35160]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:06.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:06 smithi103 ceph-mon[35160]: from='osd.8 [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:42:06.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:06 smithi103 ceph-mon[35160]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T10:42:06.672 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:42:06.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_1 2024-09-10T10:42:06.953 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:42:07.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T10:42:07.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T10:42:07.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:07.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='osd.8 [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:42:07.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:42:07.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi133 2024-09-10T10:42:07.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:07.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:07.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:07.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi133 to 5696M 2024-09-10T10:42:07.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:07.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:07.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.382+0000 7f00749e9640 1 -- 172.21.15.7:0/1864978818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0070104fd0 msgr2=0x7f00701073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.382+0000 7f00749e9640 1 --2- 172.21.15.7:0/1864978818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0070104fd0 0x7f00701073c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0064009900 tx=0x7f006402a410 comp rx=0 tx=0).stop 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.382+0000 7f00749e9640 1 -- 172.21.15.7:0/1864978818 shutdown_connections 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.382+0000 7f00749e9640 1 --2- 172.21.15.7:0/1864978818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0070107900 0x7f0070109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.382+0000 7f00749e9640 1 --2- 172.21.15.7:0/1864978818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0070104fd0 0x7f00701073c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.382+0000 7f00749e9640 1 --2- 172.21.15.7:0/1864978818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00701026a0 0x7f0070104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.382+0000 7f00749e9640 1 -- 172.21.15.7:0/1864978818 >> 172.21.15.7:0/1864978818 conn(0x7f00700fc2a0 msgr2=0x7f00700fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.383+0000 7f00749e9640 1 -- 172.21.15.7:0/1864978818 shutdown_connections 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.383+0000 7f00749e9640 1 -- 172.21.15.7:0/1864978818 wait complete. 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.383+0000 7f00749e9640 1 Processor -- start 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.383+0000 7f00749e9640 1 -- start start 2024-09-10T10:42:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.383+0000 7f00749e9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00701026a0 0x7f0070197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.383+0000 7f00749e9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0070104fd0 0x7f00701984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.383+0000 7f00749e9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0070107900 0x7f007019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.383+0000 7f00749e9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f007010d8a0 con 0x7f00701026a0 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.383+0000 7f00749e9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f007010d720 con 0x7f0070104fd0 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.383+0000 7f00749e9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f007010da20 con 0x7f0070107900 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00701026a0 0x7f0070197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00701026a0 0x7f0070197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34266/0 (socket says 172.21.15.7:34266) 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006e59c640 1 -- 172.21.15.7:0/275243627 learned_addr learned my addr 172.21.15.7:0/275243627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006ed9d640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0070107900 0x7f007019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006dd9b640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0070104fd0 0x7f00701984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006e59c640 1 -- 172.21.15.7:0/275243627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0070107900 msgr2=0x7f007019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006e59c640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0070107900 0x7f007019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006e59c640 1 -- 172.21.15.7:0/275243627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0070104fd0 msgr2=0x7f00701984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006e59c640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0070104fd0 0x7f00701984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006e59c640 1 -- 172.21.15.7:0/275243627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007019fbe0 con 0x7f00701026a0 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006dd9b640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0070104fd0 0x7f00701984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f006e59c640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00701026a0 0x7f0070197fb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f006000e9f0 tx=0x7f006000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f005f7fe640 1 -- 172.21.15.7:0/275243627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000cd80 con 0x7f00701026a0 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f005f7fe640 1 -- 172.21.15.7:0/275243627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0060005500 con 0x7f00701026a0 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f00749e9640 1 -- 172.21.15.7:0/275243627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007019fed0 con 0x7f00701026a0 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f005f7fe640 1 -- 172.21.15.7:0/275243627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060010690 con 0x7f00701026a0 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.384+0000 7f00749e9640 1 -- 172.21.15.7:0/275243627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00701a0320 con 0x7f00701026a0 2024-09-10T10:42:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.385+0000 7f00749e9640 1 -- 172.21.15.7:0/275243627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00700fdd90 con 0x7f00701026a0 2024-09-10T10:42:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.387+0000 7f005f7fe640 1 -- 172.21.15.7:0/275243627 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f00600046c0 con 0x7f00701026a0 2024-09-10T10:42:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.387+0000 7f005f7fe640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00580a2c30 0x7f00580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.387+0000 7f006dd9b640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00580a2c30 0x7f00580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.387+0000 7f005f7fe640 1 -- 172.21.15.7:0/275243627 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5549+0+0 (secure 0 0 0) 0x7f00600ce960 con 0x7f00701026a0 2024-09-10T10:42:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.388+0000 7f006dd9b640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00580a2c30 0x7f00580a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0064006010 tx=0x7f0064032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.392+0000 7f005f7fe640 1 -- 172.21.15.7:0/275243627 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f006009c390 con 0x7f00701026a0 2024-09-10T10:42:07.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T10:42:07.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T10:42:07.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:07.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='osd.8 [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:42:07.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:42:07.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi133 2024-09-10T10:42:07.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:07.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:07.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:07.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi133 to 5696M 2024-09-10T10:42:07.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:07.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:07.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T10:42:07.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T10:42:07.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:07.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='osd.8 [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:42:07.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi133 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi133 to 5696M 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:07.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:07.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:07.577+0000 7f00749e9640 1 -- 172.21.15.7:0/275243627 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0070062630 con 0x7f00580a2c30 2024-09-10T10:42:08.225 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T10:42:07.792+0000 7f00b6647640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T10:42:08.225 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 10:42:07 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T10:42:07.802+0000 7f00aec79640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:42:08.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:08 smithi133 ceph-mon[35282]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:08.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:08 smithi133 ceph-mon[35282]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:42:08.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:08 smithi133 ceph-mon[35282]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T10:42:08.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:08.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:08.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:42:08.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:42:08.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:08.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:08 smithi133 ceph-mon[35282]: from='osd.8 ' entity='osd.8' 2024-09-10T10:42:08.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:08 smithi007 ceph-mon[30873]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:08.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:08 smithi007 ceph-mon[30873]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:42:08.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:08 smithi007 ceph-mon[30873]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T10:42:08.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:08.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:08.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:42:08.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:42:08.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:08.268 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:08 smithi007 ceph-mon[30873]: from='osd.8 ' entity='osd.8' 2024-09-10T10:42:08.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:08 smithi103 ceph-mon[35160]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:08.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:08 smithi103 ceph-mon[35160]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:42:08.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:08 smithi103 ceph-mon[35160]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T10:42:08.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:08 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:08.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:08 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:08.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:08 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:42:08.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:08 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:42:08.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:08 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:08.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:08 smithi103 ceph-mon[35160]: from='osd.8 ' entity='osd.8' 2024-09-10T10:42:09.290 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:09 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:42:09.290 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:09 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:42:09.290 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:09 smithi007 ceph-mon[30873]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:09.290 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:09 smithi007 ceph-mon[30873]: osd.8 [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] boot 2024-09-10T10:42:09.290 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:09 smithi007 ceph-mon[30873]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T10:42:09.290 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:09 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:09.290 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:09 smithi007 ceph-mon[30873]: osdmap e55: 9 total, 9 up, 9 in 2024-09-10T10:42:09.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:09 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:42:09.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:09 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:42:09.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:09 smithi103 ceph-mon[35160]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:09.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:09 smithi103 ceph-mon[35160]: osd.8 [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] boot 2024-09-10T10:42:09.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:09 smithi103 ceph-mon[35160]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T10:42:09.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:09 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:09.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:09 smithi103 ceph-mon[35160]: osdmap e55: 9 total, 9 up, 9 in 2024-09-10T10:42:09.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:09 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:42:09.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:09 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:42:09.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:09 smithi133 ceph-mon[35282]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:09.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:09 smithi133 ceph-mon[35282]: osd.8 [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] boot 2024-09-10T10:42:09.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:09 smithi133 ceph-mon[35282]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T10:42:09.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:09 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:42:09.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:09 smithi133 ceph-mon[35282]: osdmap e55: 9 total, 9 up, 9 in 2024-09-10T10:42:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:10 smithi103 ceph-mon[35160]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:10 smithi103 ceph-mon[35160]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T10:42:10.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:10 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3453665828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91ebe2c9-bda3-4596-ac76-5678c71f850b"}]: dispatch 2024-09-10T10:42:10.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:10 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3453665828' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91ebe2c9-bda3-4596-ac76-5678c71f850b"}]': finished 2024-09-10T10:42:10.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:10 smithi103 ceph-mon[35160]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T10:42:10.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:10 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:10.675 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:10 smithi007 ceph-mon[30873]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:10.675 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:10 smithi007 ceph-mon[30873]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T10:42:10.675 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:10 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3453665828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91ebe2c9-bda3-4596-ac76-5678c71f850b"}]: dispatch 2024-09-10T10:42:10.675 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:10 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3453665828' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91ebe2c9-bda3-4596-ac76-5678c71f850b"}]': finished 2024-09-10T10:42:10.675 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:10 smithi007 ceph-mon[30873]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T10:42:10.676 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:10 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:10.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:10 smithi133 ceph-mon[35282]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-10T10:42:10.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:10 smithi133 ceph-mon[35282]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T10:42:10.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:10 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3453665828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91ebe2c9-bda3-4596-ac76-5678c71f850b"}]: dispatch 2024-09-10T10:42:10.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:10 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3453665828' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91ebe2c9-bda3-4596-ac76-5678c71f850b"}]': finished 2024-09-10T10:42:10.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:10 smithi133 ceph-mon[35282]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T10:42:10.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:10 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:11.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:11 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4121536462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:42:11.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:11 smithi103 ceph-mon[35160]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T10:42:11.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:11 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:11.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:11 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4121536462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:42:11.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:11 smithi007 ceph-mon[30873]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T10:42:11.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:11 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:11.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:11 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4121536462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:42:11.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:11 smithi133 ceph-mon[35282]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T10:42:11.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:11 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:12.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:12 smithi103 ceph-mon[35160]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:12.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:12 smithi007 ceph-mon[30873]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:12.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:12 smithi133 ceph-mon[35282]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:14.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:14 smithi103 ceph-mon[35160]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:14.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:14 smithi007 ceph-mon[30873]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:14.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:14 smithi133 ceph-mon[35282]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:16.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:16 smithi103 ceph-mon[35160]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T10:42:16.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:16 smithi007 ceph-mon[30873]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T10:42:16.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:16 smithi133 ceph-mon[35282]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T10:42:18.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:18 smithi007 ceph-mon[30873]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T10:42:18.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:18 smithi133 ceph-mon[35282]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T10:42:18.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:18 smithi103 ceph-mon[35160]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T10:42:19.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:19 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T10:42:19.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:19 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:19.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:19 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T10:42:19.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:19 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:19 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T10:42:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:19 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:20.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:20 smithi007 ceph-mon[30873]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T10:42:20.583 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:20 smithi007 ceph-mon[30873]: Deploying daemon osd.9 on smithi007 2024-09-10T10:42:20.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:20 smithi133 ceph-mon[35282]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T10:42:20.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:20 smithi133 ceph-mon[35282]: Deploying daemon osd.9 on smithi007 2024-09-10T10:42:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:20 smithi103 ceph-mon[35160]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T10:42:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:20 smithi103 ceph-mon[35160]: Deploying daemon osd.9 on smithi007 2024-09-10T10:42:22.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:22 smithi007 ceph-mon[30873]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:42:22.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:22 smithi133 ceph-mon[35282]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:42:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:22 smithi103 ceph-mon[35160]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:42:24.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:24 smithi007 ceph-mon[30873]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T10:42:24.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:24 smithi133 ceph-mon[35282]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T10:42:24.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:24 smithi103 ceph-mon[35160]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T10:42:25.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:25 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:25.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:25 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:25.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:25 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:25.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:25 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:25.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:25 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:25.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:25 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:25 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:25 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:25 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:26.673 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:26 smithi007 ceph-mon[30873]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:42:26.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:26 smithi133 ceph-mon[35282]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:42:26.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:26 smithi103 ceph-mon[35160]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:42:28.322 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 9 on host 'smithi007' 2024-09-10T10:42:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.320+0000 7f005f7fe640 1 -- 172.21.15.7:0/275243627 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0070062630 con 0x7f00580a2c30 2024-09-10T10:42:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 -- 172.21.15.7:0/275243627 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00580a2c30 msgr2=0x7f00580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00580a2c30 0x7f00580a50f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0064006010 tx=0x7f0064032cf0 comp rx=0 tx=0).stop 2024-09-10T10:42:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 -- 172.21.15.7:0/275243627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00701026a0 msgr2=0x7f0070197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00701026a0 0x7f0070197fb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f006000e9f0 tx=0x7f006000edf0 comp rx=0 tx=0).stop 2024-09-10T10:42:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 -- 172.21.15.7:0/275243627 shutdown_connections 2024-09-10T10:42:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00580a2c30 0x7f00580a50f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0070107900 0x7f007019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0070104fd0 0x7f00701984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 --2- 172.21.15.7:0/275243627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00701026a0 0x7f0070197fb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 -- 172.21.15.7:0/275243627 >> 172.21.15.7:0/275243627 conn(0x7f00700fc2a0 msgr2=0x7f0070103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 -- 172.21.15.7:0/275243627 shutdown_connections 2024-09-10T10:42:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:42:28.323+0000 7f00749e9640 1 -- 172.21.15.7:0/275243627 wait complete. 2024-09-10T10:42:28.625 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:28 smithi007 ceph-mon[30873]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:28.625 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.625 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.625 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:28.625 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:28.626 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.626 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:28.626 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.626 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:28 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:28 smithi133 ceph-mon[35282]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:28.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:28 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:28 smithi103 ceph-mon[35160]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:28.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:28.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:28.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:28.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:28 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:28.937 DEBUG:teuthology.orchestra.run.smithi007:osd.9> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T10:42:28.939 INFO:tasks.cephadm:Deploying osd.10 on smithi103 with /dev/vg_nvme/lv_1... 2024-09-10T10:42:28.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T10:42:29.194 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:42:30.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:30 smithi007 ceph-mon[30873]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:30.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:30 smithi133 ceph-mon[35282]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:30.727 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:30 smithi103 ceph-mon[35160]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:31.313 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 10:42:31 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T10:42:31.009+0000 7fb9e8297640 -1 osd.9 0 log_to_monitors true 2024-09-10T10:42:31.615 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:31 smithi103 ceph-mon[35160]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T10:42:31.618 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-10T10:42:31.644 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-09-10T10:42:31.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:31 smithi007 ceph-mon[30873]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T10:42:31.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:31 smithi133 ceph-mon[35282]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T10:42:31.893 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:42:32.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 -- 172.21.15.103:0/787153195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf581026d0 msgr2=0x7faf58108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:32.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 --2- 172.21.15.103:0/787153195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf581026d0 0x7faf58108f60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf50009ca0 tx=0x7faf5002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 -- 172.21.15.103:0/787153195 shutdown_connections 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 --2- 172.21.15.103:0/787153195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf581026d0 0x7faf58108f60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 --2- 172.21.15.103:0/787153195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf58101d10 0x7faf58102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 --2- 172.21.15.103:0/787153195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf58100b10 0x7faf58100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 -- 172.21.15.103:0/787153195 >> 172.21.15.103:0/787153195 conn(0x7faf580fc2a0 msgr2=0x7faf580fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 -- 172.21.15.103:0/787153195 shutdown_connections 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 -- 172.21.15.103:0/787153195 wait complete. 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 Processor -- start 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.351+0000 7faf5e650640 1 -- start start 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf5e650640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf58100b10 0x7faf5806b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:32.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf5e650640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf58101d10 0x7faf5819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:32.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf5e650640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf581026d0 0x7faf5819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:32.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf5e650640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf5810c840 con 0x7faf581026d0 2024-09-10T10:42:32.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf5e650640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faf5810c6c0 con 0x7faf58101d10 2024-09-10T10:42:32.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf5e650640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf5810c9c0 con 0x7faf58100b10 2024-09-10T10:42:32.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf58101d10 0x7faf5819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:32.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf577fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf58101d10 0x7faf5819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57714/0 (socket says 172.21.15.103:57714) 2024-09-10T10:42:32.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf577fe640 1 -- 172.21.15.103:0/3127759141 learned_addr learned my addr 172.21.15.103:0/3127759141 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-10T10:42:32.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf57fff640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf58100b10 0x7faf5806b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:32.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf5cbed640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf581026d0 0x7faf5819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:32.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf577fe640 1 -- 172.21.15.103:0/3127759141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf58100b10 msgr2=0x7faf5806b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:32.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf577fe640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf58100b10 0x7faf5806b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf577fe640 1 -- 172.21.15.103:0/3127759141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf581026d0 msgr2=0x7faf5819f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:32.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf577fe640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf581026d0 0x7faf5819f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:32.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf577fe640 1 -- 172.21.15.103:0/3127759141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5819fb70 con 0x7faf58101d10 2024-09-10T10:42:32.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf577fe640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf58101d10 0x7faf5819cc60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faf4000d770 tx=0x7faf4000db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:32.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf557fa640 1 -- 172.21.15.103:0/3127759141 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40017070 con 0x7faf58101d10 2024-09-10T10:42:32.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf557fa640 1 -- 172.21.15.103:0/3127759141 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf40009820 con 0x7faf58101d10 2024-09-10T10:42:32.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf5e650640 1 -- 172.21.15.103:0/3127759141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5819fe60 con 0x7faf58101d10 2024-09-10T10:42:32.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf557fa640 1 -- 172.21.15.103:0/3127759141 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40004cd0 con 0x7faf58101d10 2024-09-10T10:42:32.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.352+0000 7faf5e650640 1 -- 172.21.15.103:0/3127759141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf581a0340 con 0x7faf58101d10 2024-09-10T10:42:32.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.353+0000 7faf5e650640 1 -- 172.21.15.103:0/3127759141 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf58107e90 con 0x7faf58101d10 2024-09-10T10:42:32.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.355+0000 7faf557fa640 1 -- 172.21.15.103:0/3127759141 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7faf400099d0 con 0x7faf58101d10 2024-09-10T10:42:32.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.355+0000 7faf557fa640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faf440a2b80 0x7faf440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:32.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.356+0000 7faf557fa640 1 -- 172.21.15.103:0/3127759141 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5965+0+0 (secure 0 0 0) 0x7faf400ce150 con 0x7faf58101d10 2024-09-10T10:42:32.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.356+0000 7faf57fff640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faf440a2b80 0x7faf440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:32.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.356+0000 7faf57fff640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faf440a2b80 0x7faf440a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf5806b870 tx=0x7faf48009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:32.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.360+0000 7faf557fa640 1 -- 172.21.15.103:0/3127759141 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf4009b9e0 con 0x7faf58101d10 2024-09-10T10:42:32.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:32.537+0000 7faf5e650640 1 -- 172.21.15.103:0/3127759141 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7faf58062630 con 0x7faf440a2b80 2024-09-10T10:42:32.539 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:32.539 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.540 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:32.541 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:32.541 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:32.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:32.727 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:32.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T10:42:32.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T10:42:32.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-09-10T10:42:32.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:32.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:32.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:32.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:32.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:32.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:33.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T10:42:33.125+0000 7fb9e39f7640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T10:42:33.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T10:42:33.138+0000 7fb9dc029640 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:42:33.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi007 2024-09-10T10:42:33.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi007 to 4273M 2024-09-10T10:42:33.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:42:33.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T10:42:33.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:33.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:33.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:33.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:42:33.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:42:33.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:33.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:33 smithi133 ceph-mon[35282]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' 2024-09-10T10:42:33.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi007 2024-09-10T10:42:33.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi007 to 4273M 2024-09-10T10:42:33.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:42:33.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T10:42:33.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:33.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:33.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:33.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:42:33.875 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:42:33.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:33.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:33 smithi103 ceph-mon[35160]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' 2024-09-10T10:42:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi007 2024-09-10T10:42:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi007 to 4273M 2024-09-10T10:42:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-09-10T10:42:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T10:42:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:42:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:42:33.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:33.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:33 smithi007 ceph-mon[30873]: from='osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240]' entity='osd.9' 2024-09-10T10:42:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:34 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:42:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:34 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:42:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:34 smithi133 ceph-mon[35282]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:34 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:34 smithi133 ceph-mon[35282]: osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] boot 2024-09-10T10:42:34.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:34 smithi133 ceph-mon[35282]: osdmap e61: 10 total, 10 up, 10 in 2024-09-10T10:42:34.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:34 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:34.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:34 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:42:34.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:34 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:42:34.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:34 smithi007 ceph-mon[30873]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:34.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:34.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:34 smithi007 ceph-mon[30873]: osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] boot 2024-09-10T10:42:34.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:34 smithi007 ceph-mon[30873]: osdmap e61: 10 total, 10 up, 10 in 2024-09-10T10:42:34.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:34 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:34 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:42:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:34 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:42:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:34 smithi103 ceph-mon[35160]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-10T10:42:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:34 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:34 smithi103 ceph-mon[35160]: osd.9 [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] boot 2024-09-10T10:42:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:34 smithi103 ceph-mon[35160]: osdmap e61: 10 total, 10 up, 10 in 2024-09-10T10:42:34.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:34 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:42:35.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:35 smithi103 ceph-mon[35160]: from='client.? 172.21.15.103:0/3854930056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f65568f-89d1-4a67-a222-8e263d34c710"}]: dispatch 2024-09-10T10:42:35.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:35 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f65568f-89d1-4a67-a222-8e263d34c710"}]: dispatch 2024-09-10T10:42:35.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:35 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f65568f-89d1-4a67-a222-8e263d34c710"}]': finished 2024-09-10T10:42:35.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:35 smithi103 ceph-mon[35160]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T10:42:35.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:35 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:35.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:35 smithi103 ceph-mon[35160]: from='client.? 172.21.15.103:0/4274103565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:42:35.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:35 smithi133 ceph-mon[35282]: from='client.? 172.21.15.103:0/3854930056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f65568f-89d1-4a67-a222-8e263d34c710"}]: dispatch 2024-09-10T10:42:35.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:35 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f65568f-89d1-4a67-a222-8e263d34c710"}]: dispatch 2024-09-10T10:42:35.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:35 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f65568f-89d1-4a67-a222-8e263d34c710"}]': finished 2024-09-10T10:42:35.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:35 smithi133 ceph-mon[35282]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T10:42:35.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:35 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:35.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:35 smithi133 ceph-mon[35282]: from='client.? 172.21.15.103:0/4274103565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:42:35.859 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:35 smithi007 ceph-mon[30873]: from='client.? 172.21.15.103:0/3854930056' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f65568f-89d1-4a67-a222-8e263d34c710"}]: dispatch 2024-09-10T10:42:35.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:35 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f65568f-89d1-4a67-a222-8e263d34c710"}]: dispatch 2024-09-10T10:42:35.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:35 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f65568f-89d1-4a67-a222-8e263d34c710"}]': finished 2024-09-10T10:42:35.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:35 smithi007 ceph-mon[30873]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T10:42:35.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:35 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:35.860 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:35 smithi007 ceph-mon[30873]: from='client.? 172.21.15.103:0/4274103565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:42:36.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:36 smithi133 ceph-mon[35282]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:36.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:36 smithi133 ceph-mon[35282]: osdmap e63: 11 total, 10 up, 11 in 2024-09-10T10:42:36.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:36 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:36 smithi103 ceph-mon[35160]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:36 smithi103 ceph-mon[35160]: osdmap e63: 11 total, 10 up, 11 in 2024-09-10T10:42:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:36 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:36.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:36 smithi007 ceph-mon[30873]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:36.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:36 smithi007 ceph-mon[30873]: osdmap e63: 11 total, 10 up, 11 in 2024-09-10T10:42:36.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:36 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:38.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:38 smithi133 ceph-mon[35282]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:38 smithi103 ceph-mon[35160]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:38.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:38 smithi007 ceph-mon[30873]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:40.592 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:40 smithi103 ceph-mon[35160]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:40.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:40 smithi007 ceph-mon[30873]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:40.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:40 smithi133 ceph-mon[35282]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:42.603 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:42 smithi103 ceph-mon[35160]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:42.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:42 smithi007 ceph-mon[30873]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:42.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:42 smithi133 ceph-mon[35282]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:44.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:44 smithi103 ceph-mon[35160]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:44.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T10:42:44.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:44.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:44 smithi007 ceph-mon[30873]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:44.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T10:42:44.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:44.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:44 smithi133 ceph-mon[35282]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:44.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T10:42:44.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:45.668 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:45 smithi103 ceph-mon[35160]: Deploying daemon osd.10 on smithi103 2024-09-10T10:42:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:45 smithi007 ceph-mon[30873]: Deploying daemon osd.10 on smithi103 2024-09-10T10:42:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:45 smithi133 ceph-mon[35282]: Deploying daemon osd.10 on smithi103 2024-09-10T10:42:46.747 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:46 smithi103 ceph-mon[35160]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:46 smithi007 ceph-mon[30873]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:46.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:46 smithi133 ceph-mon[35282]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:48.824 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:48 smithi103 ceph-mon[35160]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:48.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:48 smithi007 ceph-mon[30873]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:48.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:48 smithi133 ceph-mon[35282]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:50.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:50 smithi103 ceph-mon[35160]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:50.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:50 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:50.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:50 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:50.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:50 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:50.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:50 smithi007 ceph-mon[30873]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:50.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:50 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:50.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:50 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:50.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:50 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:50.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:50 smithi133 ceph-mon[35282]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:50.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:50 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:50.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:50 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:50.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:50 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:52.828 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:52 smithi103 ceph-mon[35160]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:52.829 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:52 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:52.829 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:52 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:52.829 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:52 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:52.829 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:52 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:52.829 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:52 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:52.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:52 smithi007 ceph-mon[30873]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:52.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:52 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:52.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:52 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:52.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:52 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:52.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:52 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:52.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:52 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:52.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:52 smithi133 ceph-mon[35282]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:52 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:52 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:52 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:52 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:52.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:52 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:53.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.175+0000 7faf557fa640 1 -- 172.21.15.103:0/3127759141 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7faf58062630 con 0x7faf440a2b80 2024-09-10T10:42:53.178 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 10 on host 'smithi103' 2024-09-10T10:42:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 -- 172.21.15.103:0/3127759141 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faf440a2b80 msgr2=0x7faf440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faf440a2b80 0x7faf440a5040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf5806b870 tx=0x7faf48009110 comp rx=0 tx=0).stop 2024-09-10T10:42:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 -- 172.21.15.103:0/3127759141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf58101d10 msgr2=0x7faf5819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf58101d10 0x7faf5819cc60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faf4000d770 tx=0x7faf4000db70 comp rx=0 tx=0).stop 2024-09-10T10:42:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 -- 172.21.15.103:0/3127759141 shutdown_connections 2024-09-10T10:42:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faf440a2b80 0x7faf440a5040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf581026d0 0x7faf5819f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faf58101d10 0x7faf5819cc60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:53.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 --2- 172.21.15.103:0/3127759141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf58100b10 0x7faf5806b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 -- 172.21.15.103:0/3127759141 >> 172.21.15.103:0/3127759141 conn(0x7faf580fc2a0 msgr2=0x7faf58105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 -- 172.21.15.103:0/3127759141 shutdown_connections 2024-09-10T10:42:53.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-10T10:42:53.178+0000 7faf5e650640 1 -- 172.21.15.103:0/3127759141 wait complete. 2024-09-10T10:42:53.816 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:53.816 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:53.816 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:53.844 DEBUG:teuthology.orchestra.run.smithi103:osd.10> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T10:42:53.846 INFO:tasks.cephadm:Deploying osd.11 on smithi133 with /dev/vg_nvme/lv_1... 2024-09-10T10:42:53.846 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T10:42:53.874 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:53.874 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:53.875 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:53.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:42:53.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:53.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:54.101 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:42:54.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:54 smithi103 ceph-mon[35160]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:54.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:54 smithi007 ceph-mon[30873]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:54.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:54 smithi133 ceph-mon[35282]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:56.372 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 10:42:55 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T10:42:55.966+0000 7f7b6759b640 -1 osd.10 0 log_to_monitors true 2024-09-10T10:42:56.603 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:56 smithi133 ceph-mon[35282]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:56.603 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:56 smithi133 ceph-mon[35282]: from='osd.10 [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:42:56.603 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:56 smithi133 ceph-mon[35282]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:42:56.606 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T10:42:56.632 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2024-09-10T10:42:56.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:56 smithi103 ceph-mon[35160]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:56.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:56 smithi103 ceph-mon[35160]: from='osd.10 [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:42:56.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:56 smithi103 ceph-mon[35160]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:42:56.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:56 smithi007 ceph-mon[30873]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:56.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:56 smithi007 ceph-mon[30873]: from='osd.10 [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:42:56.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:56 smithi007 ceph-mon[30873]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:42:56.950 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:42:57.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.357+0000 7f3000683640 1 -- 172.21.15.133:0/3582681880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff8102e00 msgr2=0x7f2ff8103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:57.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.357+0000 7f3000683640 1 --2- 172.21.15.133:0/3582681880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff8102e00 0x7f2ff8103280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009900 tx=0x7f2ff402a410 comp rx=0 tx=0).stop 2024-09-10T10:42:57.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.357+0000 7f3000683640 1 -- 172.21.15.133:0/3582681880 shutdown_connections 2024-09-10T10:42:57.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.357+0000 7f3000683640 1 --2- 172.21.15.133:0/3582681880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ff81037c0 0x7f2ff810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.357+0000 7f3000683640 1 --2- 172.21.15.133:0/3582681880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff8102e00 0x7f2ff8103280 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.357+0000 7f3000683640 1 --2- 172.21.15.133:0/3582681880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff8101c00 0x7f2ff8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.357+0000 7f3000683640 1 -- 172.21.15.133:0/3582681880 >> 172.21.15.133:0/3582681880 conn(0x7f2ff80fd390 msgr2=0x7f2ff80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:42:57.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.357+0000 7f3000683640 1 -- 172.21.15.133:0/3582681880 shutdown_connections 2024-09-10T10:42:57.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.357+0000 7f3000683640 1 -- 172.21.15.133:0/3582681880 wait complete. 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f3000683640 1 Processor -- start 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f3000683640 1 -- start start 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f3000683640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff8101c00 0x7f2ff8197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f3000683640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ff8102e00 0x7f2ff8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f3000683640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff81037c0 0x7f2ff819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f3000683640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff810d860 con 0x7f2ff81037c0 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f3000683640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff810d6e0 con 0x7f2ff8102e00 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f3000683640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff810d9e0 con 0x7f2ff8101c00 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f2ffe41f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff8101c00 0x7f2ff8197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f2ffe41f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff8101c00 0x7f2ff8197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51640/0 (socket says 172.21.15.133:51640) 2024-09-10T10:42:57.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f2ffe41f640 1 -- 172.21.15.133:0/1462936438 learned_addr learned my addr 172.21.15.133:0/1462936438 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f2ffdc1e640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ff8102e00 0x7f2ff8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f2ffec20640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff81037c0 0x7f2ff819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f2ffe41f640 1 -- 172.21.15.133:0/1462936438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ff8102e00 msgr2=0x7f2ff8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f2ffe41f640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ff8102e00 0x7f2ff8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.358+0000 7f2ffe41f640 1 -- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff81037c0 msgr2=0x7f2ff819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.359+0000 7f2ffe41f640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff81037c0 0x7f2ff819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.359+0000 7f2ffe41f640 1 -- 172.21.15.133:0/1462936438 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff819fb10 con 0x7f2ff8101c00 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.359+0000 7f2ffec20640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff81037c0 0x7f2ff819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.359+0000 7f2ffe41f640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff8101c00 0x7f2ff8197ee0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c970 tx=0x7f2fe800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.359+0000 7f2fef7fe640 1 -- 172.21.15.133:0/1462936438 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8004220 con 0x7f2ff8101c00 2024-09-10T10:42:57.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.359+0000 7f2fef7fe640 1 -- 172.21.15.133:0/1462936438 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe80043c0 con 0x7f2ff8101c00 2024-09-10T10:42:57.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.359+0000 7f2fef7fe640 1 -- 172.21.15.133:0/1462936438 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800f450 con 0x7f2ff8101c00 2024-09-10T10:42:57.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.359+0000 7f3000683640 1 -- 172.21.15.133:0/1462936438 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff819fe00 con 0x7f2ff8101c00 2024-09-10T10:42:57.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.359+0000 7f3000683640 1 -- 172.21.15.133:0/1462936438 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff81a0250 con 0x7f2ff8101c00 2024-09-10T10:42:57.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.360+0000 7f3000683640 1 -- 172.21.15.133:0/1462936438 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff8108fd0 con 0x7f2ff8101c00 2024-09-10T10:42:57.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.361+0000 7f2fef7fe640 1 -- 172.21.15.133:0/1462936438 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2fe80079b0 con 0x7f2ff8101c00 2024-09-10T10:42:57.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.362+0000 7f2fef7fe640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2fe40a2bb0 0x7f2fe40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:42:57.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.362+0000 7f2fef7fe640 1 -- 172.21.15.133:0/1462936438 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6398+0+0 (secure 0 0 0) 0x7f2fe80ce6b0 con 0x7f2ff8101c00 2024-09-10T10:42:57.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.362+0000 7f2ffdc1e640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2fe40a2bb0 0x7f2fe40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:42:57.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.363+0000 7f2ffdc1e640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2fe40a2bb0 0x7f2fe40a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4000c00 tx=0x7f2ff40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:42:57.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.366+0000 7f2fef7fe640 1 -- 172.21.15.133:0/1462936438 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fe8014030 con 0x7f2ff8101c00 2024-09-10T10:42:57.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:42:57.546+0000 7f3000683640 1 -- 172.21.15.133:0/1462936438 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2ff8062630 con 0x7f2fe40a2bb0 2024-09-10T10:42:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T10:42:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: osdmap e64: 11 total, 10 up, 11 in 2024-09-10T10:42:57.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='osd.10 [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:57.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: osdmap e64: 11 total, 10 up, 11 in 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='osd.10 [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: osdmap e64: 11 total, 10 up, 11 in 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='osd.10 [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:42:57.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:57.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:57.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:42:57.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:42:58.672 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T10:42:58.308+0000 7f7b62cfb640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T10:42:58.672 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T10:42:58.322+0000 7f7b5b32d640 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:42:58.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:58.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi103 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: from='osd.10 ' entity='osd.10' 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: osd.10 [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] boot 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: osdmap e66: 11 total, 11 up, 11 in 2024-09-10T10:42:58.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:58 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:58.913 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:58.913 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi103 2024-09-10T10:42:58.913 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-10T10:42:58.913 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:42:58.913 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T10:42:58.913 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:58.914 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:58.914 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:58.914 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:42:58.914 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:42:58.914 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:58.914 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: from='osd.10 ' entity='osd.10' 2024-09-10T10:42:58.914 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: osd.10 [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] boot 2024-09-10T10:42:58.914 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: osdmap e66: 11 total, 11 up, 11 in 2024-09-10T10:42:58.914 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:58 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:42:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi103 2024-09-10T10:42:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-10T10:42:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-10T10:42:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T10:42:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:42:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:42:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:42:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:42:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: from='osd.10 ' entity='osd.10' 2024-09-10T10:42:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: osd.10 [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] boot 2024-09-10T10:42:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: osdmap e66: 11 total, 11 up, 11 in 2024-09-10T10:42:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:58 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:42:59.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:59 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:42:59.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:59 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:42:59.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:42:59 smithi133 ceph-mon[35282]: osdmap e67: 11 total, 11 up, 11 in 2024-09-10T10:42:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:59 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:42:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:59 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:42:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:42:59 smithi103 ceph-mon[35160]: osdmap e67: 11 total, 11 up, 11 in 2024-09-10T10:42:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:59 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:42:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:59 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:42:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:42:59 smithi007 ceph-mon[30873]: osdmap e67: 11 total, 11 up, 11 in 2024-09-10T10:43:00.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:00 smithi133 ceph-mon[35282]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 744 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:00.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:00 smithi133 ceph-mon[35282]: from='client.? 172.21.15.133:0/1258684261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b49839b-32e9-4105-abee-a253f1aa3f7a"}]: dispatch 2024-09-10T10:43:00.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:00 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b49839b-32e9-4105-abee-a253f1aa3f7a"}]: dispatch 2024-09-10T10:43:00.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:00 smithi133 ceph-mon[35282]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b49839b-32e9-4105-abee-a253f1aa3f7a"}]': finished 2024-09-10T10:43:00.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:00 smithi133 ceph-mon[35282]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T10:43:00.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:00 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:00.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:00 smithi133 ceph-mon[35282]: from='client.? 172.21.15.133:0/1657421848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:43:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:00 smithi103 ceph-mon[35160]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 744 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:00.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:00 smithi103 ceph-mon[35160]: from='client.? 172.21.15.133:0/1258684261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b49839b-32e9-4105-abee-a253f1aa3f7a"}]: dispatch 2024-09-10T10:43:00.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:00 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b49839b-32e9-4105-abee-a253f1aa3f7a"}]: dispatch 2024-09-10T10:43:00.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:00 smithi103 ceph-mon[35160]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b49839b-32e9-4105-abee-a253f1aa3f7a"}]': finished 2024-09-10T10:43:00.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:00 smithi103 ceph-mon[35160]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T10:43:00.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:00 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:00.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:00 smithi103 ceph-mon[35160]: from='client.? 172.21.15.133:0/1657421848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:43:00.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:00 smithi007 ceph-mon[30873]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 744 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:00.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:00 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/1258684261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b49839b-32e9-4105-abee-a253f1aa3f7a"}]: dispatch 2024-09-10T10:43:00.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:00 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b49839b-32e9-4105-abee-a253f1aa3f7a"}]: dispatch 2024-09-10T10:43:00.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:00 smithi007 ceph-mon[30873]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b49839b-32e9-4105-abee-a253f1aa3f7a"}]': finished 2024-09-10T10:43:00.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:00 smithi007 ceph-mon[30873]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T10:43:00.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:00 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:00.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:00 smithi007 ceph-mon[30873]: from='client.? 172.21.15.133:0/1657421848' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:43:02.093 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:01 smithi103 ceph-mon[35160]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 744 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:02.093 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:01 smithi103 ceph-mon[35160]: osdmap e69: 12 total, 11 up, 12 in 2024-09-10T10:43:02.093 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:01 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:02.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:01 smithi007 ceph-mon[30873]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 744 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:02.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:01 smithi007 ceph-mon[30873]: osdmap e69: 12 total, 11 up, 12 in 2024-09-10T10:43:02.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:01 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:02.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:01 smithi133 ceph-mon[35282]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 744 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:02.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:01 smithi133 ceph-mon[35282]: osdmap e69: 12 total, 11 up, 12 in 2024-09-10T10:43:02.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:01 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:03.056 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:02 smithi103 ceph-mon[35160]: osdmap e70: 12 total, 11 up, 12 in 2024-09-10T10:43:03.056 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:03.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:02 smithi007 ceph-mon[30873]: osdmap e70: 12 total, 11 up, 12 in 2024-09-10T10:43:03.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:03.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:02 smithi133 ceph-mon[35282]: osdmap e70: 12 total, 11 up, 12 in 2024-09-10T10:43:03.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:04.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:03 smithi007 ceph-mon[30873]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 745 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:03 smithi103 ceph-mon[35160]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 745 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:04.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:03 smithi133 ceph-mon[35282]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 745 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:06.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:05 smithi103 ceph-mon[35160]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-10T10:43:06.173 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:05 smithi133 ceph-mon[35282]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-10T10:43:06.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:05 smithi007 ceph-mon[30873]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-10T10:43:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:07 smithi133 ceph-mon[35282]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T10:43:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:07 smithi103 ceph-mon[35160]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T10:43:08.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:07 smithi007 ceph-mon[30873]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T10:43:08.904 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T10:43:08.904 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:08 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:08.904 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:08 smithi133 ceph-mon[35282]: Deploying daemon osd.11 on smithi133 2024-09-10T10:43:09.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:08 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T10:43:09.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:08 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:09.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:08 smithi103 ceph-mon[35160]: Deploying daemon osd.11 on smithi133 2024-09-10T10:43:09.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T10:43:09.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:08 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:09.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:08 smithi007 ceph-mon[30873]: Deploying daemon osd.11 on smithi133 2024-09-10T10:43:09.969 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:09 smithi133 ceph-mon[35282]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:43:10.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:09 smithi103 ceph-mon[35160]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:43:10.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:09 smithi007 ceph-mon[30873]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T10:43:12.026 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:11 smithi133 ceph-mon[35282]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:43:12.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:11 smithi103 ceph-mon[35160]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:43:12.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:11 smithi007 ceph-mon[30873]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:43:14.059 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:13 smithi133 ceph-mon[35282]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-10T10:43:14.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:13 smithi103 ceph-mon[35160]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-10T10:43:14.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:13 smithi007 ceph-mon[30873]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-10T10:43:14.813 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:14.813 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:14.813 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:15.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:15.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:15.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:15.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:15.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:15.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:16.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:16 smithi103 ceph-mon[35160]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:43:16.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:16 smithi007 ceph-mon[30873]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:43:16.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:16 smithi133 ceph-mon[35282]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T10:43:17.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.699+0000 7f2fef7fe640 1 -- 172.21.15.133:0/1462936438 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2ff8062630 con 0x7f2fe40a2bb0 2024-09-10T10:43:17.701 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 11 on host 'smithi133' 2024-09-10T10:43:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 -- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2fe40a2bb0 msgr2=0x7f2fe40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2fe40a2bb0 0x7f2fe40a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4000c00 tx=0x7f2ff40315d0 comp rx=0 tx=0).stop 2024-09-10T10:43:17.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 -- 172.21.15.133:0/1462936438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff8101c00 msgr2=0x7f2ff8197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff8101c00 0x7f2ff8197ee0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c970 tx=0x7f2fe800ce30 comp rx=0 tx=0).stop 2024-09-10T10:43:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 -- 172.21.15.133:0/1462936438 shutdown_connections 2024-09-10T10:43:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2fe40a2bb0 0x7f2fe40a5070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff81037c0 0x7f2ff819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ff8102e00 0x7f2ff8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 --2- 172.21.15.133:0/1462936438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff8101c00 0x7f2ff8197ee0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 -- 172.21.15.133:0/1462936438 >> 172.21.15.133:0/1462936438 conn(0x7f2ff80fd390 msgr2=0x7f2ff81061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 -- 172.21.15.133:0/1462936438 shutdown_connections 2024-09-10T10:43:17.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:17.702+0000 7f3000683640 1 -- 172.21.15.133:0/1462936438 wait complete. 2024-09-10T10:43:18.305 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:18 smithi133 ceph-mon[35282]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:18.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:18.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:18.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:18.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:18 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.333 DEBUG:teuthology.orchestra.run.smithi133:osd.11> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T10:43:18.335 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T10:43:18.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T10:43:18.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:18 smithi007 ceph-mon[30873]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:18.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:18.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:18.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:18.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:18 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:18 smithi103 ceph-mon[35160]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:18.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:18.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:18.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:18.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:18 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:18.606 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.059+0000 7f828295b640 1 -- 172.21.15.7:0/2368928919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f827c102690 msgr2=0x7f827c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.059+0000 7f828295b640 1 --2- 172.21.15.7:0/2368928919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f827c102690 0x7f827c108f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8274009ca0 tx=0x7f827402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.059+0000 7f828295b640 1 -- 172.21.15.7:0/2368928919 shutdown_connections 2024-09-10T10:43:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.059+0000 7f828295b640 1 --2- 172.21.15.7:0/2368928919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f827c102690 0x7f827c108f20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.059+0000 7f828295b640 1 --2- 172.21.15.7:0/2368928919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c101cd0 0x7f827c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.059+0000 7f828295b640 1 --2- 172.21.15.7:0/2368928919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f827c100ad0 0x7f827c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.059+0000 7f828295b640 1 -- 172.21.15.7:0/2368928919 >> 172.21.15.7:0/2368928919 conn(0x7f827c0fc280 msgr2=0x7f827c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.059+0000 7f828295b640 1 -- 172.21.15.7:0/2368928919 shutdown_connections 2024-09-10T10:43:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f828295b640 1 -- 172.21.15.7:0/2368928919 wait complete. 2024-09-10T10:43:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f828295b640 1 Processor -- start 2024-09-10T10:43:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f828295b640 1 -- start start 2024-09-10T10:43:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f828295b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f827c100ad0 0x7f827c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f828295b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f827c101cd0 0x7f827c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f828295b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c102690 0x7f827c1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f828295b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f827c10c750 con 0x7f827c100ad0 2024-09-10T10:43:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f828295b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f827c10c5d0 con 0x7f827c102690 2024-09-10T10:43:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f828295b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f827c10c8d0 con 0x7f827c101cd0 2024-09-10T10:43:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f827bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f827c100ad0 0x7f827c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f827bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f827c100ad0 0x7f827c06aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42406/0 (socket says 172.21.15.7:42406) 2024-09-10T10:43:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f827bfff640 1 -- 172.21.15.7:0/3932710018 learned_addr learned my addr 172.21.15.7:0/3932710018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.060+0000 7f8273fff640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f827c101cd0 0x7f827c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f8280ef8640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c102690 0x7f827c1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f827bfff640 1 -- 172.21.15.7:0/3932710018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f827c101cd0 msgr2=0x7f827c19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f827bfff640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f827c101cd0 0x7f827c19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f827bfff640 1 -- 172.21.15.7:0/3932710018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c102690 msgr2=0x7f827c1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f827bfff640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c102690 0x7f827c1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f827bfff640 1 -- 172.21.15.7:0/3932710018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c1a1d40 con 0x7f827c100ad0 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f8280ef8640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c102690 0x7f827c1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f827bfff640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f827c100ad0 0x7f827c06aad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f826c00e9c0 tx=0x7f826c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f8279ffb640 1 -- 172.21.15.7:0/3932710018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c00cd80 con 0x7f827c100ad0 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f8279ffb640 1 -- 172.21.15.7:0/3932710018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826c0035f0 con 0x7f827c100ad0 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f828295b640 1 -- 172.21.15.7:0/3932710018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827c1a1fd0 con 0x7f827c100ad0 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f8279ffb640 1 -- 172.21.15.7:0/3932710018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c010690 con 0x7f827c100ad0 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.061+0000 7f828295b640 1 -- 172.21.15.7:0/3932710018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827c1a2430 con 0x7f827c100ad0 2024-09-10T10:43:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.062+0000 7f828295b640 1 -- 172.21.15.7:0/3932710018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827c107ea0 con 0x7f827c100ad0 2024-09-10T10:43:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.064+0000 7f8279ffb640 1 -- 172.21.15.7:0/3932710018 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f826c0037a0 con 0x7f827c100ad0 2024-09-10T10:43:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.065+0000 7f8279ffb640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82680a2bb0 0x7f82680a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.065+0000 7f8273fff640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82680a2bb0 0x7f82680a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.065+0000 7f8279ffb640 1 -- 172.21.15.7:0/3932710018 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6809+0+0 (secure 0 0 0) 0x7f826c0ceec0 con 0x7f827c100ad0 2024-09-10T10:43:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.065+0000 7f8273fff640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82680a2bb0 0x7f82680a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f827c06b050 tx=0x7f826400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.068+0000 7f8279ffb640 1 -- 172.21.15.7:0/3932710018 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f826c09c400 con 0x7f827c100ad0 2024-09-10T10:43:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.244+0000 7f828295b640 1 -- 172.21.15.7:0/3932710018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f827c02da50 con 0x7f827c100ad0 2024-09-10T10:43:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.244+0000 7f8279ffb640 1 -- 172.21.15.7:0/3932710018 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f826c09bd00 con 0x7f827c100ad0 2024-09-10T10:43:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:19.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.246+0000 7f828295b640 1 -- 172.21.15.7:0/3932710018 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82680a2bb0 msgr2=0x7f82680a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:19.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.246+0000 7f828295b640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82680a2bb0 0x7f82680a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f827c06b050 tx=0x7f826400b410 comp rx=0 tx=0).stop 2024-09-10T10:43:19.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.246+0000 7f828295b640 1 -- 172.21.15.7:0/3932710018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f827c100ad0 msgr2=0x7f827c06aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:19.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.246+0000 7f828295b640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f827c100ad0 0x7f827c06aad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f826c00e9c0 tx=0x7f826c00ee80 comp rx=0 tx=0).stop 2024-09-10T10:43:19.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.246+0000 7f828295b640 1 -- 172.21.15.7:0/3932710018 shutdown_connections 2024-09-10T10:43:19.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.246+0000 7f828295b640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82680a2bb0 0x7f82680a5070 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.246+0000 7f828295b640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f827c102690 0x7f827c1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.246+0000 7f828295b640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f827c101cd0 0x7f827c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.246+0000 7f828295b640 1 --2- 172.21.15.7:0/3932710018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f827c100ad0 0x7f827c06aad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.246+0000 7f828295b640 1 -- 172.21.15.7:0/3932710018 >> 172.21.15.7:0/3932710018 conn(0x7f827c0fc280 msgr2=0x7f827c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:19.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.247+0000 7f828295b640 1 -- 172.21.15.7:0/3932710018 shutdown_connections 2024-09-10T10:43:19.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:19.247+0000 7f828295b640 1 -- 172.21.15.7:0/3932710018 wait complete. 2024-09-10T10:43:20.015 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1725964978,"num_in_osds":12,"osd_in_since":1725964979,"num_remapped_pgs":0} 2024-09-10T10:43:20.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:20 smithi133 ceph-mon[35282]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:20.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:20 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3932710018' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:43:20.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:20 smithi007 ceph-mon[30873]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:20.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:20 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3932710018' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:43:20.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:20 smithi103 ceph-mon[35160]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:20.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:20 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3932710018' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:43:20.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 10:43:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T10:43:20.400+0000 7fc926be9640 -1 osd.11 0 log_to_monitors true 2024-09-10T10:43:21.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T10:43:21.291 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:21.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:21 smithi007 ceph-mon[30873]: from='osd.11 [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:43:21.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:21 smithi007 ceph-mon[30873]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:43:21.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:21 smithi103 ceph-mon[35160]: from='osd.11 [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:43:21.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:21 smithi103 ceph-mon[35160]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:43:21.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:21 smithi133 ceph-mon[35282]: from='osd.11 [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:43:21.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:21 smithi133 ceph-mon[35282]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.755+0000 7faea273a640 1 -- 172.21.15.7:0/1529523438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c1037c0 msgr2=0x7fae9c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.755+0000 7faea273a640 1 --2- 172.21.15.7:0/1529523438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c1037c0 0x7fae9c10a050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fae94009ca0 tx=0x7fae9402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.755+0000 7faea273a640 1 -- 172.21.15.7:0/1529523438 shutdown_connections 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.755+0000 7faea273a640 1 --2- 172.21.15.7:0/1529523438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c1037c0 0x7fae9c10a050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.755+0000 7faea273a640 1 --2- 172.21.15.7:0/1529523438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae9c102e00 0x7fae9c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.755+0000 7faea273a640 1 --2- 172.21.15.7:0/1529523438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9c101c00 0x7fae9c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.755+0000 7faea273a640 1 -- 172.21.15.7:0/1529523438 >> 172.21.15.7:0/1529523438 conn(0x7fae9c0fd390 msgr2=0x7fae9c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.755+0000 7faea273a640 1 -- 172.21.15.7:0/1529523438 shutdown_connections 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.755+0000 7faea273a640 1 -- 172.21.15.7:0/1529523438 wait complete. 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7faea273a640 1 Processor -- start 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7faea273a640 1 -- start start 2024-09-10T10:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7faea273a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9c101c00 0x7fae9c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7faea273a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c102e00 0x7fae9c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7faea273a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae9c1037c0 0x7fae9c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7faea273a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c10d860 con 0x7fae9c102e00 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7faea273a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c10d6e0 con 0x7fae9c101c00 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7faea273a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c10d9e0 con 0x7fae9c1037c0 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7fae9b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c102e00 0x7fae9c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7fae9b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c102e00 0x7fae9c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46700/0 (socket says 172.21.15.7:46700) 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.756+0000 7fae9b7fe640 1 -- 172.21.15.7:0/4001162303 learned_addr learned my addr 172.21.15.7:0/4001162303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7faea0cd7640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae9c1037c0 0x7fae9c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae9bfff640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9c101c00 0x7fae9c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae9b7fe640 1 -- 172.21.15.7:0/4001162303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae9c1037c0 msgr2=0x7fae9c19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae9b7fe640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae9c1037c0 0x7fae9c19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae9b7fe640 1 -- 172.21.15.7:0/4001162303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9c101c00 msgr2=0x7fae9c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae9b7fe640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9c101c00 0x7fae9c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae9b7fe640 1 -- 172.21.15.7:0/4001162303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9c19fab0 con 0x7fae9c102e00 2024-09-10T10:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae9bfff640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9c101c00 0x7fae9c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae9b7fe640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c102e00 0x7fae9c1983c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fae9000e9f0 tx=0x7fae9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae997fa640 1 -- 172.21.15.7:0/4001162303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9000cd80 con 0x7fae9c102e00 2024-09-10T10:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae997fa640 1 -- 172.21.15.7:0/4001162303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae90010910 con 0x7fae9c102e00 2024-09-10T10:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7faea273a640 1 -- 172.21.15.7:0/4001162303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9c19fda0 con 0x7fae9c102e00 2024-09-10T10:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7fae997fa640 1 -- 172.21.15.7:0/4001162303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90018990 con 0x7fae9c102e00 2024-09-10T10:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.757+0000 7faea273a640 1 -- 172.21.15.7:0/4001162303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9c1a0260 con 0x7fae9c102e00 2024-09-10T10:43:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.758+0000 7faea273a640 1 -- 172.21.15.7:0/4001162303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9c102000 con 0x7fae9c102e00 2024-09-10T10:43:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.759+0000 7fae997fa640 1 -- 172.21.15.7:0/4001162303 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fae90010ac0 con 0x7fae9c102e00 2024-09-10T10:43:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.760+0000 7fae997fa640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fae840a2c30 0x7fae840a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.760+0000 7fae9bfff640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fae840a2c30 0x7fae840a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.760+0000 7fae997fa640 1 -- 172.21.15.7:0/4001162303 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6831+0+0 (secure 0 0 0) 0x7fae900ced60 con 0x7fae9c102e00 2024-09-10T10:43:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.761+0000 7fae9bfff640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fae840a2c30 0x7fae840a50f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0058c0 tx=0x7fae8c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.764+0000 7fae997fa640 1 -- 172.21.15.7:0/4001162303 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae9009c290 con 0x7fae9c102e00 2024-09-10T10:43:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.939+0000 7faea273a640 1 -- 172.21.15.7:0/4001162303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fae9c062df0 con 0x7fae9c102e00 2024-09-10T10:43:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.939+0000 7fae997fa640 1 -- 172.21.15.7:0/4001162303 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7fae9009bb90 con 0x7fae9c102e00 2024-09-10T10:43:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 -- 172.21.15.7:0/4001162303 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fae840a2c30 msgr2=0x7fae840a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fae840a2c30 0x7fae840a50f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0058c0 tx=0x7fae8c005850 comp rx=0 tx=0).stop 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 -- 172.21.15.7:0/4001162303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c102e00 msgr2=0x7fae9c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c102e00 0x7fae9c1983c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fae9000e9f0 tx=0x7fae9000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 -- 172.21.15.7:0/4001162303 shutdown_connections 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fae840a2c30 0x7fae840a50f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae9c1037c0 0x7fae9c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c102e00 0x7fae9c1983c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 --2- 172.21.15.7:0/4001162303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae9c101c00 0x7fae9c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 -- 172.21.15.7:0/4001162303 >> 172.21.15.7:0/4001162303 conn(0x7fae9c0fd390 msgr2=0x7fae9c1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 -- 172.21.15.7:0/4001162303 shutdown_connections 2024-09-10T10:43:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:21.941+0000 7faea273a640 1 -- 172.21.15.7:0/4001162303 wait complete. 2024-09-10T10:43:22.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:22.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T10:43:22.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: osdmap e71: 12 total, 11 up, 12 in 2024-09-10T10:43:22.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:22.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='osd.11 [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:43:22.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: Detected new or changed devices on smithi133 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi133 to 4272M 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:22.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:22.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:22 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4001162303' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:43:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T10:43:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: osdmap e71: 12 total, 11 up, 12 in 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='osd.11 [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: Detected new or changed devices on smithi133 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi133 to 4272M 2024-09-10T10:43:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:22.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:22.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.424 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:22 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4001162303' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:43:22.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:22.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: osdmap e71: 12 total, 11 up, 12 in 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='osd.11 [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: Detected new or changed devices on smithi133 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:43:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi133 to 4272M 2024-09-10T10:43:22.477 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.477 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:22.477 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:22.477 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:22.477 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4001162303' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:43:22.620 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1725964978,"num_in_osds":12,"osd_in_since":1725964979,"num_remapped_pgs":0} 2024-09-10T10:43:23.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:23 smithi133 ceph-mon[35282]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:43:23.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:23 smithi133 ceph-mon[35282]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T10:43:23.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:23 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:23.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:23 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:23.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:23 smithi133 ceph-mon[35282]: from='osd.11 ' entity='osd.11' 2024-09-10T10:43:23.226 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T10:43:22.910+0000 7fc923380640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T10:43:23.226 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 10:43:22 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T10:43:22.929+0000 7fc91a97b640 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:43:23.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:23 smithi103 ceph-mon[35160]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:43:23.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:23 smithi103 ceph-mon[35160]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T10:43:23.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:23 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:23.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:23 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:23.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:23 smithi103 ceph-mon[35160]: from='osd.11 ' entity='osd.11' 2024-09-10T10:43:23.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:23 smithi007 ceph-mon[30873]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-09-10T10:43:23.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:23 smithi007 ceph-mon[30873]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T10:43:23.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:23 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:23.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:23 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:23.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:23 smithi007 ceph-mon[30873]: from='osd.11 ' entity='osd.11' 2024-09-10T10:43:23.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T10:43:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:24.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:24 smithi007 ceph-mon[30873]: purged_snaps scrub starts 2024-09-10T10:43:24.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:24 smithi007 ceph-mon[30873]: purged_snaps scrub ok 2024-09-10T10:43:24.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:24 smithi007 ceph-mon[30873]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:24.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:24 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:24.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:24 smithi007 ceph-mon[30873]: osd.11 [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] boot 2024-09-10T10:43:24.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:24 smithi007 ceph-mon[30873]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T10:43:24.273 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:24 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.354+0000 7fb3c5058640 1 -- 172.21.15.7:0/3973209304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0100ad0 msgr2=0x7fb3c0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.354+0000 7fb3c5058640 1 --2- 172.21.15.7:0/3973209304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0100ad0 0x7fb3c0100ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009900 tx=0x7fb3b002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.354+0000 7fb3c5058640 1 -- 172.21.15.7:0/3973209304 shutdown_connections 2024-09-10T10:43:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.354+0000 7fb3c5058640 1 --2- 172.21.15.7:0/3973209304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c0102690 0x7fb3c0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.354+0000 7fb3c5058640 1 --2- 172.21.15.7:0/3973209304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.354+0000 7fb3c5058640 1 --2- 172.21.15.7:0/3973209304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0100ad0 0x7fb3c0100ed0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.354+0000 7fb3c5058640 1 -- 172.21.15.7:0/3973209304 >> 172.21.15.7:0/3973209304 conn(0x7fb3c00fc280 msgr2=0x7fb3c00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.354+0000 7fb3c5058640 1 -- 172.21.15.7:0/3973209304 shutdown_connections 2024-09-10T10:43:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.354+0000 7fb3c5058640 1 -- 172.21.15.7:0/3973209304 wait complete. 2024-09-10T10:43:24.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.354+0000 7fb3c5058640 1 Processor -- start 2024-09-10T10:43:24.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3c5058640 1 -- start start 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3c5058640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c0100ad0 0x7fb3c010b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3c5058640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3c5058640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c0102690 0x7fb3c0109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3c5058640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c010d840 con 0x7fb3c0101cd0 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3c5058640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c010d6c0 con 0x7fb3c0102690 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3c5058640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c010d9c0 con 0x7fb3c0100ad0 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3be59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3be59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46722/0 (socket says 172.21.15.7:46722) 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3be59c640 1 -- 172.21.15.7:0/3633473659 learned_addr learned my addr 172.21.15.7:0/3633473659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3bf59e640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c0102690 0x7fb3c0109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3bed9d640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c0100ad0 0x7fb3c010b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3be59c640 1 -- 172.21.15.7:0/3633473659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c0100ad0 msgr2=0x7fb3c010b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3be59c640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c0100ad0 0x7fb3c010b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3be59c640 1 -- 172.21.15.7:0/3633473659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c0102690 msgr2=0x7fb3c0109fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3be59c640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c0102690 0x7fb3c0109fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.355+0000 7fb3be59c640 1 -- 172.21.15.7:0/3633473659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c010a8a0 con 0x7fb3c0101cd0 2024-09-10T10:43:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.356+0000 7fb3be59c640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0109aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400c890 tx=0x7fb3b400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.356+0000 7fb3a7fff640 1 -- 172.21.15.7:0/3633473659 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004220 con 0x7fb3c0101cd0 2024-09-10T10:43:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.356+0000 7fb3a7fff640 1 -- 172.21.15.7:0/3633473659 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b40043c0 con 0x7fb3c0101cd0 2024-09-10T10:43:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.356+0000 7fb3c5058640 1 -- 172.21.15.7:0/3633473659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c01a1ff0 con 0x7fb3c0101cd0 2024-09-10T10:43:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.356+0000 7fb3a7fff640 1 -- 172.21.15.7:0/3633473659 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40049e0 con 0x7fb3c0101cd0 2024-09-10T10:43:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.356+0000 7fb3c5058640 1 -- 172.21.15.7:0/3633473659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c01a24f0 con 0x7fb3c0101cd0 2024-09-10T10:43:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.357+0000 7fb3c5058640 1 -- 172.21.15.7:0/3633473659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c0107e70 con 0x7fb3c0101cd0 2024-09-10T10:43:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.358+0000 7fb3a7fff640 1 -- 172.21.15.7:0/3633473659 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb3b40079b0 con 0x7fb3c0101cd0 2024-09-10T10:43:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.359+0000 7fb3a7fff640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb3a80a2c00 0x7fb3a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.359+0000 7fb3a7fff640 1 -- 172.21.15.7:0/3633473659 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fb3b40cec10 con 0x7fb3c0101cd0 2024-09-10T10:43:24.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.359+0000 7fb3bed9d640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb3a80a2c00 0x7fb3a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.359+0000 7fb3bed9d640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb3a80a2c00 0x7fb3a80a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0006010 tx=0x7fb3b0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.363+0000 7fb3a7fff640 1 -- 172.21.15.7:0/3633473659 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3b4014030 con 0x7fb3c0101cd0 2024-09-10T10:43:24.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:24 smithi103 ceph-mon[35160]: purged_snaps scrub starts 2024-09-10T10:43:24.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:24 smithi103 ceph-mon[35160]: purged_snaps scrub ok 2024-09-10T10:43:24.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:24 smithi103 ceph-mon[35160]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:24.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:24 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:24.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:24 smithi103 ceph-mon[35160]: osd.11 [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] boot 2024-09-10T10:43:24.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:24 smithi103 ceph-mon[35160]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T10:43:24.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:24 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:24.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:24 smithi133 ceph-mon[35282]: purged_snaps scrub starts 2024-09-10T10:43:24.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:24 smithi133 ceph-mon[35282]: purged_snaps scrub ok 2024-09-10T10:43:24.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:24 smithi133 ceph-mon[35282]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:43:24.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:24 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:24.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:24 smithi133 ceph-mon[35282]: osd.11 [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] boot 2024-09-10T10:43:24.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:24 smithi133 ceph-mon[35282]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T10:43:24.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:24 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:43:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.538+0000 7fb3c5058640 1 -- 172.21.15.7:0/3633473659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb3c0062e90 con 0x7fb3c0101cd0 2024-09-10T10:43:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.538+0000 7fb3a7fff640 1 -- 172.21.15.7:0/3633473659 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fb3b409bf70 con 0x7fb3c0101cd0 2024-09-10T10:43:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 -- 172.21.15.7:0/3633473659 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb3a80a2c00 msgr2=0x7fb3a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb3a80a2c00 0x7fb3a80a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0006010 tx=0x7fb3b0000bc0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 -- 172.21.15.7:0/3633473659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0101cd0 msgr2=0x7fb3c0109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0109aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400c890 tx=0x7fb3b400cd50 comp rx=0 tx=0).stop 2024-09-10T10:43:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 -- 172.21.15.7:0/3633473659 shutdown_connections 2024-09-10T10:43:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb3a80a2c00 0x7fb3a80a50c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3c0102690 0x7fb3c0109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3c0101cd0 0x7fb3c0109aa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 --2- 172.21.15.7:0/3633473659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3c0100ad0 0x7fb3c010b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 -- 172.21.15.7:0/3633473659 >> 172.21.15.7:0/3633473659 conn(0x7fb3c00fc280 msgr2=0x7fb3c0105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 -- 172.21.15.7:0/3633473659 shutdown_connections 2024-09-10T10:43:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:24.541+0000 7fb3c5058640 1 -- 172.21.15.7:0/3633473659 wait complete. 2024-09-10T10:43:25.200 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1725965003,"num_in_osds":12,"osd_in_since":1725964979,"num_remapped_pgs":0} 2024-09-10T10:43:25.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:43:25.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:25 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3633473659' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:43:25.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:25 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3633473659' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:43:25.456 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:25.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:25 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3633473659' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:43:25.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.902+0000 7f326aed7640 1 -- 172.21.15.7:0/3658736781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32641049c0 msgr2=0x7f3264108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.902+0000 7f326aed7640 1 --2- 172.21.15.7:0/3658736781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32641049c0 0x7f3264108a10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f324c009900 tx=0x7f324c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.902+0000 7f326aed7640 1 -- 172.21.15.7:0/3658736781 shutdown_connections 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.902+0000 7f326aed7640 1 --2- 172.21.15.7:0/3658736781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3264109140 0x7f3264110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.902+0000 7f326aed7640 1 --2- 172.21.15.7:0/3658736781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32641049c0 0x7f3264108a10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.902+0000 7f326aed7640 1 --2- 172.21.15.7:0/3658736781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3264104010 0x7f32641043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.902+0000 7f326aed7640 1 -- 172.21.15.7:0/3658736781 >> 172.21.15.7:0/3658736781 conn(0x7f32640ffa50 msgr2=0x7f3264101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.902+0000 7f326aed7640 1 -- 172.21.15.7:0/3658736781 shutdown_connections 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.902+0000 7f326aed7640 1 -- 172.21.15.7:0/3658736781 wait complete. 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f326aed7640 1 Processor -- start 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f326aed7640 1 -- start start 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f326aed7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3264104010 0x7f326419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f326aed7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32641049c0 0x7f326419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f326aed7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3264109140 0x7f326419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f326aed7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3264114190 con 0x7f3264104010 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f326aed7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3264114010 con 0x7f3264109140 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f326aed7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3264114310 con 0x7f32641049c0 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f3268c73640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3264104010 0x7f326419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f3268c73640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3264104010 0x7f326419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46732/0 (socket says 172.21.15.7:46732) 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f3268c73640 1 -- 172.21.15.7:0/3700357171 learned_addr learned my addr 172.21.15.7:0/3700357171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f3269474640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3264109140 0x7f326419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f3263fff640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32641049c0 0x7f326419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f3268c73640 1 -- 172.21.15.7:0/3700357171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32641049c0 msgr2=0x7f326419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f3268c73640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32641049c0 0x7f326419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f3268c73640 1 -- 172.21.15.7:0/3700357171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3264109140 msgr2=0x7f326419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f3268c73640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3264109140 0x7f326419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.903+0000 7f3268c73640 1 -- 172.21.15.7:0/3700357171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326419f540 con 0x7f3264104010 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.904+0000 7f3268c73640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3264104010 0x7f326419a590 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f325400cce0 tx=0x7f325400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.904+0000 7f3261ffb640 1 -- 172.21.15.7:0/3700357171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325400f650 con 0x7f3264104010 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.904+0000 7f326aed7640 1 -- 172.21.15.7:0/3700357171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326419f830 con 0x7f3264104010 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.904+0000 7f3261ffb640 1 -- 172.21.15.7:0/3700357171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f325400fcd0 con 0x7f3264104010 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.904+0000 7f3261ffb640 1 -- 172.21.15.7:0/3700357171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254018c70 con 0x7f3264104010 2024-09-10T10:43:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.904+0000 7f326aed7640 1 -- 172.21.15.7:0/3700357171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32641a7110 con 0x7f3264104010 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.905+0000 7f326aed7640 1 -- 172.21.15.7:0/3700357171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32641043f0 con 0x7f3264104010 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.906+0000 7f3261ffb640 1 -- 172.21.15.7:0/3700357171 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f325400f7f0 con 0x7f3264104010 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.907+0000 7f3261ffb640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f32500a2b80 0x7f32500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.907+0000 7f3263fff640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f32500a2b80 0x7f32500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.907+0000 7f3261ffb640 1 -- 172.21.15.7:0/3700357171 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f32540cf090 con 0x7f3264104010 2024-09-10T10:43:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.907+0000 7f3263fff640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f32500a2b80 0x7f32500a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f324c000c00 tx=0x7f324c005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:25.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:25.911+0000 7f3261ffb640 1 -- 172.21.15.7:0/3700357171 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3254014030 con 0x7f3264104010 2024-09-10T10:43:26.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.090+0000 7f326aed7640 1 -- 172.21.15.7:0/3700357171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32641a7340 con 0x7f3264104010 2024-09-10T10:43:26.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.091+0000 7f3261ffb640 1 -- 172.21.15.7:0/3700357171 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18384 (secure 0 0 0) 0x7f325409c3f0 con 0x7f3264104010 2024-09-10T10:43:26.093 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:26.093 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":74,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:36:33.117443+0000","modified":"2024-09-10T10:43:25.070138+0000","last_up_change":"2024-09-10T10:43:23.913978+0000","last_in_change":"2024-09-10T10:42:59.733844+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:39:37.104114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9239e99-3a62-4883-b2f1-8964ebb54492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6803","nonce":704452390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6805","nonce":704452390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6809","nonce":704452390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6807","nonce":704452390}]},"public_addr":"172.21.15.7:6803/704452390","cluster_addr":"172.21.15.7:6805/704452390","heartbeat_back_addr":"172.21.15.7:6809/704452390","heartbeat_front_addr":"172.21.15.7:6807/704452390","state":["exists","up"]},{"osd":1,"uuid":"d750ff16-48c8-421d-b46b-49f0d2b7f4e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6801","nonce":2409158189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6803","nonce":2409158189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6807","nonce":2409158189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6805","nonce":2409158189}]},"public_addr":"172.21.15.103:6801/2409158189","cluster_addr":"172.21.15.103:6803/2409158189","heartbeat_back_addr":"172.21.15.103:6807/2409158189","heartbeat_front_addr":"172.21.15.103:6805/2409158189","state":["exists","up"]},{"osd":2,"uuid":"1ecb070a-a2f1-499a-99e7-f1967ebb52ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6801","nonce":1657349858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6803","nonce":1657349858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6807","nonce":1657349858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6805","nonce":1657349858}]},"public_addr":"172.21.15.133:6801/1657349858","cluster_addr":"172.21.15.133:6803/1657349858","heartbeat_back_addr":"172.21.15.133:6807/1657349858","heartbeat_front_addr":"172.21.15.133:6805/1657349858","state":["exists","up"]},{"osd":3,"uuid":"8e53d79e-038f-4a7f-9ac9-16b60f47bb4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6811","nonce":871781144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6813","nonce":871781144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6817","nonce":871781144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6815","nonce":871781144}]},"public_addr":"172.21.15.7:6811/871781144","cluster_addr":"172.21.15.7:6813/871781144","heartbeat_back_addr":"172.21.15.7:6817/871781144","heartbeat_front_addr":"172.21.15.7:6815/871781144","state":["exists","up"]},{"osd":4,"uuid":"e4f04c13-594c-43dd-9a98-d2138061b1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6809","nonce":1937243043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6811","nonce":1937243043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6815","nonce":1937243043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6813","nonce":1937243043}]},"public_addr":"172.21.15.103:6809/1937243043","cluster_addr":"172.21.15.103:6811/1937243043","heartbeat_back_addr":"172.21.15.103:6815/1937243043","heartbeat_front_addr":"172.21.15.103:6813/1937243043","state":["exists","up"]},{"osd":5,"uuid":"73143170-5816-407d-ae03-dcf896057681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6809","nonce":577882774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6811","nonce":577882774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6815","nonce":577882774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6813","nonce":577882774}]},"public_addr":"172.21.15.133:6809/577882774","cluster_addr":"172.21.15.133:6811/577882774","heartbeat_back_addr":"172.21.15.133:6815/577882774","heartbeat_front_addr":"172.21.15.133:6813/577882774","state":["exists","up"]},{"osd":6,"uuid":"3a5904d5-36de-47b0-97fa-5a10b24d0c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6819","nonce":2646192734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6821","nonce":2646192734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6825","nonce":2646192734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6823","nonce":2646192734}]},"public_addr":"172.21.15.7:6819/2646192734","cluster_addr":"172.21.15.7:6821/2646192734","heartbeat_back_addr":"172.21.15.7:6825/2646192734","heartbeat_front_addr":"172.21.15.7:6823/2646192734","state":["exists","up"]},{"osd":7,"uuid":"91e79d42-decd-4405-b9f2-9480f7edfca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6817","nonce":3858867236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6819","nonce":3858867236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6823","nonce":3858867236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6821","nonce":3858867236}]},"public_addr":"172.21.15.103:6817/3858867236","cluster_addr":"172.21.15.103:6819/3858867236","heartbeat_back_addr":"172.21.15.103:6823/3858867236","heartbeat_front_addr":"172.21.15.103:6821/3858867236","state":["exists","up"]},{"osd":8,"uuid":"64373754-e81a-430c-967d-4b4f289f3473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6817","nonce":131428321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6819","nonce":131428321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6823","nonce":131428321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6821","nonce":131428321}]},"public_addr":"172.21.15.133:6817/131428321","cluster_addr":"172.21.15.133:6819/131428321","heartbeat_back_addr":"172.21.15.133:6823/131428321","heartbeat_front_addr":"172.21.15.133:6821/131428321","state":["exists","up"]},{"osd":9,"uuid":"91ebe2c9-bda3-4596-ac76-5678c71f850b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6827","nonce":945633240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6829","nonce":945633240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6833","nonce":945633240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6831","nonce":945633240}]},"public_addr":"172.21.15.7:6827/945633240","cluster_addr":"172.21.15.7:6829/945633240","heartbeat_back_addr":"172.21.15.7:6833/945633240","heartbeat_front_addr":"172.21.15.7:6831/945633240","state":["exists","up"]},{"osd":10,"uuid":"2f65568f-89d1-4a67-a222-8e263d34c710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6825","nonce":907835659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6827","nonce":907835659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6831","nonce":907835659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6829","nonce":907835659}]},"public_addr":"172.21.15.103:6825/907835659","cluster_addr":"172.21.15.103:6827/907835659","heartbeat_back_addr":"172.21.15.103:6831/907835659","heartbeat_front_addr":"172.21.15.103:6829/907835659","state":["exists","up"]},{"osd":11,"uuid":"4b49839b-32e9-4105-abee-a253f1aa3f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6825","nonce":2538184038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6827","nonce":2538184038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6831","nonce":2538184038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6829","nonce":2538184038}]},"public_addr":"172.21.15.133:6825/2538184038","cluster_addr":"172.21.15.133:6827/2538184038","heartbeat_back_addr":"172.21.15.133:6831/2538184038","heartbeat_front_addr":"172.21.15.133:6829/2538184038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:44.541279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:10.103733+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:35.290268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:01.070763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:25.666941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:50.925733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:16.486537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:41.464155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:06.610089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:31.994080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:57.000696+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:21.398669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3394895925":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6800/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/262340568":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/3768226338":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/258851660":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3411487770":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/11850255":"2024-09-11T10:36:55.407956+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:26.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.093+0000 7f326aed7640 1 -- 172.21.15.7:0/3700357171 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f32500a2b80 msgr2=0x7f32500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:26.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.093+0000 7f326aed7640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f32500a2b80 0x7f32500a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f324c000c00 tx=0x7f324c005df0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.093+0000 7f326aed7640 1 -- 172.21.15.7:0/3700357171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3264104010 msgr2=0x7f326419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:26.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.093+0000 7f326aed7640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3264104010 0x7f326419a590 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f325400cce0 tx=0x7f325400a620 comp rx=0 tx=0).stop 2024-09-10T10:43:26.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.094+0000 7f326aed7640 1 -- 172.21.15.7:0/3700357171 shutdown_connections 2024-09-10T10:43:26.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.094+0000 7f326aed7640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f32500a2b80 0x7f32500a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.094+0000 7f326aed7640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3264109140 0x7f326419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.094+0000 7f326aed7640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32641049c0 0x7f326419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.094+0000 7f326aed7640 1 --2- 172.21.15.7:0/3700357171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3264104010 0x7f326419a590 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.094+0000 7f326aed7640 1 -- 172.21.15.7:0/3700357171 >> 172.21.15.7:0/3700357171 conn(0x7f32640ffa50 msgr2=0x7f3264101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:26.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.094+0000 7f326aed7640 1 -- 172.21.15.7:0/3700357171 shutdown_connections 2024-09-10T10:43:26.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:26.094+0000 7f326aed7640 1 -- 172.21.15.7:0/3700357171 wait complete. 2024-09-10T10:43:26.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:26 smithi007 ceph-mon[30873]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:26.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:26 smithi007 ceph-mon[30873]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T10:43:26.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:26 smithi103 ceph-mon[35160]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:26.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:26 smithi103 ceph-mon[35160]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T10:43:26.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:26 smithi133 ceph-mon[35282]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:26.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:26 smithi133 ceph-mon[35282]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T10:43:26.810 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:39:37.104114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:43:26.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T10:43:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:27.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:27 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3700357171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:27.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:27 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3700357171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:27.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:27 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3700357171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:43:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.505+0000 7f8734bd1640 1 -- 172.21.15.7:0/3177656170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8730106a80 msgr2=0x7f8730106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:27.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.505+0000 7f8734bd1640 1 --2- 172.21.15.7:0/3177656170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8730106a80 0x7f8730106e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8720009930 tx=0x7f872002a420 comp rx=0 tx=0).stop 2024-09-10T10:43:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.505+0000 7f8734bd1640 1 -- 172.21.15.7:0/3177656170 shutdown_connections 2024-09-10T10:43:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.505+0000 7f8734bd1640 1 --2- 172.21.15.7:0/3177656170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8730101420 0x7f873010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.505+0000 7f8734bd1640 1 --2- 172.21.15.7:0/3177656170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8730100a80 0x7f8730100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.505+0000 7f8734bd1640 1 --2- 172.21.15.7:0/3177656170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8730106a80 0x7f8730106e60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.505+0000 7f8734bd1640 1 -- 172.21.15.7:0/3177656170 >> 172.21.15.7:0/3177656170 conn(0x7f87300fc7a0 msgr2=0x7f87300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.505+0000 7f8734bd1640 1 -- 172.21.15.7:0/3177656170 shutdown_connections 2024-09-10T10:43:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.505+0000 7f8734bd1640 1 -- 172.21.15.7:0/3177656170 wait complete. 2024-09-10T10:43:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8734bd1640 1 Processor -- start 2024-09-10T10:43:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8734bd1640 1 -- start start 2024-09-10T10:43:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8734bd1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8730100a80 0x7f8730102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8734bd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8730101420 0x7f8730102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8734bd1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8730106a80 0x7f87301030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8734bd1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8730110ed0 con 0x7f8730101420 2024-09-10T10:43:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8734bd1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8730110d50 con 0x7f8730106a80 2024-09-10T10:43:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8734bd1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8730111050 con 0x7f8730100a80 2024-09-10T10:43:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8725d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8730101420 0x7f8730102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8725d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8730101420 0x7f8730102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46760/0 (socket says 172.21.15.7:46760) 2024-09-10T10:43:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f8725d9b640 1 -- 172.21.15.7:0/1455764217 learned_addr learned my addr 172.21.15.7:0/1455764217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.506+0000 7f872e59c640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8730100a80 0x7f8730102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f872ed9d640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8730106a80 0x7f87301030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8725d9b640 1 -- 172.21.15.7:0/1455764217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8730100a80 msgr2=0x7f8730102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:27.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8725d9b640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8730100a80 0x7f8730102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:27.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8725d9b640 1 -- 172.21.15.7:0/1455764217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8730106a80 msgr2=0x7f87301030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:27.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8725d9b640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8730106a80 0x7f87301030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:27.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8725d9b640 1 -- 172.21.15.7:0/1455764217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87301a6950 con 0x7f8730101420 2024-09-10T10:43:27.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f872ed9d640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8730106a80 0x7f87301030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:27.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8725d9b640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8730101420 0x7f8730102b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f871800d950 tx=0x7f871800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8727fff640 1 -- 172.21.15.7:0/1455764217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718017070 con 0x7f8730101420 2024-09-10T10:43:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8727fff640 1 -- 172.21.15.7:0/1455764217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8718009c30 con 0x7f8730101420 2024-09-10T10:43:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8734bd1640 1 -- 172.21.15.7:0/1455764217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87301a6c40 con 0x7f8730101420 2024-09-10T10:43:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8727fff640 1 -- 172.21.15.7:0/1455764217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871800f6b0 con 0x7f8730101420 2024-09-10T10:43:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.507+0000 7f8734bd1640 1 -- 172.21.15.7:0/1455764217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87301a7180 con 0x7f8730101420 2024-09-10T10:43:27.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.508+0000 7f8734bd1640 1 -- 172.21.15.7:0/1455764217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8730100ee0 con 0x7f8730101420 2024-09-10T10:43:27.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.510+0000 7f8727fff640 1 -- 172.21.15.7:0/1455764217 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f87180109d0 con 0x7f8730101420 2024-09-10T10:43:27.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.511+0000 7f8727fff640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f871c0a2b80 0x7f871c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:27.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.511+0000 7f872e59c640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f871c0a2b80 0x7f871c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:27.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.511+0000 7f8727fff640 1 -- 172.21.15.7:0/1455764217 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f87180ce7c0 con 0x7f8730101420 2024-09-10T10:43:27.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.511+0000 7f872e59c640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f871c0a2b80 0x7f871c0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8720006010 tx=0x7f8720000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:27.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.514+0000 7f8727fff640 1 -- 172.21.15.7:0/1455764217 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8718009de0 con 0x7f8730101420 2024-09-10T10:43:27.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.689+0000 7f8734bd1640 1 -- 172.21.15.7:0/1455764217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87301a73e0 con 0x7f8730101420 2024-09-10T10:43:27.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.689+0000 7f8727fff640 1 -- 172.21.15.7:0/1455764217 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f871809bbc0 con 0x7f8730101420 2024-09-10T10:43:27.690 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-10T10:43:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 -- 172.21.15.7:0/1455764217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f871c0a2b80 msgr2=0x7f871c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f871c0a2b80 0x7f871c0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8720006010 tx=0x7f8720000bc0 comp rx=0 tx=0).stop 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 -- 172.21.15.7:0/1455764217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8730101420 msgr2=0x7f8730102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8730101420 0x7f8730102b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f871800d950 tx=0x7f871800de10 comp rx=0 tx=0).stop 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 -- 172.21.15.7:0/1455764217 shutdown_connections 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f871c0a2b80 0x7f871c0a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8730106a80 0x7f87301030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8730101420 0x7f8730102b90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 --2- 172.21.15.7:0/1455764217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8730100a80 0x7f8730102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 -- 172.21.15.7:0/1455764217 >> 172.21.15.7:0/1455764217 conn(0x7f87300fc7a0 msgr2=0x7f873006a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 -- 172.21.15.7:0/1455764217 shutdown_connections 2024-09-10T10:43:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:27.691+0000 7f8734bd1640 1 -- 172.21.15.7:0/1455764217 wait complete. 2024-09-10T10:43:28.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:28 smithi103 ceph-mon[35160]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:28.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:28 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1455764217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:43:28.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:28 smithi007 ceph-mon[30873]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:28.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:28 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1455764217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:43:28.461 INFO:tasks.cephadm:Adding mds.a on smithi007 2024-09-10T10:43:28.461 INFO:tasks.cephadm:Adding mds.d on smithi007 2024-09-10T10:43:28.461 INFO:tasks.cephadm:Adding mds.g on smithi007 2024-09-10T10:43:28.461 INFO:tasks.cephadm:Adding mds.j on smithi007 2024-09-10T10:43:28.462 INFO:tasks.cephadm:Adding mds.b on smithi103 2024-09-10T10:43:28.462 INFO:tasks.cephadm:Adding mds.e on smithi103 2024-09-10T10:43:28.462 INFO:tasks.cephadm:Adding mds.h on smithi103 2024-09-10T10:43:28.462 INFO:tasks.cephadm:Adding mds.k on smithi103 2024-09-10T10:43:28.462 INFO:tasks.cephadm:Adding mds.c on smithi133 2024-09-10T10:43:28.462 INFO:tasks.cephadm:Adding mds.f on smithi133 2024-09-10T10:43:28.462 INFO:tasks.cephadm:Adding mds.i on smithi133 2024-09-10T10:43:28.462 INFO:tasks.cephadm:Adding mds.l on smithi133 2024-09-10T10:43:28.462 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l' 2024-09-10T10:43:28.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:28 smithi133 ceph-mon[35282]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:28.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:28 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1455764217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:43:28.722 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:43:29.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 -- 172.21.15.133:0/2408387377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f501037c0 msgr2=0x7f2f5010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 --2- 172.21.15.133:0/2408387377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f501037c0 0x7f2f5010a050 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009900 tx=0x7f2f4002a410 comp rx=0 tx=0).stop 2024-09-10T10:43:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 -- 172.21.15.133:0/2408387377 shutdown_connections 2024-09-10T10:43:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 --2- 172.21.15.133:0/2408387377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f501037c0 0x7f2f5010a050 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 --2- 172.21.15.133:0/2408387377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f50102e00 0x7f2f50103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 --2- 172.21.15.133:0/2408387377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f50101c00 0x7f2f50102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 -- 172.21.15.133:0/2408387377 >> 172.21.15.133:0/2408387377 conn(0x7f2f500fd390 msgr2=0x7f2f500ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 -- 172.21.15.133:0/2408387377 shutdown_connections 2024-09-10T10:43:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 -- 172.21.15.133:0/2408387377 wait complete. 2024-09-10T10:43:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 Processor -- start 2024-09-10T10:43:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.202+0000 7f2f553e4640 1 -- start start 2024-09-10T10:43:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f553e4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f50101c00 0x7f2f50198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f553e4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f50102e00 0x7f2f50198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f553e4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f501037c0 0x7f2f5019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f553e4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5010d980 con 0x7f2f50101c00 2024-09-10T10:43:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f553e4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5010d800 con 0x7f2f50102e00 2024-09-10T10:43:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f553e4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5010db00 con 0x7f2f501037c0 2024-09-10T10:43:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f4f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f501037c0 0x7f2f5019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f4f7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f501037c0 0x7f2f5019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48750/0 (socket says 172.21.15.133:48750) 2024-09-10T10:43:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f4f7fe640 1 -- 172.21.15.133:0/3558117155 learned_addr learned my addr 172.21.15.133:0/3558117155 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T10:43:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f4effd640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f50101c00 0x7f2f50198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f4f7fe640 1 -- 172.21.15.133:0/3558117155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f50102e00 msgr2=0x7f2f50198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f4f7fe640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f50102e00 0x7f2f50198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:29.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f4f7fe640 1 -- 172.21.15.133:0/3558117155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f50101c00 msgr2=0x7f2f50198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:29.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f4f7fe640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f50101c00 0x7f2f50198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:29.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f4f7fe640 1 -- 172.21.15.133:0/3558117155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5019fc70 con 0x7f2f501037c0 2024-09-10T10:43:29.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f4f7fe640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f501037c0 0x7f2f5019f600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2f400314e0 tx=0x7f2f400043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:29.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f37fff640 1 -- 172.21.15.133:0/3558117155 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4003f030 con 0x7f2f501037c0 2024-09-10T10:43:29.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f553e4640 1 -- 172.21.15.133:0/3558117155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5019ff00 con 0x7f2f501037c0 2024-09-10T10:43:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f37fff640 1 -- 172.21.15.133:0/3558117155 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f40003ce0 con 0x7f2f501037c0 2024-09-10T10:43:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f37fff640 1 -- 172.21.15.133:0/3558117155 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40035070 con 0x7f2f501037c0 2024-09-10T10:43:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.203+0000 7f2f553e4640 1 -- 172.21.15.133:0/3558117155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f501a0430 con 0x7f2f501037c0 2024-09-10T10:43:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.205+0000 7f2f553e4640 1 -- 172.21.15.133:0/3558117155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f50109050 con 0x7f2f501037c0 2024-09-10T10:43:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.206+0000 7f2f37fff640 1 -- 172.21.15.133:0/3558117155 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2f40042030 con 0x7f2f501037c0 2024-09-10T10:43:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.206+0000 7f2f37fff640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2f380a2bb0 0x7f2f380a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.206+0000 7f2f37fff640 1 -- 172.21.15.133:0/3558117155 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f2f400ef6c0 con 0x7f2f501037c0 2024-09-10T10:43:29.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.207+0000 7f2f4effd640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2f380a2bb0 0x7f2f380a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:29.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.207+0000 7f2f4effd640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2f380a2bb0 0x7f2f380a5070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f50102c60 tx=0x7f2f48009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:29.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.210+0000 7f2f37fff640 1 -- 172.21.15.133:0/3558117155 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f400bcaa0 con 0x7f2f501037c0 2024-09-10T10:43:29.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.383+0000 7f2f553e4640 1 -- 172.21.15.133:0/3558117155 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}) -- 0x7f2f500627f0 con 0x7f2f380a2bb0 2024-09-10T10:43:29.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.392+0000 7f2f37fff640 1 -- 172.21.15.133:0/3558117155 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f2f500627f0 con 0x7f2f380a2bb0 2024-09-10T10:43:29.393 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mds.all update... 2024-09-10T10:43:29.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 -- 172.21.15.133:0/3558117155 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2f380a2bb0 msgr2=0x7f2f380a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:29.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2f380a2bb0 0x7f2f380a5070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f50102c60 tx=0x7f2f48009110 comp rx=0 tx=0).stop 2024-09-10T10:43:29.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 -- 172.21.15.133:0/3558117155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f501037c0 msgr2=0x7f2f5019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:29.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f501037c0 0x7f2f5019f600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2f400314e0 tx=0x7f2f400043c0 comp rx=0 tx=0).stop 2024-09-10T10:43:29.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 -- 172.21.15.133:0/3558117155 shutdown_connections 2024-09-10T10:43:29.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2f380a2bb0 0x7f2f380a5070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:29.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f501037c0 0x7f2f5019f600 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:29.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f50102e00 0x7f2f50198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:29.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 --2- 172.21.15.133:0/3558117155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f50101c00 0x7f2f50198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:29.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 -- 172.21.15.133:0/3558117155 >> 172.21.15.133:0/3558117155 conn(0x7f2f500fd390 msgr2=0x7f2f50106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:29.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 -- 172.21.15.133:0/3558117155 shutdown_connections 2024-09-10T10:43:29.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T10:43:29.394+0000 7f2f553e4640 1 -- 172.21.15.133:0/3558117155 wait complete. 2024-09-10T10:43:30.032 DEBUG:teuthology.orchestra.run.smithi007:mds.a> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T10:43:30.036 DEBUG:teuthology.orchestra.run.smithi007:mds.d> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T10:43:30.079 DEBUG:teuthology.orchestra.run.smithi007:mds.g> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T10:43:30.122 DEBUG:teuthology.orchestra.run.smithi007:mds.j> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T10:43:30.165 DEBUG:teuthology.orchestra.run.smithi103:mds.b> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T10:43:30.168 DEBUG:teuthology.orchestra.run.smithi103:mds.e> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T10:43:30.212 DEBUG:teuthology.orchestra.run.smithi103:mds.h> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T10:43:30.254 DEBUG:teuthology.orchestra.run.smithi103:mds.k> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T10:43:30.298 DEBUG:teuthology.orchestra.run.smithi133:mds.c> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T10:43:30.300 DEBUG:teuthology.orchestra.run.smithi133:mds.f> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T10:43:30.343 DEBUG:teuthology.orchestra.run.smithi133:mds.i> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T10:43:30.386 DEBUG:teuthology.orchestra.run.smithi133:mds.l> sudo journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T10:43:30.429 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T10:43:30.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:30.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:30.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:30.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-09-10T10:43:30.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:30.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:30.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:30.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:30.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:30.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:30.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:30.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:30 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:30.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:30.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:30.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-09-10T10:43:30.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:30.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:30.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:30.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:30.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:30.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:30.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:30.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:30 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:30.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:43:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-09-10T10:43:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:43:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:43:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:30.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:30 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.119+0000 7fec7585d640 1 -- 172.21.15.7:0/3431371859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec70106f20 msgr2=0x7fec7010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.119+0000 7fec7585d640 1 --2- 172.21.15.7:0/3431371859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec70106f20 0x7fec7010eaa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec68009ca0 tx=0x7fec6802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:43:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.119+0000 7fec7585d640 1 -- 172.21.15.7:0/3431371859 shutdown_connections 2024-09-10T10:43:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.119+0000 7fec7585d640 1 --2- 172.21.15.7:0/3431371859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec70106f20 0x7fec7010eaa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.119+0000 7fec7585d640 1 --2- 172.21.15.7:0/3431371859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec701027a0 0x7fec701067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.119+0000 7fec7585d640 1 --2- 172.21.15.7:0/3431371859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec70101df0 0x7fec701021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.119+0000 7fec7585d640 1 -- 172.21.15.7:0/3431371859 >> 172.21.15.7:0/3431371859 conn(0x7fec700fd8b0 msgr2=0x7fec700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.119+0000 7fec7585d640 1 -- 172.21.15.7:0/3431371859 shutdown_connections 2024-09-10T10:43:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.119+0000 7fec7585d640 1 -- 172.21.15.7:0/3431371859 wait complete. 2024-09-10T10:43:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec7585d640 1 Processor -- start 2024-09-10T10:43:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec7585d640 1 -- start start 2024-09-10T10:43:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec7585d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec70101df0 0x7fec701984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec7585d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec701027a0 0x7fec701989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec7585d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec70106f20 0x7fec7019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec7585d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec70111fd0 con 0x7fec70106f20 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec7585d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec70111e50 con 0x7fec70101df0 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec7585d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fec70112150 con 0x7fec701027a0 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec6f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec70106f20 0x7fec7019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec6f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec70106f20 0x7fec7019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55340/0 (socket says 172.21.15.7:55340) 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec6f7fe640 1 -- 172.21.15.7:0/1721053251 learned_addr learned my addr 172.21.15.7:0/1721053251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.120+0000 7fec6effd640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec70101df0 0x7fec701984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec6e7fc640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec701027a0 0x7fec701989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec6f7fe640 1 -- 172.21.15.7:0/1721053251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec701027a0 msgr2=0x7fec701989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec6f7fe640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec701027a0 0x7fec701989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec6f7fe640 1 -- 172.21.15.7:0/1721053251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec70101df0 msgr2=0x7fec701984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec6f7fe640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec70101df0 0x7fec701984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec6f7fe640 1 -- 172.21.15.7:0/1721053251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7019d460 con 0x7fec70106f20 2024-09-10T10:43:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec6f7fe640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec70106f20 0x7fec7019cd80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fec68003f10 tx=0x7fec68003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec7485b640 1 -- 172.21.15.7:0/1721053251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68031710 con 0x7fec70106f20 2024-09-10T10:43:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec7485b640 1 -- 172.21.15.7:0/1721053251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec680318b0 con 0x7fec70106f20 2024-09-10T10:43:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec7585d640 1 -- 172.21.15.7:0/1721053251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7019d6f0 con 0x7fec70106f20 2024-09-10T10:43:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec7485b640 1 -- 172.21.15.7:0/1721053251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6802fb50 con 0x7fec70106f20 2024-09-10T10:43:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.121+0000 7fec7585d640 1 -- 172.21.15.7:0/1721053251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec701a5030 con 0x7fec70106f20 2024-09-10T10:43:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.122+0000 7fec7585d640 1 -- 172.21.15.7:0/1721053251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec701021d0 con 0x7fec70106f20 2024-09-10T10:43:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.123+0000 7fec7485b640 1 -- 172.21.15.7:0/1721053251 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fec6802fcf0 con 0x7fec70106f20 2024-09-10T10:43:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.124+0000 7fec7485b640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fec580a2b80 0x7fec580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.124+0000 7fec6effd640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fec580a2b80 0x7fec580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.124+0000 7fec7485b640 1 -- 172.21.15.7:0/1721053251 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fec680f03a0 con 0x7fec70106f20 2024-09-10T10:43:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.125+0000 7fec6effd640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fec580a2b80 0x7fec580a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec6000bbe0 tx=0x7fec6000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.127+0000 7fec7485b640 1 -- 172.21.15.7:0/1721053251 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec680bd700 con 0x7fec70106f20 2024-09-10T10:43:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.339+0000 7fec7585d640 1 -- 172.21.15.7:0/1721053251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec701a5200 con 0x7fec70106f20 2024-09-10T10:43:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.339+0000 7fec7485b640 1 -- 172.21.15.7:0/1721053251 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fec680dd0c0 con 0x7fec70106f20 2024-09-10T10:43:31.341 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:31.341 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":1,"btime":"2024-09-10T10:36:33:117248+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T10:43:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 1 2024-09-10T10:43:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 -- 172.21.15.7:0/1721053251 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fec580a2b80 msgr2=0x7fec580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fec580a2b80 0x7fec580a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec6000bbe0 tx=0x7fec6000b390 comp rx=0 tx=0).stop 2024-09-10T10:43:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 -- 172.21.15.7:0/1721053251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec70106f20 msgr2=0x7fec7019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec70106f20 0x7fec7019cd80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fec68003f10 tx=0x7fec68003f40 comp rx=0 tx=0).stop 2024-09-10T10:43:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 -- 172.21.15.7:0/1721053251 shutdown_connections 2024-09-10T10:43:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fec580a2b80 0x7fec580a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec70106f20 0x7fec7019cd80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec701027a0 0x7fec701989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 --2- 172.21.15.7:0/1721053251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec70101df0 0x7fec701984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 -- 172.21.15.7:0/1721053251 >> 172.21.15.7:0/1721053251 conn(0x7fec700fd8b0 msgr2=0x7fec700ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 -- 172.21.15.7:0/1721053251 shutdown_connections 2024-09-10T10:43:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:31.342+0000 7fec7585d640 1 -- 172.21.15.7:0/1721053251 wait complete. 2024-09-10T10:43:31.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:31 smithi103 ceph-mon[35160]: Deploying daemon mds.l on smithi133 2024-09-10T10:43:31.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:31 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1721053251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:31.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:31 smithi007 ceph-mon[30873]: Deploying daemon mds.l on smithi133 2024-09-10T10:43:31.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:31 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1721053251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:31.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:31 smithi133 ceph-mon[35282]: Deploying daemon mds.l on smithi133 2024-09-10T10:43:31.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:31 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1721053251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:31.726 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 10:43:31 smithi133 systemd[1]: Starting Ceph mds.l for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:43:32.030 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 10:43:31 smithi133 podman[65149]: 2024-09-10 10:43:31.679300538 +0000 UTC m=+0.016367139 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:43:32.031 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 10:43:31 smithi133 podman[65149]: 2024-09-10 10:43:31.782731516 +0000 UTC m=+0.119798110 container create 185a2aa8bf29858bc3e92a2e5e68e4fa44c15a301e2a08c1bc40e6ab92f28869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T10:43:32.031 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 10:43:32 smithi133 podman[65149]: 2024-09-10 10:43:32.026829567 +0000 UTC m=+0.363896170 container init 185a2aa8bf29858bc3e92a2e5e68e4fa44c15a301e2a08c1bc40e6ab92f28869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:43:32.031 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 10:43:32 smithi133 podman[65149]: 2024-09-10 10:43:32.029942666 +0000 UTC m=+0.367009268 container start 185a2aa8bf29858bc3e92a2e5e68e4fa44c15a301e2a08c1bc40e6ab92f28869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:43:32.398 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mds[65168]: -- 172.21.15.133:0/554755095 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55722c687860 con 0x55722c665400 2024-09-10T10:43:32.399 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l[65164]: starting mds.l at 2024-09-10T10:43:32.399 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 10:43:32 smithi133 bash[65149]: 185a2aa8bf29858bc3e92a2e5e68e4fa44c15a301e2a08c1bc40e6ab92f28869 2024-09-10T10:43:32.399 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 10:43:32 smithi133 systemd[1]: Started Ceph mds.l for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:32.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:32 smithi007 ceph-mon[30873]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:32.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:32 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6832/691908645,v1:172.21.15.133:6833/691908645] up:boot 2024-09-10T10:43:32.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:32 smithi007 ceph-mon[30873]: fsmap 1 up:standby 2024-09-10T10:43:32.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:43:32.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:32.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:32.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:32.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:32.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:32.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:32 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:32.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:32 smithi103 ceph-mon[35160]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:32.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:32 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6832/691908645,v1:172.21.15.133:6833/691908645] up:boot 2024-09-10T10:43:32.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:32 smithi103 ceph-mon[35160]: fsmap 1 up:standby 2024-09-10T10:43:32.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:43:32.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:32.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:32.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:32.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:32.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:32.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:32 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:32.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mon[35282]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6832/691908645,v1:172.21.15.133:6833/691908645] up:boot 2024-09-10T10:43:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mon[35282]: fsmap 1 up:standby 2024-09-10T10:43:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:43:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:32.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:32 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:33.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:33 smithi103 ceph-mon[35160]: Deploying daemon mds.k on smithi103 2024-09-10T10:43:33.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:33 smithi007 ceph-mon[30873]: Deploying daemon mds.k on smithi103 2024-09-10T10:43:33.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:33 smithi133 ceph-mon[35282]: Deploying daemon mds.k on smithi103 2024-09-10T10:43:34.214 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:34.357 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 10:43:34 smithi103 systemd[1]: Starting Ceph mds.k for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:43:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:34.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:34 smithi007 ceph-mon[30873]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:34 smithi103 ceph-mon[35160]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:34.627 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 10:43:34 smithi103 podman[64977]: 2024-09-10 10:43:34.356071823 +0000 UTC m=+0.106728298 container create 17da4644861975fe7e3146e9df7080c77f4e4446c72dd8c6732d9125d2ff7f43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS) 2024-09-10T10:43:34.627 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 10:43:34 smithi103 podman[64977]: 2024-09-10 10:43:34.265052026 +0000 UTC m=+0.015708513 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:43:34.627 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 10:43:34 smithi103 podman[64977]: 2024-09-10 10:43:34.600281273 +0000 UTC m=+0.350937748 container init 17da4644861975fe7e3146e9df7080c77f4e4446c72dd8c6732d9125d2ff7f43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T10:43:34.627 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 10:43:34 smithi103 podman[64977]: 2024-09-10 10:43:34.602999384 +0000 UTC m=+0.353655858 container start 17da4644861975fe7e3146e9df7080c77f4e4446c72dd8c6732d9125d2ff7f43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T10:43:34.628 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 10:43:34 smithi103 ceph-mds[64996]: -- 172.21.15.103:0/324811767 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2a3bb9860 con 0x55d2a3b97000 2024-09-10T10:43:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:34 smithi133 ceph-mon[35282]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:34.922 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 10:43:34 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-k[64992]: starting mds.k at 2024-09-10T10:43:34.922 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 10:43:34 smithi103 bash[64977]: 17da4644861975fe7e3146e9df7080c77f4e4446c72dd8c6732d9125d2ff7f43 2024-09-10T10:43:34.922 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 10:43:34 smithi103 systemd[1]: Started Ceph mds.k for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:34.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.922+0000 7f119c570640 1 -- 172.21.15.7:0/699373513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1194102d30 msgr2=0x7f1194103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:34.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.922+0000 7f119c570640 1 --2- 172.21.15.7:0/699373513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1194102d30 0x7f1194103110 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1190009930 tx=0x7f119002a420 comp rx=0 tx=0).stop 2024-09-10T10:43:34.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.922+0000 7f119c570640 1 -- 172.21.15.7:0/699373513 shutdown_connections 2024-09-10T10:43:34.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.922+0000 7f119c570640 1 --2- 172.21.15.7:0/699373513 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11941082a0 0x7f119410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.922+0000 7f119c570640 1 --2- 172.21.15.7:0/699373513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11941036e0 0x7f1194107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.922+0000 7f119c570640 1 --2- 172.21.15.7:0/699373513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1194102d30 0x7f1194103110 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.922+0000 7f119c570640 1 -- 172.21.15.7:0/699373513 >> 172.21.15.7:0/699373513 conn(0x7f11940fc7d0 msgr2=0x7f11940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:34.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.922+0000 7f119c570640 1 -- 172.21.15.7:0/699373513 shutdown_connections 2024-09-10T10:43:34.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.922+0000 7f119c570640 1 -- 172.21.15.7:0/699373513 wait complete. 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.923+0000 7f119c570640 1 Processor -- start 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.923+0000 7f119c570640 1 -- start start 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.923+0000 7f119c570640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1194102d30 0x7f11941985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119c570640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11941036e0 0x7f1194198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119c570640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11941082a0 0x7f119419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119a30c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1194102d30 0x7f11941985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119a30c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1194102d30 0x7f11941985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55364/0 (socket says 172.21.15.7:55364) 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119a30c640 1 -- 172.21.15.7:0/4255265690 learned_addr learned my addr 172.21.15.7:0/4255265690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f11941120a0 con 0x7f1194102d30 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1194111f20 con 0x7f11941036e0 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1194112220 con 0x7f11941082a0 2024-09-10T10:43:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f1199b0b640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11941036e0 0x7f1194198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119ab0d640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11941082a0 0x7f119419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119a30c640 1 -- 172.21.15.7:0/4255265690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11941082a0 msgr2=0x7f119419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119a30c640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11941082a0 0x7f119419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119a30c640 1 -- 172.21.15.7:0/4255265690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11941036e0 msgr2=0x7f1194198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119a30c640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11941036e0 0x7f1194198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119a30c640 1 -- 172.21.15.7:0/4255265690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119419d570 con 0x7f1194102d30 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f1199b0b640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11941036e0 0x7f1194198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f119a30c640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1194102d30 0x7f11941985c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1190009230 tx=0x7f1190004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.924+0000 7f118b7fe640 1 -- 172.21.15.7:0/4255265690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119003f030 con 0x7f1194102d30 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.925+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119419d800 con 0x7f1194102d30 2024-09-10T10:43:34.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.925+0000 7f118b7fe640 1 -- 172.21.15.7:0/4255265690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1190003ad0 con 0x7f1194102d30 2024-09-10T10:43:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.925+0000 7f118b7fe640 1 -- 172.21.15.7:0/4255265690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190035070 con 0x7f1194102d30 2024-09-10T10:43:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.925+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11941a5140 con 0x7f1194102d30 2024-09-10T10:43:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.926+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1194103110 con 0x7f1194102d30 2024-09-10T10:43:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.927+0000 7f118b7fe640 1 -- 172.21.15.7:0/4255265690 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1190042030 con 0x7f1194102d30 2024-09-10T10:43:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.928+0000 7f118b7fe640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f11800a2b80 0x7f11800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.928+0000 7f1199b0b640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f11800a2b80 0x7f11800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.928+0000 7f118b7fe640 1 -- 172.21.15.7:0/4255265690 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f11900ef4d0 con 0x7f1194102d30 2024-09-10T10:43:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.928+0000 7f1199b0b640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f11800a2b80 0x7f11800a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f11940fee60 tx=0x7f118400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:34.935+0000 7f118b7fe640 1 -- 172.21.15.7:0/4255265690 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11900bc8d0 con 0x7f1194102d30 2024-09-10T10:43:35.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.173+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1194199920 con 0x7f1194102d30 2024-09-10T10:43:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.174+0000 7f118b7fe640 1 -- 172.21.15.7:0/4255265690 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f11900bc1d0 con 0x7f1194102d30 2024-09-10T10:43:35.175 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:35.175 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"btime":"2024-09-10T10:43:32:058616+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T10:43:35.176 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-09-10T10:43:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.176+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f11800a2b80 msgr2=0x7f11800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.176+0000 7f119c570640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f11800a2b80 0x7f11800a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f11940fee60 tx=0x7f118400b410 comp rx=0 tx=0).stop 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.176+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1194102d30 msgr2=0x7f11941985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.176+0000 7f119c570640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1194102d30 0x7f11941985c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1190009230 tx=0x7f1190004270 comp rx=0 tx=0).stop 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.176+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 shutdown_connections 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.176+0000 7f119c570640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f11800a2b80 0x7f11800a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.176+0000 7f119c570640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11941082a0 0x7f119419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.176+0000 7f119c570640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11941036e0 0x7f1194198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.176+0000 7f119c570640 1 --2- 172.21.15.7:0/4255265690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1194102d30 0x7f11941985c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.176+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 >> 172.21.15.7:0/4255265690 conn(0x7f11940fc7d0 msgr2=0x7f1194105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.177+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 shutdown_connections 2024-09-10T10:43:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:35.177+0000 7f119c570640 1 -- 172.21.15.7:0/4255265690 wait complete. 2024-09-10T10:43:36.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:36.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:36.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:36.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:36.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:36.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:36.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:36.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: Deploying daemon mds.j on smithi007 2024-09-10T10:43:36.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4255265690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:36.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.103:6832/2889573301,v1:172.21.15.103:6833/2889573301] up:boot 2024-09-10T10:43:36.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: fsmap 2 up:standby 2024-09-10T10:43:36.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:35 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:43:36.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:36.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:36.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:36.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:36.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:36.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:36.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:36.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: Deploying daemon mds.j on smithi007 2024-09-10T10:43:36.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4255265690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:36.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.103:6832/2889573301,v1:172.21.15.103:6833/2889573301] up:boot 2024-09-10T10:43:36.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: fsmap 2 up:standby 2024-09-10T10:43:36.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:35 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:43:36.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:36.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:36.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:36.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:36.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:36.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:36.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:36.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: Deploying daemon mds.j on smithi007 2024-09-10T10:43:36.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4255265690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:36.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.103:6832/2889573301,v1:172.21.15.103:6833/2889573301] up:boot 2024-09-10T10:43:36.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: fsmap 2 up:standby 2024-09-10T10:43:36.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:35 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:43:37.662 INFO:journalctl@ceph.mds.j.smithi007.stdout:Sep 10 10:43:37 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-j[70999]: starting mds.j at 2024-09-10T10:43:37.934 INFO:journalctl@ceph.mds.j.smithi007.stdout:Sep 10 10:43:37 smithi007 bash[70984]: 5f3188481593127170025eed3e235830114db0e0048f6ecdd98fa87e7c02b049 2024-09-10T10:43:37.934 INFO:journalctl@ceph.mds.j.smithi007.stdout:Sep 10 10:43:37 smithi007 systemd[1]: Started Ceph mds.j for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:37 smithi007 ceph-mon[30873]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:37.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:37.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:37 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:37 smithi103 ceph-mon[35160]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:37 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:38.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:37 smithi133 ceph-mon[35282]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:38.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:38.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:38.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:38.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:38.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:38.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:37 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:39.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:39.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:38 smithi103 ceph-mon[35160]: Deploying daemon mds.d on smithi007 2024-09-10T10:43:39.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:38 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.7:6834/3661855188,v1:172.21.15.7:6835/3661855188] up:boot 2024-09-10T10:43:39.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:38 smithi103 ceph-mon[35160]: fsmap 3 up:standby 2024-09-10T10:43:39.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:38 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:43:39.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:38 smithi007 ceph-mon[30873]: Deploying daemon mds.d on smithi007 2024-09-10T10:43:39.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:38 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.7:6834/3661855188,v1:172.21.15.7:6835/3661855188] up:boot 2024-09-10T10:43:39.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:38 smithi007 ceph-mon[30873]: fsmap 3 up:standby 2024-09-10T10:43:39.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:38 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:43:39.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:38 smithi133 ceph-mon[35282]: Deploying daemon mds.d on smithi007 2024-09-10T10:43:39.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:38 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.7:6834/3661855188,v1:172.21.15.7:6835/3661855188] up:boot 2024-09-10T10:43:39.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:38 smithi133 ceph-mon[35282]: fsmap 3 up:standby 2024-09-10T10:43:39.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:38 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:43:39.458 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:39.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.884+0000 7f0601b31640 1 -- 172.21.15.7:0/4188067906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc102ef0 msgr2=0x7f05fc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:39.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.884+0000 7f0601b31640 1 --2- 172.21.15.7:0/4188067906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc102ef0 0x7f05fc10da40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009ca0 tx=0x7f05f402a950 comp rx=0 tx=0).stop 2024-09-10T10:43:39.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.884+0000 7f0601b31640 1 -- 172.21.15.7:0/4188067906 shutdown_connections 2024-09-10T10:43:39.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.884+0000 7f0601b31640 1 --2- 172.21.15.7:0/4188067906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc102ef0 0x7f05fc10da40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.884+0000 7f0601b31640 1 --2- 172.21.15.7:0/4188067906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05fc102570 0x7f05fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.884+0000 7f0601b31640 1 --2- 172.21.15.7:0/4188067906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05fc100e20 0x7f05fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.884+0000 7f0601b31640 1 -- 172.21.15.7:0/4188067906 >> 172.21.15.7:0/4188067906 conn(0x7f05fc0fc7d0 msgr2=0x7f05fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:39.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.884+0000 7f0601b31640 1 -- 172.21.15.7:0/4188067906 shutdown_connections 2024-09-10T10:43:39.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.884+0000 7f0601b31640 1 -- 172.21.15.7:0/4188067906 wait complete. 2024-09-10T10:43:39.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f0601b31640 1 Processor -- start 2024-09-10T10:43:39.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f0601b31640 1 -- start start 2024-09-10T10:43:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f0601b31640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05fc100e20 0x7f05fc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f0601b31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05fc102570 0x7f05fc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f0601b31640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc102ef0 0x7f05fc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f0601b31640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc111020 con 0x7f05fc102ef0 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f0601b31640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc110ea0 con 0x7f05fc100e20 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f0601b31640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc1111a0 con 0x7f05fc102570 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f05fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc102ef0 0x7f05fc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f05fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc102ef0 0x7f05fc0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55392/0 (socket says 172.21.15.7:55392) 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f05fbfff640 1 -- 172.21.15.7:0/3002446242 learned_addr learned my addr 172.21.15.7:0/3002446242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f05fb7fe640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05fc100e20 0x7f05fc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.885+0000 7f05f2ffd640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05fc102570 0x7f05fc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f05fbfff640 1 -- 172.21.15.7:0/3002446242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05fc102570 msgr2=0x7f05fc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f05fbfff640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05fc102570 0x7f05fc0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f05fbfff640 1 -- 172.21.15.7:0/3002446242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05fc100e20 msgr2=0x7f05fc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f05fbfff640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05fc100e20 0x7f05fc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f05fbfff640 1 -- 172.21.15.7:0/3002446242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc1a6aa0 con 0x7f05fc102ef0 2024-09-10T10:43:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f05fbfff640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc102ef0 0x7f05fc0fd700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f05f4006010 tx=0x7f05f402f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f05f97fa640 1 -- 172.21.15.7:0/3002446242 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4034040 con 0x7f05fc102ef0 2024-09-10T10:43:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f05f97fa640 1 -- 172.21.15.7:0/3002446242 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05f4003e00 con 0x7f05fc102ef0 2024-09-10T10:43:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f0601b31640 1 -- 172.21.15.7:0/3002446242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1a6d30 con 0x7f05fc102ef0 2024-09-10T10:43:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f05f97fa640 1 -- 172.21.15.7:0/3002446242 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4045480 con 0x7f05fc102ef0 2024-09-10T10:43:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.886+0000 7f0601b31640 1 -- 172.21.15.7:0/3002446242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1a7210 con 0x7f05fc102ef0 2024-09-10T10:43:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.887+0000 7f0601b31640 1 -- 172.21.15.7:0/3002446242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc101280 con 0x7f05fc102ef0 2024-09-10T10:43:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.888+0000 7f05f97fa640 1 -- 172.21.15.7:0/3002446242 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f05f4037020 con 0x7f05fc102ef0 2024-09-10T10:43:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.889+0000 7f05f97fa640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05e80a2b80 0x7f05e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.889+0000 7f05fb7fe640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05e80a2b80 0x7f05e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.889+0000 7f05f97fa640 1 -- 172.21.15.7:0/3002446242 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f05f40efcd0 con 0x7f05fc102ef0 2024-09-10T10:43:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.890+0000 7f05fb7fe640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05e80a2b80 0x7f05e80a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009990 tx=0x7f05ec009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:39.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:39.893+0000 7f05f97fa640 1 -- 172.21.15.7:0/3002446242 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05f40bd0d0 con 0x7f05fc102ef0 2024-09-10T10:43:40.087 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:39 smithi007 ceph-mon[30873]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:40.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.111+0000 7f0601b31640 1 -- 172.21.15.7:0/3002446242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05fc10a0a0 con 0x7f05fc102ef0 2024-09-10T10:43:40.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.111+0000 7f05f97fa640 1 -- 172.21.15.7:0/3002446242 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f05f40bc9d0 con 0x7f05fc102ef0 2024-09-10T10:43:40.113 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:40.113 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":4,"btime":"2024-09-10T10:43:37:783644+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T10:43:40.113 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 4 2024-09-10T10:43:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.113+0000 7f0601b31640 1 -- 172.21.15.7:0/3002446242 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05e80a2b80 msgr2=0x7f05e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.113+0000 7f0601b31640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05e80a2b80 0x7f05e80a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009990 tx=0x7f05ec009110 comp rx=0 tx=0).stop 2024-09-10T10:43:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.113+0000 7f0601b31640 1 -- 172.21.15.7:0/3002446242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc102ef0 msgr2=0x7f05fc0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.113+0000 7f0601b31640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc102ef0 0x7f05fc0fd700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f05f4006010 tx=0x7f05f402f6f0 comp rx=0 tx=0).stop 2024-09-10T10:43:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.114+0000 7f0601b31640 1 -- 172.21.15.7:0/3002446242 shutdown_connections 2024-09-10T10:43:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.114+0000 7f0601b31640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f05e80a2b80 0x7f05e80a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.114+0000 7f0601b31640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc102ef0 0x7f05fc0fd700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.114+0000 7f0601b31640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05fc102570 0x7f05fc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.114+0000 7f0601b31640 1 --2- 172.21.15.7:0/3002446242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05fc100e20 0x7f05fc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.114+0000 7f0601b31640 1 -- 172.21.15.7:0/3002446242 >> 172.21.15.7:0/3002446242 conn(0x7f05fc0fc7d0 msgr2=0x7f05fc06a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.114+0000 7f0601b31640 1 -- 172.21.15.7:0/3002446242 shutdown_connections 2024-09-10T10:43:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:40.114+0000 7f0601b31640 1 -- 172.21.15.7:0/3002446242 wait complete. 2024-09-10T10:43:40.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:39 smithi103 ceph-mon[35160]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:40.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:39 smithi133 ceph-mon[35282]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:40.804 INFO:journalctl@ceph.mds.d.smithi007.stdout:Sep 10 10:43:40 smithi007 systemd[1]: Started Ceph mds.d for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:41.162 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:40 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3002446242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:41.163 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:41.163 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:41.163 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:41.163 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:41.163 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:41.163 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:40 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:41.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:40 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3002446242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:41.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:41.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:41.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:41.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:41.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:41.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:40 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:41.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:40 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3002446242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:41.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:41.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:41.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:41.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:41.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:41.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:40 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:42.108 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:41 smithi103 ceph-mon[35160]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:42.108 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:41 smithi103 ceph-mon[35160]: Deploying daemon mds.e on smithi103 2024-09-10T10:43:42.109 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:41 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.7:6836/2814690837,v1:172.21.15.7:6837/2814690837] up:boot 2024-09-10T10:43:42.109 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:41 smithi103 ceph-mon[35160]: fsmap 4 up:standby 2024-09-10T10:43:42.109 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:41 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:43:42.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:41 smithi007 ceph-mon[30873]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:42.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:41 smithi007 ceph-mon[30873]: Deploying daemon mds.e on smithi103 2024-09-10T10:43:42.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:41 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.7:6836/2814690837,v1:172.21.15.7:6837/2814690837] up:boot 2024-09-10T10:43:42.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:41 smithi007 ceph-mon[30873]: fsmap 4 up:standby 2024-09-10T10:43:42.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:41 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:43:42.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:41 smithi133 ceph-mon[35282]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:42.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:41 smithi133 ceph-mon[35282]: Deploying daemon mds.e on smithi103 2024-09-10T10:43:42.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:41 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.7:6836/2814690837,v1:172.21.15.7:6837/2814690837] up:boot 2024-09-10T10:43:42.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:41 smithi133 ceph-mon[35282]: fsmap 4 up:standby 2024-09-10T10:43:42.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:41 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:43:42.672 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 10:43:42 smithi103 systemd[1]: Starting Ceph mds.e for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:43:43.045 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 10:43:42 smithi103 podman[65288]: 2024-09-10 10:43:42.77676858 +0000 UTC m=+0.107127376 container create 26f870833e690ef1b84f25ba7ffdde23952ee8e878ac4047c14eb63ca9953bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True) 2024-09-10T10:43:43.045 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 10:43:42 smithi103 podman[65288]: 2024-09-10 10:43:42.685953358 +0000 UTC m=+0.016312168 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:43:43.045 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 10:43:42 smithi103 podman[65288]: 2024-09-10 10:43:42.954519244 +0000 UTC m=+0.284878050 container init 26f870833e690ef1b84f25ba7ffdde23952ee8e878ac4047c14eb63ca9953bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:43:43.045 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 10:43:42 smithi103 podman[65288]: 2024-09-10 10:43:42.957397537 +0000 UTC m=+0.287756350 container start 26f870833e690ef1b84f25ba7ffdde23952ee8e878ac4047c14eb63ca9953bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T10:43:43.045 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 10:43:42 smithi103 ceph-mds[65307]: -- 172.21.15.103:0/1597464279 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b5f3c7860 con 0x560b5f3a4800 2024-09-10T10:43:43.046 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 10:43:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e[65303]: starting mds.e at 2024-09-10T10:43:43.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:43.422 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 10:43:43 smithi103 bash[65288]: 26f870833e690ef1b84f25ba7ffdde23952ee8e878ac4047c14eb63ca9953bc6 2024-09-10T10:43:43.422 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 10:43:43 smithi103 systemd[1]: Started Ceph mds.e for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 -- 172.21.15.7:0/3851741156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d01036e0 msgr2=0x7f71d0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 --2- 172.21.15.7:0/3851741156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d01036e0 0x7f71d0107b90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009930 tx=0x7f71c402a420 comp rx=0 tx=0).stop 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 -- 172.21.15.7:0/3851741156 shutdown_connections 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 --2- 172.21.15.7:0/3851741156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d01082a0 0x7f71d010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 --2- 172.21.15.7:0/3851741156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d01036e0 0x7f71d0107b90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 --2- 172.21.15.7:0/3851741156 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d0102d30 0x7f71d0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 -- 172.21.15.7:0/3851741156 >> 172.21.15.7:0/3851741156 conn(0x7f71d00fc7d0 msgr2=0x7f71d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 -- 172.21.15.7:0/3851741156 shutdown_connections 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 -- 172.21.15.7:0/3851741156 wait complete. 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 Processor -- start 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.971+0000 7f71d6e1a640 1 -- start start 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d6e1a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d0102d30 0x7f71d0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d6e1a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d01036e0 0x7f71d0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d6e1a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d01082a0 0x7f71d019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d6e1a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71d01120a0 con 0x7f71d0102d30 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d6e1a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0111f20 con 0x7f71d01036e0 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d6e1a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f71d0112220 con 0x7f71d01082a0 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d4bb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d0102d30 0x7f71d0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d4bb6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d0102d30 0x7f71d0198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44568/0 (socket says 172.21.15.7:44568) 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d4bb6640 1 -- 172.21.15.7:0/314407001 learned_addr learned my addr 172.21.15.7:0/314407001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d53b7640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d01082a0 0x7f71d019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d4bb6640 1 -- 172.21.15.7:0/314407001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d01082a0 msgr2=0x7f71d019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71cffff640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d01036e0 0x7f71d0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d4bb6640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d01082a0 0x7f71d019cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d4bb6640 1 -- 172.21.15.7:0/314407001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d01036e0 msgr2=0x7f71d0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d4bb6640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d01036e0 0x7f71d0198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d4bb6640 1 -- 172.21.15.7:0/314407001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d019d5f0 con 0x7f71d0102d30 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71cffff640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d01036e0 0x7f71d0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d4bb6640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d0102d30 0x7f71d0198640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71c000c890 tx=0x7f71c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71cdffb640 1 -- 172.21.15.7:0/314407001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0004220 con 0x7f71d0102d30 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71cdffb640 1 -- 172.21.15.7:0/314407001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c00043c0 con 0x7f71d0102d30 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71d6e1a640 1 -- 172.21.15.7:0/314407001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d019d8e0 con 0x7f71d0102d30 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.972+0000 7f71cdffb640 1 -- 172.21.15.7:0/314407001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00049e0 con 0x7f71d0102d30 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.973+0000 7f71d6e1a640 1 -- 172.21.15.7:0/314407001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d01a51c0 con 0x7f71d0102d30 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.974+0000 7f71d6e1a640 1 -- 172.21.15.7:0/314407001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d0103110 con 0x7f71d0102d30 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.975+0000 7f71cdffb640 1 -- 172.21.15.7:0/314407001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f71c00079b0 con 0x7f71d0102d30 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.975+0000 7f71cdffb640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f71b80a2c00 0x7f71b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.975+0000 7f71cffff640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f71b80a2c00 0x7f71b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.975+0000 7f71cdffb640 1 -- 172.21.15.7:0/314407001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f71c00cec70 con 0x7f71d0102d30 2024-09-10T10:43:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.976+0000 7f71cffff640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f71b80a2c00 0x7f71b80a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f71c402a940 tx=0x7f71c4032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:43.980+0000 7f71cdffb640 1 -- 172.21.15.7:0/314407001 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71c0014030 con 0x7f71d0102d30 2024-09-10T10:43:44.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.195+0000 7f71d6e1a640 1 -- 172.21.15.7:0/314407001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71d010a2b0 con 0x7f71d0102d30 2024-09-10T10:43:44.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.195+0000 7f71cdffb640 1 -- 172.21.15.7:0/314407001 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f71c009c070 con 0x7f71d0102d30 2024-09-10T10:43:44.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:44.197 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":6,"btime":"2024-09-10T10:43:43:806199+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T10:43:44.197 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 6 2024-09-10T10:43:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.197+0000 7f71d6e1a640 1 -- 172.21.15.7:0/314407001 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f71b80a2c00 msgr2=0x7f71b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.197+0000 7f71d6e1a640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f71b80a2c00 0x7f71b80a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f71c402a940 tx=0x7f71c4032c70 comp rx=0 tx=0).stop 2024-09-10T10:43:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.197+0000 7f71d6e1a640 1 -- 172.21.15.7:0/314407001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d0102d30 msgr2=0x7f71d0198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.197+0000 7f71d6e1a640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d0102d30 0x7f71d0198640 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f71c000c890 tx=0x7f71c000cd50 comp rx=0 tx=0).stop 2024-09-10T10:43:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.197+0000 7f71d6e1a640 1 -- 172.21.15.7:0/314407001 shutdown_connections 2024-09-10T10:43:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.197+0000 7f71d6e1a640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f71b80a2c00 0x7f71b80a50c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.197+0000 7f71d6e1a640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d01082a0 0x7f71d019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.197+0000 7f71d6e1a640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d01036e0 0x7f71d0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.197+0000 7f71d6e1a640 1 --2- 172.21.15.7:0/314407001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d0102d30 0x7f71d0198640 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.197+0000 7f71d6e1a640 1 -- 172.21.15.7:0/314407001 >> 172.21.15.7:0/314407001 conn(0x7f71d00fc7d0 msgr2=0x7f71d0106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:44.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.198+0000 7f71d6e1a640 1 -- 172.21.15.7:0/314407001 shutdown_connections 2024-09-10T10:43:44.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:44.198+0000 7f71d6e1a640 1 -- 172.21.15.7:0/314407001 wait complete. 2024-09-10T10:43:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: Deploying daemon mds.f on smithi133 2024-09-10T10:43:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.103:6834/3115622627,v1:172.21.15.103:6835/3115622627] up:boot 2024-09-10T10:43:44.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: fsmap 5 up:standby 2024-09-10T10:43:44.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:44 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:43:44.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:44.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:44.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:44.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:44.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: Deploying daemon mds.f on smithi133 2024-09-10T10:43:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.103:6834/3115622627,v1:172.21.15.103:6835/3115622627] up:boot 2024-09-10T10:43:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: fsmap 5 up:standby 2024-09-10T10:43:44.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:44 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:43:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:44.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:44.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: Deploying daemon mds.f on smithi133 2024-09-10T10:43:44.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.103:6834/3115622627,v1:172.21.15.103:6835/3115622627] up:boot 2024-09-10T10:43:44.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: fsmap 5 up:standby 2024-09-10T10:43:44.435 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:44 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:43:45.099 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:43:44 smithi133 systemd[1]: Starting Ceph mds.f for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:43:45.380 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:45 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/314407001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:45.380 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:43:45 smithi133 podman[65453]: 2024-09-10 10:43:45.069135548 +0000 UTC m=+0.016135090 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:43:45.380 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:43:45 smithi133 podman[65453]: 2024-09-10 10:43:45.16877707 +0000 UTC m=+0.115776651 container create 78ce81a2e7d0250238fd69ec042aab2d4aa46259fa6fd1a80815fa82e5682af4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T10:43:45.380 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:43:45 smithi133 podman[65453]: 2024-09-10 10:43:45.379429557 +0000 UTC m=+0.326429096 container init 78ce81a2e7d0250238fd69ec042aab2d4aa46259fa6fd1a80815fa82e5682af4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T10:43:45.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:45 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/314407001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:45.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:45 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/314407001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:45.725 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:43:45 smithi133 podman[65453]: 2024-09-10 10:43:45.38227742 +0000 UTC m=+0.329276959 container start 78ce81a2e7d0250238fd69ec042aab2d4aa46259fa6fd1a80815fa82e5682af4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:43:45.725 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:43:45 smithi133 ceph-mds[65473]: -- 172.21.15.133:0/1910597300 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a75561860 con 0x556a7553f000 2024-09-10T10:43:45.725 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:43:45 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: starting mds.f at 2024-09-10T10:43:45.726 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:43:45 smithi133 bash[65453]: 78ce81a2e7d0250238fd69ec042aab2d4aa46259fa6fd1a80815fa82e5682af4 2024-09-10T10:43:45.726 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:43:45 smithi133 systemd[1]: Started Ceph mds.f for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:46.653 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:46.653 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:46.653 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:46.654 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:46.654 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:46.654 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:46.654 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:46.654 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: Deploying daemon mds.c on smithi133 2024-09-10T10:43:46.654 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6834/4118505620,v1:172.21.15.133:6835/4118505620] up:boot 2024-09-10T10:43:46.654 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: fsmap 6 up:standby 2024-09-10T10:43:46.654 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:46 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:43:46.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:46.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:46.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:46.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: Deploying daemon mds.c on smithi133 2024-09-10T10:43:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6834/4118505620,v1:172.21.15.133:6835/4118505620] up:boot 2024-09-10T10:43:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: fsmap 6 up:standby 2024-09-10T10:43:46.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:46 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: Deploying daemon mds.c on smithi133 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6834/4118505620,v1:172.21.15.133:6835/4118505620] up:boot 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: fsmap 6 up:standby 2024-09-10T10:43:46.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:46 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:43:46.974 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:47.244 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:47.475 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 10:43:47 smithi133 systemd[1]: Starting Ceph mds.c for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:43:47.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.652+0000 7faecc1f5640 1 -- 172.21.15.7:0/3588312140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec4100b30 msgr2=0x7faec4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.652+0000 7faecc1f5640 1 --2- 172.21.15.7:0/3588312140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec4100b30 0x7faec4100f10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faec0009900 tx=0x7faec002a470 comp rx=0 tx=0).stop 2024-09-10T10:43:47.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.652+0000 7faecc1f5640 1 -- 172.21.15.7:0/3588312140 shutdown_connections 2024-09-10T10:43:47.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.652+0000 7faecc1f5640 1 --2- 172.21.15.7:0/3588312140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec4105f40 0x7faec41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.652+0000 7faecc1f5640 1 --2- 172.21.15.7:0/3588312140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4101450 0x7faec4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.652+0000 7faecc1f5640 1 --2- 172.21.15.7:0/3588312140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec4100b30 0x7faec4100f10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.652+0000 7faecc1f5640 1 -- 172.21.15.7:0/3588312140 >> 172.21.15.7:0/3588312140 conn(0x7faec40fc7a0 msgr2=0x7faec40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.653+0000 7faecc1f5640 1 -- 172.21.15.7:0/3588312140 shutdown_connections 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.653+0000 7faecc1f5640 1 -- 172.21.15.7:0/3588312140 wait complete. 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.653+0000 7faecc1f5640 1 Processor -- start 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.653+0000 7faecc1f5640 1 -- start start 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.653+0000 7faecc1f5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec4100b30 0x7faec41a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.653+0000 7faecc1f5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec4101450 0x7faec41a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faecc1f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4105f40 0x7faec419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faecc1f5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faec4118550 con 0x7faec4101450 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faecc1f5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faec41183d0 con 0x7faec4105f40 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faecc1f5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faec41186d0 con 0x7faec4100b30 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faec9790640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec4101450 0x7faec41a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faec9790640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec4101450 0x7faec41a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44592/0 (socket says 172.21.15.7:44592) 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faec9790640 1 -- 172.21.15.7:0/889559046 learned_addr learned my addr 172.21.15.7:0/889559046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faec9f91640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec4100b30 0x7faec41a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faeca792640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4105f40 0x7faec419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faec9790640 1 -- 172.21.15.7:0/889559046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec4100b30 msgr2=0x7faec41a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faec9790640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec4100b30 0x7faec41a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faec9790640 1 -- 172.21.15.7:0/889559046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4105f40 msgr2=0x7faec419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faec9790640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4105f40 0x7faec419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faec9790640 1 -- 172.21.15.7:0/889559046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec419ef40 con 0x7faec4101450 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faec9790640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec4101450 0x7faec41a4b60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faeb400c970 tx=0x7faeb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faebaffd640 1 -- 172.21.15.7:0/889559046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4007bb0 con 0x7faec4101450 2024-09-10T10:43:47.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faecc1f5640 1 -- 172.21.15.7:0/889559046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec419f230 con 0x7faec4101450 2024-09-10T10:43:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faebaffd640 1 -- 172.21.15.7:0/889559046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeb4007d50 con 0x7faec4101450 2024-09-10T10:43:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faecc1f5640 1 -- 172.21.15.7:0/889559046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec41ab480 con 0x7faec4101450 2024-09-10T10:43:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.654+0000 7faebaffd640 1 -- 172.21.15.7:0/889559046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4005260 con 0x7faec4101450 2024-09-10T10:43:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.655+0000 7faecc1f5640 1 -- 172.21.15.7:0/889559046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec406b070 con 0x7faec4101450 2024-09-10T10:43:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.657+0000 7faebaffd640 1 -- 172.21.15.7:0/889559046 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7faeb4005400 con 0x7faec4101450 2024-09-10T10:43:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.657+0000 7faebaffd640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faeb00a2b80 0x7faeb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.657+0000 7faec9f91640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faeb00a2b80 0x7faeb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.657+0000 7faebaffd640 1 -- 172.21.15.7:0/889559046 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7faeb40ced90 con 0x7faec4101450 2024-09-10T10:43:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.658+0000 7faec9f91640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faeb00a2b80 0x7faeb00a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faec0000c00 tx=0x7faec00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.660+0000 7faebaffd640 1 -- 172.21.15.7:0/889559046 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faeb4014030 con 0x7faec4101450 2024-09-10T10:43:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.875+0000 7faecc1f5640 1 -- 172.21.15.7:0/889559046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faec4105920 con 0x7faec4101450 2024-09-10T10:43:47.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.876+0000 7faebaffd640 1 -- 172.21.15.7:0/889559046 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7faec4105920 con 0x7faec4101450 2024-09-10T10:43:47.878 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:47.878 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"btime":"2024-09-10T10:43:46:098377+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-10T10:43:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2024-09-10T10:43:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.878+0000 7faecc1f5640 1 -- 172.21.15.7:0/889559046 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faeb00a2b80 msgr2=0x7faeb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.878+0000 7faecc1f5640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faeb00a2b80 0x7faeb00a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faec0000c00 tx=0x7faec00058d0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.878+0000 7faecc1f5640 1 -- 172.21.15.7:0/889559046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec4101450 msgr2=0x7faec41a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.878+0000 7faecc1f5640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec4101450 0x7faec41a4b60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faeb400c970 tx=0x7faeb400ce30 comp rx=0 tx=0).stop 2024-09-10T10:43:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.879+0000 7faecc1f5640 1 -- 172.21.15.7:0/889559046 shutdown_connections 2024-09-10T10:43:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.879+0000 7faecc1f5640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7faeb00a2b80 0x7faeb00a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.879+0000 7faecc1f5640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4105f40 0x7faec419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.879+0000 7faecc1f5640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faec4101450 0x7faec41a4b60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.879+0000 7faecc1f5640 1 --2- 172.21.15.7:0/889559046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec4100b30 0x7faec41a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.879+0000 7faecc1f5640 1 -- 172.21.15.7:0/889559046 >> 172.21.15.7:0/889559046 conn(0x7faec40fc7a0 msgr2=0x7faec41038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.879+0000 7faecc1f5640 1 -- 172.21.15.7:0/889559046 shutdown_connections 2024-09-10T10:43:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:47.879+0000 7faecc1f5640 1 -- 172.21.15.7:0/889559046 wait complete. 2024-09-10T10:43:47.958 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 10:43:47 smithi133 podman[65758]: 2024-09-10 10:43:47.518755809 +0000 UTC m=+0.015743284 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:43:47.958 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 10:43:47 smithi133 podman[65758]: 2024-09-10 10:43:47.704969264 +0000 UTC m=+0.201956740 container create 87822b8b05ce593d8c9ec0a04115bf7cc56750e2f1fc8e95d4a323791051c2f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T10:43:47.958 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 10:43:47 smithi133 podman[65758]: 2024-09-10 10:43:47.957152272 +0000 UTC m=+0.454139747 container init 87822b8b05ce593d8c9ec0a04115bf7cc56750e2f1fc8e95d4a323791051c2f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T10:43:48.225 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 10:43:47 smithi133 podman[65758]: 2024-09-10 10:43:47.960256103 +0000 UTC m=+0.457243579 container start 87822b8b05ce593d8c9ec0a04115bf7cc56750e2f1fc8e95d4a323791051c2f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T10:43:48.225 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 10:43:47 smithi133 ceph-mds[65778]: -- 172.21.15.133:0/2258188120 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8fee8b860 con 0x55e8fee68800 2024-09-10T10:43:48.225 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 10:43:47 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c[65774]: starting mds.c at 2024-09-10T10:43:48.225 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 10:43:48 smithi133 bash[65758]: 87822b8b05ce593d8c9ec0a04115bf7cc56750e2f1fc8e95d4a323791051c2f8 2024-09-10T10:43:48.225 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 10:43:48 smithi133 systemd[1]: Started Ceph mds.c for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:48 smithi007 ceph-mon[30873]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:48 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/889559046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:48 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:48 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:48 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:48 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:48 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:48.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:48 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:48.825 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:48 smithi103 ceph-mon[35160]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:48.825 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:48 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/889559046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:48.825 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:48 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:48.825 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:48 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:48.825 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:48 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:48.825 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:48 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:48.825 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:48 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:48.826 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:48 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:48.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:48 smithi133 ceph-mon[35282]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:48.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:48 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/889559046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:48.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:48 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:48.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:48 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:48.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:48 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:48.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:48 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:48.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:48 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:48.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:48 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:49.743 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:49 smithi103 ceph-mon[35160]: Deploying daemon mds.h on smithi103 2024-09-10T10:43:49.743 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:49 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] up:boot 2024-09-10T10:43:49.743 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:49 smithi103 ceph-mon[35160]: fsmap 7 up:standby 2024-09-10T10:43:49.743 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:49 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:43:49.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:49 smithi007 ceph-mon[30873]: Deploying daemon mds.h on smithi103 2024-09-10T10:43:49.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:49 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] up:boot 2024-09-10T10:43:49.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:49 smithi007 ceph-mon[30873]: fsmap 7 up:standby 2024-09-10T10:43:49.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:49 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:43:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:49 smithi133 ceph-mon[35282]: Deploying daemon mds.h on smithi103 2024-09-10T10:43:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:49 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] up:boot 2024-09-10T10:43:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:49 smithi133 ceph-mon[35282]: fsmap 7 up:standby 2024-09-10T10:43:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:49 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:43:50.133 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 10:43:49 smithi103 systemd[1]: Starting Ceph mds.h for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:43:50.134 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 10:43:50 smithi103 podman[65590]: 2024-09-10 10:43:50.035210471 +0000 UTC m=+0.015941638 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:43:50.385 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 10:43:50 smithi103 podman[65590]: 2024-09-10 10:43:50.139640396 +0000 UTC m=+0.120371615 container create 95c1fb125d95c1a226be3cc6eb77133a64fda04ca881833d3f93eb8827ee792d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T10:43:50.385 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 10:43:50 smithi103 podman[65590]: 2024-09-10 10:43:50.358656447 +0000 UTC m=+0.339387613 container init 95c1fb125d95c1a226be3cc6eb77133a64fda04ca881833d3f93eb8827ee792d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T10:43:50.385 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 10:43:50 smithi103 podman[65590]: 2024-09-10 10:43:50.361425464 +0000 UTC m=+0.342156622 container start 95c1fb125d95c1a226be3cc6eb77133a64fda04ca881833d3f93eb8827ee792d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T10:43:50.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:50.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:50 smithi103 ceph-mon[35160]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:50.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:50 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:50.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:50 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:50.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:50 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:50.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:50 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:50.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:50 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:50.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:50 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:50.673 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 10:43:50 smithi103 ceph-mds[65609]: -- 172.21.15.103:0/2191138530 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fa87bb860 con 0x563fa8798800 2024-09-10T10:43:50.673 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 10:43:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h[65605]: starting mds.h at 2024-09-10T10:43:50.673 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 10:43:50 smithi103 bash[65590]: 95c1fb125d95c1a226be3cc6eb77133a64fda04ca881833d3f93eb8827ee792d 2024-09-10T10:43:50.673 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 10:43:50 smithi103 systemd[1]: Started Ceph mds.h for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:50.779 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:50.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:50 smithi007 ceph-mon[30873]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:50.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:50 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:50.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:50 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:50.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:50 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:50.901 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:50 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:50.902 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:50 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:50.902 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:50 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:50.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:50 smithi133 ceph-mon[35282]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:50.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:50 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:50.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:50 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:50.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:50 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:50.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:50 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:50.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:50 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:50.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:50 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.293+0000 7fde75a74640 1 -- 172.21.15.7:0/1455247214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde701036e0 msgr2=0x7fde70107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.293+0000 7fde75a74640 1 --2- 172.21.15.7:0/1455247214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde701036e0 0x7fde70107b90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fde58009930 tx=0x7fde5802a420 comp rx=0 tx=0).stop 2024-09-10T10:43:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.293+0000 7fde75a74640 1 -- 172.21.15.7:0/1455247214 shutdown_connections 2024-09-10T10:43:51.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.293+0000 7fde75a74640 1 --2- 172.21.15.7:0/1455247214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde701082a0 0x7fde7010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.293+0000 7fde75a74640 1 --2- 172.21.15.7:0/1455247214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde701036e0 0x7fde70107b90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.293+0000 7fde75a74640 1 --2- 172.21.15.7:0/1455247214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde70102d30 0x7fde70103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.293+0000 7fde75a74640 1 -- 172.21.15.7:0/1455247214 >> 172.21.15.7:0/1455247214 conn(0x7fde700fc7d0 msgr2=0x7fde700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.293+0000 7fde75a74640 1 -- 172.21.15.7:0/1455247214 shutdown_connections 2024-09-10T10:43:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.293+0000 7fde75a74640 1 -- 172.21.15.7:0/1455247214 wait complete. 2024-09-10T10:43:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.294+0000 7fde75a74640 1 Processor -- start 2024-09-10T10:43:51.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.294+0000 7fde75a74640 1 -- start start 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.294+0000 7fde75a74640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde70102d30 0x7fde701985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.294+0000 7fde75a74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde701036e0 0x7fde70198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde75a74640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde701082a0 0x7fde7019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde75a74640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde701120a0 con 0x7fde70102d30 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde75a74640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fde70111f20 con 0x7fde701082a0 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde75a74640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fde70112220 con 0x7fde701036e0 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde70102d30 0x7fde701985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde70102d30 0x7fde701985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33612/0 (socket says 172.21.15.7:33612) 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6f7fe640 1 -- 172.21.15.7:0/2193022199 learned_addr learned my addr 172.21.15.7:0/2193022199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6effd640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde701036e0 0x7fde70198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6ffff640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde701082a0 0x7fde7019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:51.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6f7fe640 1 -- 172.21.15.7:0/2193022199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde701036e0 msgr2=0x7fde70198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6f7fe640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde701036e0 0x7fde70198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6f7fe640 1 -- 172.21.15.7:0/2193022199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde701082a0 msgr2=0x7fde7019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6f7fe640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde701082a0 0x7fde7019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6f7fe640 1 -- 172.21.15.7:0/2193022199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7019d570 con 0x7fde70102d30 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6f7fe640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde70102d30 0x7fde701985c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fde6000e9f0 tx=0x7fde6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6cff9640 1 -- 172.21.15.7:0/2193022199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6000cd80 con 0x7fde70102d30 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6cff9640 1 -- 172.21.15.7:0/2193022199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde60010910 con 0x7fde70102d30 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde75a74640 1 -- 172.21.15.7:0/2193022199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7019d860 con 0x7fde70102d30 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.295+0000 7fde6cff9640 1 -- 172.21.15.7:0/2193022199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60018990 con 0x7fde70102d30 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.296+0000 7fde75a74640 1 -- 172.21.15.7:0/2193022199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde701a5140 con 0x7fde70102d30 2024-09-10T10:43:51.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.296+0000 7fde75a74640 1 -- 172.21.15.7:0/2193022199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7006a850 con 0x7fde70102d30 2024-09-10T10:43:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.298+0000 7fde6cff9640 1 -- 172.21.15.7:0/2193022199 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fde60010ac0 con 0x7fde70102d30 2024-09-10T10:43:51.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.299+0000 7fde6cff9640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fde5c0a2b80 0x7fde5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:51.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.299+0000 7fde6effd640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fde5c0a2b80 0x7fde5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:51.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.299+0000 7fde6cff9640 1 -- 172.21.15.7:0/2193022199 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fde600cf2f0 con 0x7fde70102d30 2024-09-10T10:43:51.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.300+0000 7fde6effd640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fde5c0a2b80 0x7fde5c0a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fde58005940 tx=0x7fde580058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:51.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.303+0000 7fde6cff9640 1 -- 172.21.15.7:0/2193022199 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde6009c0a0 con 0x7fde70102d30 2024-09-10T10:43:51.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.515+0000 7fde75a74640 1 -- 172.21.15.7:0/2193022199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde70103110 con 0x7fde70102d30 2024-09-10T10:43:51.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.516+0000 7fde6cff9640 1 -- 172.21.15.7:0/2193022199 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7219 (secure 0 0 0) 0x7fde600a03a0 con 0x7fde70102d30 2024-09-10T10:43:51.517 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:51.518 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"btime":"2024-09-10T10:43:50:550076+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T10:43:51.518 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-09-10T10:43:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 -- 172.21.15.7:0/2193022199 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fde5c0a2b80 msgr2=0x7fde5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fde5c0a2b80 0x7fde5c0a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fde58005940 tx=0x7fde580058d0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 -- 172.21.15.7:0/2193022199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde70102d30 msgr2=0x7fde701985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde70102d30 0x7fde701985c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fde6000e9f0 tx=0x7fde6000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 -- 172.21.15.7:0/2193022199 shutdown_connections 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fde5c0a2b80 0x7fde5c0a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fde701082a0 0x7fde7019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde701036e0 0x7fde70198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 --2- 172.21.15.7:0/2193022199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde70102d30 0x7fde701985c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 -- 172.21.15.7:0/2193022199 >> 172.21.15.7:0/2193022199 conn(0x7fde700fc7d0 msgr2=0x7fde70105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 -- 172.21.15.7:0/2193022199 shutdown_connections 2024-09-10T10:43:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:51.518+0000 7fde75a74640 1 -- 172.21.15.7:0/2193022199 wait complete. 2024-09-10T10:43:51.846 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:51 smithi133 ceph-mon[35282]: Deploying daemon mds.i on smithi133 2024-09-10T10:43:51.846 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:51 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.103:6836/2960409846,v1:172.21.15.103:6837/2960409846] up:boot 2024-09-10T10:43:51.846 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:51 smithi133 ceph-mon[35282]: fsmap 8 up:standby 2024-09-10T10:43:51.847 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:51 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:43:51.847 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:51 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2193022199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:51 smithi103 ceph-mon[35160]: Deploying daemon mds.i on smithi133 2024-09-10T10:43:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:51 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.103:6836/2960409846,v1:172.21.15.103:6837/2960409846] up:boot 2024-09-10T10:43:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:51 smithi103 ceph-mon[35160]: fsmap 8 up:standby 2024-09-10T10:43:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:51 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:43:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:51 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2193022199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:51.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:51 smithi007 ceph-mon[30873]: Deploying daemon mds.i on smithi133 2024-09-10T10:43:51.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:51 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.103:6836/2960409846,v1:172.21.15.103:6837/2960409846] up:boot 2024-09-10T10:43:51.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:51 smithi007 ceph-mon[30873]: fsmap 8 up:standby 2024-09-10T10:43:51.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:51 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:43:51.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:51 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2193022199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:52.877 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:52 smithi133 ceph-mon[35282]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:52.878 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 10:43:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-i[66074]: starting mds.i at 2024-09-10T10:43:52.878 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 10:43:52 smithi133 bash[66059]: 354565dfd37317baf857afc12e4acefc7a880984699f095de4651e5a7e52b1e2 2024-09-10T10:43:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:52 smithi103 ceph-mon[35160]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:52.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:52 smithi007 ceph-mon[30873]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:53.225 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 10:43:52 smithi133 systemd[1]: Started Ceph mds.i for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:54.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:54.181 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:54.181 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:54.181 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:54.181 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:54.181 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:54.181 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:54.181 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:54.182 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: Deploying daemon mds.g on smithi007 2024-09-10T10:43:54.182 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] up:boot 2024-09-10T10:43:54.182 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: fsmap 9 up:standby 2024-09-10T10:43:54.182 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:53 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:43:54.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:54.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:54.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:54.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:54.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:54.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:54.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:54.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: Deploying daemon mds.g on smithi007 2024-09-10T10:43:54.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] up:boot 2024-09-10T10:43:54.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: fsmap 9 up:standby 2024-09-10T10:43:54.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:53 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:43:54.420 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:54.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:54.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:54.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:54.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:54.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:54.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:54.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:54.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: Deploying daemon mds.g on smithi007 2024-09-10T10:43:54.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] up:boot 2024-09-10T10:43:54.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: fsmap 9 up:standby 2024-09-10T10:43:54.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:53 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:43:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.021+0000 7f202a6fc640 1 -- 172.21.15.7:0/3386334866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2024106f10 msgr2=0x7f202410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.021+0000 7f202a6fc640 1 --2- 172.21.15.7:0/3386334866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2024106f10 0x7f202410ea90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f201c009ca0 tx=0x7f201c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.021+0000 7f202a6fc640 1 -- 172.21.15.7:0/3386334866 shutdown_connections 2024-09-10T10:43:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.021+0000 7f202a6fc640 1 --2- 172.21.15.7:0/3386334866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2024106f10 0x7f202410ea90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.021+0000 7f202a6fc640 1 --2- 172.21.15.7:0/3386334866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2024102790 0x7f20241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.021+0000 7f202a6fc640 1 --2- 172.21.15.7:0/3386334866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2024101de0 0x7f20241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.021+0000 7f202a6fc640 1 -- 172.21.15.7:0/3386334866 >> 172.21.15.7:0/3386334866 conn(0x7f20240fd8c0 msgr2=0x7f20240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.021+0000 7f202a6fc640 1 -- 172.21.15.7:0/3386334866 shutdown_connections 2024-09-10T10:43:55.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.021+0000 7f202a6fc640 1 -- 172.21.15.7:0/3386334866 wait complete. 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f202a6fc640 1 Processor -- start 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f202a6fc640 1 -- start start 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f202a6fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2024101de0 0x7f20241983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f202a6fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2024102790 0x7f2024198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f202a6fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2024106f10 0x7f202419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f202a6fc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2024111f70 con 0x7f2024106f10 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f202a6fc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2024111df0 con 0x7f2024101de0 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f202a6fc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f20241120f0 con 0x7f2024102790 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f2028c99640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2024106f10 0x7f202419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f2028c99640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2024106f10 0x7f202419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33622/0 (socket says 172.21.15.7:33622) 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f2028c99640 1 -- 172.21.15.7:0/2119678992 learned_addr learned my addr 172.21.15.7:0/2119678992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f201bfff640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2024102790 0x7f2024198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f2023fff640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2024101de0 0x7f20241983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f2028c99640 1 -- 172.21.15.7:0/2119678992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2024102790 msgr2=0x7f2024198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f2028c99640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2024102790 0x7f2024198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f2028c99640 1 -- 172.21.15.7:0/2119678992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2024101de0 msgr2=0x7f20241983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f2028c99640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2024101de0 0x7f20241983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.022+0000 7f2028c99640 1 -- 172.21.15.7:0/2119678992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202419d370 con 0x7f2024106f10 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.023+0000 7f2028c99640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2024106f10 0x7f202419cc90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f201c031a60 tx=0x7f201c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.023+0000 7f2021ffb640 1 -- 172.21.15.7:0/2119678992 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c003c30 con 0x7f2024106f10 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.023+0000 7f2021ffb640 1 -- 172.21.15.7:0/2119678992 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201c003dd0 con 0x7f2024106f10 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.023+0000 7f202a6fc640 1 -- 172.21.15.7:0/2119678992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202419d600 con 0x7f2024106f10 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.023+0000 7f2021ffb640 1 -- 172.21.15.7:0/2119678992 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c0047e0 con 0x7f2024106f10 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.023+0000 7f202a6fc640 1 -- 172.21.15.7:0/2119678992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20241a4f40 con 0x7f2024106f10 2024-09-10T10:43:55.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.024+0000 7f202a6fc640 1 -- 172.21.15.7:0/2119678992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20241021c0 con 0x7f2024106f10 2024-09-10T10:43:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.025+0000 7f2021ffb640 1 -- 172.21.15.7:0/2119678992 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f201c004b40 con 0x7f2024106f10 2024-09-10T10:43:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.026+0000 7f2021ffb640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f20100a2c00 0x7f20100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.026+0000 7f2023fff640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f20100a2c00 0x7f20100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.026+0000 7f2021ffb640 1 -- 172.21.15.7:0/2119678992 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f201c0f00a0 con 0x7f2024106f10 2024-09-10T10:43:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.026+0000 7f2023fff640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f20100a2c00 0x7f20100a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f20140058c0 tx=0x7f201400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.029+0000 7f2021ffb640 1 -- 172.21.15.7:0/2119678992 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f201c0bd4a0 con 0x7f2024106f10 2024-09-10T10:43:55.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.252+0000 7f202a6fc640 1 -- 172.21.15.7:0/2119678992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2024199720 con 0x7f2024106f10 2024-09-10T10:43:55.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.252+0000 7f2021ffb640 1 -- 172.21.15.7:0/2119678992 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8049 (secure 0 0 0) 0x7f2024199720 con 0x7f2024106f10 2024-09-10T10:43:55.254 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:55.254 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"btime":"2024-09-10T10:43:53:563111+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T10:43:55.254 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 10 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.254+0000 7f202a6fc640 1 -- 172.21.15.7:0/2119678992 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f20100a2c00 msgr2=0x7f20100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.254+0000 7f202a6fc640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f20100a2c00 0x7f20100a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f20140058c0 tx=0x7f201400a4c0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.254+0000 7f202a6fc640 1 -- 172.21.15.7:0/2119678992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2024106f10 msgr2=0x7f202419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.254+0000 7f202a6fc640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2024106f10 0x7f202419cc90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f201c031a60 tx=0x7f201c031a90 comp rx=0 tx=0).stop 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.254+0000 7f202a6fc640 1 -- 172.21.15.7:0/2119678992 shutdown_connections 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.254+0000 7f202a6fc640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f20100a2c00 0x7f20100a50c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.254+0000 7f202a6fc640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2024106f10 0x7f202419cc90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.254+0000 7f202a6fc640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2024102790 0x7f2024198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.254+0000 7f202a6fc640 1 --2- 172.21.15.7:0/2119678992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2024101de0 0x7f20241983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.254+0000 7f202a6fc640 1 -- 172.21.15.7:0/2119678992 >> 172.21.15.7:0/2119678992 conn(0x7f20240fd8c0 msgr2=0x7f20240fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.255+0000 7f202a6fc640 1 -- 172.21.15.7:0/2119678992 shutdown_connections 2024-09-10T10:43:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:55.255+0000 7f202a6fc640 1 -- 172.21.15.7:0/2119678992 wait complete. 2024-09-10T10:43:55.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:55 smithi007 ceph-mon[30873]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:55.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:55 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2119678992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:56.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:55 smithi133 ceph-mon[35282]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:56.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:55 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2119678992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:56.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:55 smithi103 ceph-mon[35160]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:56.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:55 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2119678992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:56.433 INFO:journalctl@ceph.mds.g.smithi007.stdout:Sep 10 10:43:56 smithi007 systemd[1]: Started Ceph mds.g for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:57.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:57.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:57.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:57.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:57.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:57.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:57.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:57 smithi007 ceph-mon[30873]: Deploying daemon mds.a on smithi007 2024-09-10T10:43:57.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:57 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.7:6838/1984917528,v1:172.21.15.7:6839/1984917528] up:boot 2024-09-10T10:43:57.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:57 smithi007 ceph-mon[30873]: fsmap 10 up:standby 2024-09-10T10:43:57.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:57 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:43:57.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:57.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:57.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:57.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:57.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:57.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:57.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:57 smithi133 ceph-mon[35282]: Deploying daemon mds.a on smithi007 2024-09-10T10:43:57.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:57 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.7:6838/1984917528,v1:172.21.15.7:6839/1984917528] up:boot 2024-09-10T10:43:57.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:57 smithi133 ceph-mon[35282]: fsmap 10 up:standby 2024-09-10T10:43:57.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:57 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:43:57.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:57.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:57.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:57.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:57.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:57.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:57.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:57 smithi103 ceph-mon[35160]: Deploying daemon mds.a on smithi007 2024-09-10T10:43:57.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:57 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.7:6838/1984917528,v1:172.21.15.7:6839/1984917528] up:boot 2024-09-10T10:43:57.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:57 smithi103 ceph-mon[35160]: fsmap 10 up:standby 2024-09-10T10:43:57.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:57 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:43:57.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:43:58.332 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:58 smithi007 ceph-mon[30873]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:58.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:58 smithi133 ceph-mon[35282]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:58.587 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 10:43:58 smithi007 podman[72489]: 2024-09-10 10:43:58.562359346 +0000 UTC m=+0.344598756 container start 064f0afb982e97ca509425b94ccfb0aa634862fd8d2ade74cfae80f440681f29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:43:58.665 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:43:58.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:58 smithi103 ceph-mon[35160]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:58.929 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 10:43:58 smithi007 ceph-mds[72535]: -- 172.21.15.7:0/3746091610 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648fe607860 con 0x5648fe5e5000 2024-09-10T10:43:58.929 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 10:43:58 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-a[72531]: starting mds.a at 2024-09-10T10:43:58.929 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 10:43:58 smithi007 bash[72489]: 064f0afb982e97ca509425b94ccfb0aa634862fd8d2ade74cfae80f440681f29 2024-09-10T10:43:58.930 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 10:43:58 smithi007 systemd[1]: Started Ceph mds.a for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:43:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.145+0000 7f44df8cf640 1 -- 172.21.15.7:0/3561589963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d8100480 msgr2=0x7f44d8100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.145+0000 7f44df8cf640 1 --2- 172.21.15.7:0/3561589963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d8100480 0x7f44d8100860 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009900 tx=0x7f44d402a410 comp rx=0 tx=0).stop 2024-09-10T10:43:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.145+0000 7f44df8cf640 1 -- 172.21.15.7:0/3561589963 shutdown_connections 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.145+0000 7f44df8cf640 1 --2- 172.21.15.7:0/3561589963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44d81055b0 0x7f44d810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.145+0000 7f44df8cf640 1 --2- 172.21.15.7:0/3561589963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d8100e30 0x7f44d8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.145+0000 7f44df8cf640 1 --2- 172.21.15.7:0/3561589963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d8100480 0x7f44d8100860 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.145+0000 7f44df8cf640 1 -- 172.21.15.7:0/3561589963 >> 172.21.15.7:0/3561589963 conn(0x7f44d8077210 msgr2=0x7f44d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.146+0000 7f44df8cf640 1 -- 172.21.15.7:0/3561589963 shutdown_connections 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.146+0000 7f44df8cf640 1 -- 172.21.15.7:0/3561589963 wait complete. 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.146+0000 7f44df8cf640 1 Processor -- start 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.146+0000 7f44df8cf640 1 -- start start 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.146+0000 7f44df8cf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d8100480 0x7f44d80fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.146+0000 7f44df8cf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44d8100e30 0x7f44d80fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.146+0000 7f44df8cf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d81055b0 0x7f44d80fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.146+0000 7f44df8cf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44d8110ec0 con 0x7f44d81055b0 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44df8cf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44d8110d40 con 0x7f44d8100480 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44df8cf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44d8111040 con 0x7f44d8100e30 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dde6c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d81055b0 0x7f44d80fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dde6c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d81055b0 0x7f44d80fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33680/0 (socket says 172.21.15.7:33680) 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dde6c640 1 -- 172.21.15.7:0/2545050813 learned_addr learned my addr 172.21.15.7:0/2545050813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dd66b640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d8100480 0x7f44d80fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dce6a640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44d8100e30 0x7f44d80fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dde6c640 1 -- 172.21.15.7:0/2545050813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44d8100e30 msgr2=0x7f44d80fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dde6c640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44d8100e30 0x7f44d80fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dde6c640 1 -- 172.21.15.7:0/2545050813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d8100480 msgr2=0x7f44d80fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dde6c640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d8100480 0x7f44d80fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dde6c640 1 -- 172.21.15.7:0/2545050813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d81a6a30 con 0x7f44d81055b0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44dde6c640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d81055b0 0x7f44d80fd110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44d000b970 tx=0x7f44d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44ce7fc640 1 -- 172.21.15.7:0/2545050813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0019070 con 0x7f44d81055b0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44df8cf640 1 -- 172.21.15.7:0/2545050813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d81a6d20 con 0x7f44d81055b0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44ce7fc640 1 -- 172.21.15.7:0/2545050813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d000cb40 con 0x7f44d81055b0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44ce7fc640 1 -- 172.21.15.7:0/2545050813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0007610 con 0x7f44d81055b0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.147+0000 7f44df8cf640 1 -- 172.21.15.7:0/2545050813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d81a7260 con 0x7f44d81055b0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.148+0000 7f44df8cf640 1 -- 172.21.15.7:0/2545050813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d8100860 con 0x7f44d81055b0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.150+0000 7f44ce7fc640 1 -- 172.21.15.7:0/2545050813 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f44d001c070 con 0x7f44d81055b0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.151+0000 7f44ce7fc640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44c40a2b80 0x7f44c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.151+0000 7f44dd66b640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44c40a2b80 0x7f44c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.151+0000 7f44ce7fc640 1 -- 172.21.15.7:0/2545050813 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f44d00ce600 con 0x7f44d81055b0 2024-09-10T10:43:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.151+0000 7f44dd66b640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44c40a2b80 0x7f44c40a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44d4000c00 tx=0x7f44d40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.154+0000 7f44ce7fc640 1 -- 172.21.15.7:0/2545050813 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44d009b310 con 0x7f44d81055b0 2024-09-10T10:43:59.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.369+0000 7f44df8cf640 1 -- 172.21.15.7:0/2545050813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44d81a74f0 con 0x7f44d81055b0 2024-09-10T10:43:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.370+0000 7f44ce7fc640 1 -- 172.21.15.7:0/2545050813 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7f44d009f610 con 0x7f44d81055b0 2024-09-10T10:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:43:59.371 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"btime":"2024-09-10T10:43:59:195089+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1984917528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1984917528},{"type":"v1","addr":"172.21.15.7:6839","nonce":1984917528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1887416712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1887416712},{"type":"v1","addr":"172.21.15.7:6841","nonce":1887416712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T10:43:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2024-09-10T10:43:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.372+0000 7f44df8cf640 1 -- 172.21.15.7:0/2545050813 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44c40a2b80 msgr2=0x7f44c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.372+0000 7f44df8cf640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44c40a2b80 0x7f44c40a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f44d4000c00 tx=0x7f44d40058d0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.372+0000 7f44df8cf640 1 -- 172.21.15.7:0/2545050813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d81055b0 msgr2=0x7f44d80fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.372+0000 7f44df8cf640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d81055b0 0x7f44d80fd110 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44d000b970 tx=0x7f44d000be30 comp rx=0 tx=0).stop 2024-09-10T10:43:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.372+0000 7f44df8cf640 1 -- 172.21.15.7:0/2545050813 shutdown_connections 2024-09-10T10:43:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.372+0000 7f44df8cf640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44c40a2b80 0x7f44c40a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.372+0000 7f44df8cf640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d81055b0 0x7f44d80fd110 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.372+0000 7f44df8cf640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44d8100e30 0x7f44d80fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.372+0000 7f44df8cf640 1 --2- 172.21.15.7:0/2545050813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44d8100480 0x7f44d80fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.372+0000 7f44df8cf640 1 -- 172.21.15.7:0/2545050813 >> 172.21.15.7:0/2545050813 conn(0x7f44d8077210 msgr2=0x7f44d810df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.373+0000 7f44df8cf640 1 -- 172.21.15.7:0/2545050813 shutdown_connections 2024-09-10T10:43:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:43:59.373+0000 7f44df8cf640 1 -- 172.21.15.7:0/2545050813 wait complete. 2024-09-10T10:43:59.782 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:59.782 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:59.783 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:59.783 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:59.783 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:59.783 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:59.783 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:59.783 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: Deploying daemon mds.b on smithi103 2024-09-10T10:43:59.783 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.7:6840/1887416712,v1:172.21.15.7:6841/1887416712] up:boot 2024-09-10T10:43:59.783 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: fsmap 11 up:standby 2024-09-10T10:43:59.783 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:43:59.783 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:43:59 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2545050813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: Deploying daemon mds.b on smithi103 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.7:6840/1887416712,v1:172.21.15.7:6841/1887416712] up:boot 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: fsmap 11 up:standby 2024-09-10T10:43:59.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:43:59.983 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:43:59 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2545050813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:43:59.983 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:43:59.983 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:59.983 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:59.983 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:43:59.983 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:43:59.983 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:43:59.983 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:43:59.983 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: Deploying daemon mds.b on smithi103 2024-09-10T10:43:59.984 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.7:6840/1887416712,v1:172.21.15.7:6841/1887416712] up:boot 2024-09-10T10:43:59.984 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: fsmap 11 up:standby 2024-09-10T10:43:59.984 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:43:59.984 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:43:59 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2545050813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:00.869 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 10:44:00 smithi103 systemd[1]: Starting Ceph mds.b for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T10:44:01.172 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 10:44:00 smithi103 podman[65895]: 2024-09-10 10:44:00.770976613 +0000 UTC m=+0.015824490 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:44:01.172 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 10:44:00 smithi103 podman[65895]: 2024-09-10 10:44:00.898260435 +0000 UTC m=+0.143108314 container create 0aeddf290aee43d07cf712323d32fcd69a4515346ce38006ab163ac7419bf798 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T10:44:01.672 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 10:44:01 smithi103 podman[65895]: 2024-09-10 10:44:01.196819023 +0000 UTC m=+0.441666894 container init 0aeddf290aee43d07cf712323d32fcd69a4515346ce38006ab163ac7419bf798 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:44:01.672 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 10:44:01 smithi103 podman[65895]: 2024-09-10 10:44:01.199829341 +0000 UTC m=+0.444677212 container start 0aeddf290aee43d07cf712323d32fcd69a4515346ce38006ab163ac7419bf798 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T10:44:01.672 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 10:44:01 smithi103 ceph-mds[65915]: -- 172.21.15.103:0/809820567 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56242fad3860 con 0x56242fab1400 2024-09-10T10:44:01.672 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 10:44:01 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b[65911]: starting mds.b at 2024-09-10T10:44:01.672 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 10:44:01 smithi103 bash[65895]: 0aeddf290aee43d07cf712323d32fcd69a4515346ce38006ab163ac7419bf798 2024-09-10T10:44:01.673 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 10:44:01 smithi103 systemd[1]: Started Ceph mds.b for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T10:44:02.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:44:02.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:02 smithi133 ceph-mon[35282]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:02.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:44:02.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:02 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] up:boot 2024-09-10T10:44:02.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:02 smithi133 ceph-mon[35282]: fsmap 12 up:standby 2024-09-10T10:44:02.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:02 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:44:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:02.597 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:02 smithi007 ceph-mon[30873]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:02.597 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.597 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.597 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.597 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.597 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.598 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:44:02.598 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:02 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] up:boot 2024-09-10T10:44:02.598 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:02 smithi007 ceph-mon[30873]: fsmap 12 up:standby 2024-09-10T10:44:02.598 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:02 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:44:02.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:02 smithi103 ceph-mon[35160]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:02.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:44:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:02 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] up:boot 2024-09-10T10:44:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:02 smithi103 ceph-mon[35160]: fsmap 12 up:standby 2024-09-10T10:44:02.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:02 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:44:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.025+0000 7f3f83a42640 1 -- 172.21.15.7:0/1484516867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f7c101430 msgr2=0x7f3f7c10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.025+0000 7f3f83a42640 1 --2- 172.21.15.7:0/1484516867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f7c101430 0x7f3f7c10d960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009cd0 tx=0x7f3f7402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.025+0000 7f3f83a42640 1 -- 172.21.15.7:0/1484516867 shutdown_connections 2024-09-10T10:44:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.025+0000 7f3f83a42640 1 --2- 172.21.15.7:0/1484516867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f7c101430 0x7f3f7c10d960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.025+0000 7f3f83a42640 1 --2- 172.21.15.7:0/1484516867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f7c100a90 0x7f3f7c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.025+0000 7f3f83a42640 1 --2- 172.21.15.7:0/1484516867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f7c106a90 0x7f3f7c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.025+0000 7f3f83a42640 1 -- 172.21.15.7:0/1484516867 >> 172.21.15.7:0/1484516867 conn(0x7f3f7c0fc7d0 msgr2=0x7f3f7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.025+0000 7f3f83a42640 1 -- 172.21.15.7:0/1484516867 shutdown_connections 2024-09-10T10:44:03.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.026+0000 7f3f83a42640 1 -- 172.21.15.7:0/1484516867 wait complete. 2024-09-10T10:44:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.026+0000 7f3f83a42640 1 Processor -- start 2024-09-10T10:44:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.026+0000 7f3f83a42640 1 -- start start 2024-09-10T10:44:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.026+0000 7f3f83a42640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f7c100a90 0x7f3f7c10fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.026+0000 7f3f83a42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f7c101430 0x7f3f7c110300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.026+0000 7f3f83a42640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f7c106a90 0x7f3f7c110840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.026+0000 7f3f83a42640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c112080 con 0x7f3f7c100a90 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.026+0000 7f3f83a42640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c111f00 con 0x7f3f7c101430 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.026+0000 7f3f83a42640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c112200 con 0x7f3f7c106a90 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.027+0000 7f3f817de640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f7c100a90 0x7f3f7c10fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.027+0000 7f3f817de640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f7c100a90 0x7f3f7c10fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58056/0 (socket says 172.21.15.7:58056) 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.027+0000 7f3f817de640 1 -- 172.21.15.7:0/605241952 learned_addr learned my addr 172.21.15.7:0/605241952 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.027+0000 7f3f80fdd640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f7c101430 0x7f3f7c110300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.027+0000 7f3f817de640 1 -- 172.21.15.7:0/605241952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f7c106a90 msgr2=0x7f3f7c110840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:44:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.027+0000 7f3f817de640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f7c106a90 0x7f3f7c110840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.027+0000 7f3f817de640 1 -- 172.21.15.7:0/605241952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f7c101430 msgr2=0x7f3f7c110300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.027+0000 7f3f817de640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f7c101430 0x7f3f7c110300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.027+0000 7f3f817de640 1 -- 172.21.15.7:0/605241952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c1a6870 con 0x7f3f7c100a90 2024-09-10T10:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.027+0000 7f3f817de640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f7c100a90 0x7f3f7c10fdc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800d950 tx=0x7f3f7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.028+0000 7f3f727fc640 1 -- 172.21.15.7:0/605241952 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78017030 con 0x7f3f7c100a90 2024-09-10T10:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.028+0000 7f3f727fc640 1 -- 172.21.15.7:0/605241952 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f78004020 con 0x7f3f7c100a90 2024-09-10T10:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.028+0000 7f3f83a42640 1 -- 172.21.15.7:0/605241952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c1a6b60 con 0x7f3f7c100a90 2024-09-10T10:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.028+0000 7f3f727fc640 1 -- 172.21.15.7:0/605241952 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f780047a0 con 0x7f3f7c100a90 2024-09-10T10:44:03.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.028+0000 7f3f83a42640 1 -- 172.21.15.7:0/605241952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c1a70a0 con 0x7f3f7c100a90 2024-09-10T10:44:03.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.030+0000 7f3f83a42640 1 -- 172.21.15.7:0/605241952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7c100ef0 con 0x7f3f7c100a90 2024-09-10T10:44:03.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.030+0000 7f3f727fc640 1 -- 172.21.15.7:0/605241952 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3f780041d0 con 0x7f3f7c100a90 2024-09-10T10:44:03.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.031+0000 7f3f727fc640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3f680a2b80 0x7f3f680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:03.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.031+0000 7f3f80fdd640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3f680a2b80 0x7f3f680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:03.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.031+0000 7f3f727fc640 1 -- 172.21.15.7:0/605241952 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f3f780ce4e0 con 0x7f3f7c100a90 2024-09-10T10:44:03.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.032+0000 7f3f80fdd640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3f680a2b80 0x7f3f680a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c111850 tx=0x7f3f6c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:03.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.035+0000 7f3f727fc640 1 -- 172.21.15.7:0/605241952 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f7809b8e0 con 0x7f3f7c100a90 2024-09-10T10:44:03.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.266+0000 7f3f83a42640 1 -- 172.21.15.7:0/605241952 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f7c1a7370 con 0x7f3f7c100a90 2024-09-10T10:44:03.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.266+0000 7f3f727fc640 1 -- 172.21.15.7:0/605241952 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7f3f780b9e20 con 0x7f3f7c100a90 2024-09-10T10:44:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:03.268 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"btime":"2024-09-10T10:44:01:686361+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1984917528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1984917528},{"type":"v1","addr":"172.21.15.7:6839","nonce":1984917528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1887416712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1887416712},{"type":"v1","addr":"172.21.15.7:6841","nonce":1887416712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2537001809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2537001809},{"type":"v1","addr":"172.21.15.103:6839","nonce":2537001809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24499,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T10:44:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2024-09-10T10:44:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 -- 172.21.15.7:0/605241952 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3f680a2b80 msgr2=0x7f3f680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3f680a2b80 0x7f3f680a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c111850 tx=0x7f3f6c00b390 comp rx=0 tx=0).stop 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 -- 172.21.15.7:0/605241952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f7c100a90 msgr2=0x7f3f7c10fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f7c100a90 0x7f3f7c10fdc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800d950 tx=0x7f3f7800de10 comp rx=0 tx=0).stop 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 -- 172.21.15.7:0/605241952 shutdown_connections 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3f680a2b80 0x7f3f680a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f7c106a90 0x7f3f7c110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f7c101430 0x7f3f7c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 --2- 172.21.15.7:0/605241952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f7c100a90 0x7f3f7c10fdc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 -- 172.21.15.7:0/605241952 >> 172.21.15.7:0/605241952 conn(0x7f3f7c0fc7d0 msgr2=0x7f3f7c0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 -- 172.21.15.7:0/605241952 shutdown_connections 2024-09-10T10:44:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:03.270+0000 7f3f83a42640 1 -- 172.21.15.7:0/605241952 wait complete. 2024-09-10T10:44:03.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T10:44:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:04.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/605241952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:04.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:44:04.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:44:04.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:03 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:44:04.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:04.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/605241952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:04.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:44:04.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:44:04.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:03 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:44:04.187 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:04.187 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.187 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.187 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.188 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.188 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/605241952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:04.188 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.188 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.188 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:44:04.188 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:44:04.189 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:04.189 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:03 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:44:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.109+0000 7fc245561640 1 -- 172.21.15.7:0/3200632121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc24010c170 msgr2=0x7fc24010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.109+0000 7fc245561640 1 --2- 172.21.15.7:0/3200632121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc24010c170 0x7fc24010e610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc238009ca0 tx=0x7fc23802a950 comp rx=0 tx=0).stop 2024-09-10T10:44:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.110+0000 7fc245561640 1 -- 172.21.15.7:0/3200632121 shutdown_connections 2024-09-10T10:44:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.110+0000 7fc245561640 1 --2- 172.21.15.7:0/3200632121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc24010c170 0x7fc24010e610 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.110+0000 7fc245561640 1 --2- 172.21.15.7:0/3200632121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc240103660 0x7fc24010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.110+0000 7fc245561640 1 --2- 172.21.15.7:0/3200632121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc240102d40 0x7fc240103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.110+0000 7fc245561640 1 -- 172.21.15.7:0/3200632121 >> 172.21.15.7:0/3200632121 conn(0x7fc2400fc7a0 msgr2=0x7fc2400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.110+0000 7fc245561640 1 -- 172.21.15.7:0/3200632121 shutdown_connections 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.110+0000 7fc245561640 1 -- 172.21.15.7:0/3200632121 wait complete. 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.110+0000 7fc245561640 1 Processor -- start 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.110+0000 7fc245561640 1 -- start start 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc245561640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc240102d40 0x7fc24019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc245561640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc240103660 0x7fc24019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc245561640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc24010c170 0x7fc24019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc245561640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc240111f70 con 0x7fc240102d40 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc245561640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc240111df0 con 0x7fc24010c170 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc245561640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2401120f0 con 0x7fc240103660 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc23effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc240102d40 0x7fc24019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc23effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc240102d40 0x7fc24019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58064/0 (socket says 172.21.15.7:58064) 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc23effd640 1 -- 172.21.15.7:0/3231462377 learned_addr learned my addr 172.21.15.7:0/3231462377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc2367fc640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc240103660 0x7fc24019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc23f7fe640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc24010c170 0x7fc24019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc23effd640 1 -- 172.21.15.7:0/3231462377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc240103660 msgr2=0x7fc24019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc23effd640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc240103660 0x7fc24019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc23effd640 1 -- 172.21.15.7:0/3231462377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc24010c170 msgr2=0x7fc24019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc23effd640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc24010c170 0x7fc24019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc23effd640 1 -- 172.21.15.7:0/3231462377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24019f5d0 con 0x7fc240102d40 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc2367fc640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc240103660 0x7fc24019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.111+0000 7fc23effd640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc240102d40 0x7fc24019a620 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc23000d950 tx=0x7fc23000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.112+0000 7fc23cff9640 1 -- 172.21.15.7:0/3231462377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230017070 con 0x7fc240102d40 2024-09-10T10:44:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.112+0000 7fc245561640 1 -- 172.21.15.7:0/3231462377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24019f8c0 con 0x7fc240102d40 2024-09-10T10:44:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.112+0000 7fc23cff9640 1 -- 172.21.15.7:0/3231462377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc230009c30 con 0x7fc240102d40 2024-09-10T10:44:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.112+0000 7fc23cff9640 1 -- 172.21.15.7:0/3231462377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23000f6b0 con 0x7fc240102d40 2024-09-10T10:44:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.112+0000 7fc245561640 1 -- 172.21.15.7:0/3231462377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2401a71a0 con 0x7fc240102d40 2024-09-10T10:44:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.113+0000 7fc245561640 1 -- 172.21.15.7:0/3231462377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc240103120 con 0x7fc240102d40 2024-09-10T10:44:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.114+0000 7fc23cff9640 1 -- 172.21.15.7:0/3231462377 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc2300109d0 con 0x7fc240102d40 2024-09-10T10:44:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.115+0000 7fc23cff9640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc22c0a2b80 0x7fc22c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.115+0000 7fc2367fc640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc22c0a2b80 0x7fc22c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.115+0000 7fc23cff9640 1 -- 172.21.15.7:0/3231462377 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fc2300ce980 con 0x7fc240102d40 2024-09-10T10:44:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.115+0000 7fc2367fc640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc22c0a2b80 0x7fc22c0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc2401029c0 tx=0x7fc228005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:05.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.119+0000 7fc23cff9640 1 -- 172.21.15.7:0/3231462377 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc230009de0 con 0x7fc240102d40 2024-09-10T10:44:05.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.307+0000 7fc245561640 1 -- 172.21.15.7:0/3231462377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc2401a7490 con 0x7fc240102d40 2024-09-10T10:44:05.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.308+0000 7fc23cff9640 1 -- 172.21.15.7:0/3231462377 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7fc23009bce0 con 0x7fc240102d40 2024-09-10T10:44:05.309 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:05.310 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T10:44:05.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T10:44:05.310 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-10T10:44:05.310 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-10T10:44:05.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T10:44:05.310 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T10:44:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.310+0000 7fc245561640 1 -- 172.21.15.7:0/3231462377 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc22c0a2b80 msgr2=0x7fc22c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.310+0000 7fc245561640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc22c0a2b80 0x7fc22c0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc2401029c0 tx=0x7fc228005f90 comp rx=0 tx=0).stop 2024-09-10T10:44:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.310+0000 7fc245561640 1 -- 172.21.15.7:0/3231462377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc240102d40 msgr2=0x7fc24019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.310+0000 7fc245561640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc240102d40 0x7fc24019a620 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc23000d950 tx=0x7fc23000de10 comp rx=0 tx=0).stop 2024-09-10T10:44:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.310+0000 7fc245561640 1 -- 172.21.15.7:0/3231462377 shutdown_connections 2024-09-10T10:44:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.310+0000 7fc245561640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc22c0a2b80 0x7fc22c0a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.310+0000 7fc245561640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc24010c170 0x7fc24019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.310+0000 7fc245561640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc240103660 0x7fc24019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.310+0000 7fc245561640 1 --2- 172.21.15.7:0/3231462377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc240102d40 0x7fc24019a620 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.310+0000 7fc245561640 1 -- 172.21.15.7:0/3231462377 >> 172.21.15.7:0/3231462377 conn(0x7fc2400fc7a0 msgr2=0x7fc2400fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:05.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.311+0000 7fc245561640 1 -- 172.21.15.7:0/3231462377 shutdown_connections 2024-09-10T10:44:05.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:05.311+0000 7fc245561640 1 -- 172.21.15.7:0/3231462377 wait complete. 2024-09-10T10:44:06.089 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:44:06.089 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:44:06.350 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:06.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:06.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3231462377' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T10:44:06.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:44:06.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:44:06.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:05 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3231462377' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:44:06.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:05 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:06.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3231462377' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T10:44:06.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:44:06.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:44:06.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:05 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:44:06.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.824+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3099110281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c4106f10 msgr2=0x7ff8c410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.824+0000 7ff8cb1f5640 1 --2- 172.21.15.7:0/3099110281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c4106f10 0x7ff8c410ea90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009ca0 tx=0x7ff8bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.824+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3099110281 shutdown_connections 2024-09-10T10:44:06.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.824+0000 7ff8cb1f5640 1 --2- 172.21.15.7:0/3099110281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c4106f10 0x7ff8c410ea90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.824+0000 7ff8cb1f5640 1 --2- 172.21.15.7:0/3099110281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c4102790 0x7ff8c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.824+0000 7ff8cb1f5640 1 --2- 172.21.15.7:0/3099110281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4101de0 0x7ff8c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.824+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3099110281 >> 172.21.15.7:0/3099110281 conn(0x7ff8c40fd8c0 msgr2=0x7ff8c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:06.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.824+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3099110281 shutdown_connections 2024-09-10T10:44:06.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.824+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3099110281 wait complete. 2024-09-10T10:44:06.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8cb1f5640 1 Processor -- start 2024-09-10T10:44:06.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8cb1f5640 1 -- start start 2024-09-10T10:44:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8cb1f5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c4101de0 0x7ff8c4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8cb1f5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4102790 0x7ff8c41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8cb1f5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c4106f10 0x7ff8c419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8cb1f5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c4111f90 con 0x7ff8c4101de0 2024-09-10T10:44:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8cb1f5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c4111e10 con 0x7ff8c4106f10 2024-09-10T10:44:06.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8cb1f5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c4112110 con 0x7ff8c4102790 2024-09-10T10:44:06.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8c8f91640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c4101de0 0x7ff8c4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8c8f91640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c4101de0 0x7ff8c4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58080/0 (socket says 172.21.15.7:58080) 2024-09-10T10:44:06.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.825+0000 7ff8c8f91640 1 -- 172.21.15.7:0/3896545624 learned_addr learned my addr 172.21.15.7:0/3896545624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:06.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8c9792640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c4106f10 0x7ff8c419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8c8f91640 1 -- 172.21.15.7:0/3896545624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4102790 msgr2=0x7ff8c41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8bbfff640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4102790 0x7ff8c41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8c8f91640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4102790 0x7ff8c41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8c8f91640 1 -- 172.21.15.7:0/3896545624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c4106f10 msgr2=0x7ff8c419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8c8f91640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c4106f10 0x7ff8c419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8c8f91640 1 -- 172.21.15.7:0/3896545624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c419d420 con 0x7ff8c4101de0 2024-09-10T10:44:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8c9792640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c4106f10 0x7ff8c419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8bbfff640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4102790 0x7ff8c41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8c8f91640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c4101de0 0x7ff8c4198470 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b4a0 tx=0x7ff8c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8b9ffb640 1 -- 172.21.15.7:0/3896545624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0014070 con 0x7ff8c4101de0 2024-09-10T10:44:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8b9ffb640 1 -- 172.21.15.7:0/3896545624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8c0010af0 con 0x7ff8c4101de0 2024-09-10T10:44:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3896545624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c419d710 con 0x7ff8c4101de0 2024-09-10T10:44:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8b9ffb640 1 -- 172.21.15.7:0/3896545624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0017d10 con 0x7ff8c4101de0 2024-09-10T10:44:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.826+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3896545624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c41a4ff0 con 0x7ff8c4101de0 2024-09-10T10:44:06.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.828+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3896545624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c41021c0 con 0x7ff8c4101de0 2024-09-10T10:44:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.828+0000 7ff8b9ffb640 1 -- 172.21.15.7:0/3896545624 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff8c0017070 con 0x7ff8c4101de0 2024-09-10T10:44:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.829+0000 7ff8b9ffb640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff8ac0a2b80 0x7ff8ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.829+0000 7ff8bbfff640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff8ac0a2b80 0x7ff8ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.829+0000 7ff8b9ffb640 1 -- 172.21.15.7:0/3896545624 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7ff8c00ce020 con 0x7ff8c4101de0 2024-09-10T10:44:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.830+0000 7ff8bbfff640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff8ac0a2b80 0x7ff8ac0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff8c41088e0 tx=0x7ff8b4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:06.833+0000 7ff8b9ffb640 1 -- 172.21.15.7:0/3896545624 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8c009b420 con 0x7ff8c4101de0 2024-09-10T10:44:07.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:07.011+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3896545624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff8c4109b60 con 0x7ff8c4101de0 2024-09-10T10:44:08.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:07.999+0000 7ff8b9ffb640 1 -- 172.21.15.7:0/3896545624 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7ff8c009ad20 con 0x7ff8c4101de0 2024-09-10T10:44:08.000 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-10T10:44:08.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.002+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3896545624 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff8ac0a2b80 msgr2=0x7ff8ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.002+0000 7ff8cb1f5640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff8ac0a2b80 0x7ff8ac0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff8c41088e0 tx=0x7ff8b4005d30 comp rx=0 tx=0).stop 2024-09-10T10:44:08.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.002+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3896545624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c4101de0 msgr2=0x7ff8c4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.002+0000 7ff8cb1f5640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c4101de0 0x7ff8c4198470 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b4a0 tx=0x7ff8c000b960 comp rx=0 tx=0).stop 2024-09-10T10:44:08.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.002+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3896545624 shutdown_connections 2024-09-10T10:44:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.002+0000 7ff8cb1f5640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff8ac0a2b80 0x7ff8ac0a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.002+0000 7ff8cb1f5640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff8c4106f10 0x7ff8c419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.002+0000 7ff8cb1f5640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4102790 0x7ff8c41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.002+0000 7ff8cb1f5640 1 --2- 172.21.15.7:0/3896545624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c4101de0 0x7ff8c4198470 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.003+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3896545624 >> 172.21.15.7:0/3896545624 conn(0x7ff8c40fd8c0 msgr2=0x7ff8c40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.003+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3896545624 shutdown_connections 2024-09-10T10:44:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:08.003+0000 7ff8cb1f5640 1 -- 172.21.15.7:0/3896545624 wait complete. 2024-09-10T10:44:08.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:07 smithi007 ceph-mon[30873]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:08.316 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:07 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3896545624' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T10:44:08.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:07 smithi103 ceph-mon[35160]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:08.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:07 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3896545624' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T10:44:08.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:07 smithi133 ceph-mon[35282]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:08.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:07 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3896545624' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T10:44:08.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:44:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:09.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:09 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3896545624' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T10:44:09.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:09 smithi007 ceph-mon[30873]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T10:44:09.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.349+0000 7f6823bb1640 1 -- 172.21.15.7:0/568077265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f681c101de0 msgr2=0x7f681c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.349+0000 7f6823bb1640 1 --2- 172.21.15.7:0/568077265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f681c101de0 0x7f681c1021c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6814009900 tx=0x7f681402a410 comp rx=0 tx=0).stop 2024-09-10T10:44:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.349+0000 7f6823bb1640 1 -- 172.21.15.7:0/568077265 shutdown_connections 2024-09-10T10:44:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.349+0000 7f6823bb1640 1 --2- 172.21.15.7:0/568077265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f681c106f10 0x7f681c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.349+0000 7f6823bb1640 1 --2- 172.21.15.7:0/568077265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c102790 0x7f681c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.349+0000 7f6823bb1640 1 --2- 172.21.15.7:0/568077265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f681c101de0 0x7f681c1021c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.349+0000 7f6823bb1640 1 -- 172.21.15.7:0/568077265 >> 172.21.15.7:0/568077265 conn(0x7f681c0fd8c0 msgr2=0x7f681c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f6823bb1640 1 -- 172.21.15.7:0/568077265 shutdown_connections 2024-09-10T10:44:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f6823bb1640 1 -- 172.21.15.7:0/568077265 wait complete. 2024-09-10T10:44:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f6823bb1640 1 Processor -- start 2024-09-10T10:44:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f6823bb1640 1 -- start start 2024-09-10T10:44:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f6823bb1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f681c101de0 0x7f681c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f6823bb1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f681c102790 0x7f681c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f6823bb1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c106f10 0x7f681c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f6823bb1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f681c111f70 con 0x7f681c101de0 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f6823bb1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f681c111df0 con 0x7f681c102790 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f6823bb1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f681c1120f0 con 0x7f681c106f10 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.350+0000 7f682194d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f681c101de0 0x7f681c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682194d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f681c101de0 0x7f681c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58092/0 (socket says 172.21.15.7:58092) 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682194d640 1 -- 172.21.15.7:0/2258187231 learned_addr learned my addr 172.21.15.7:0/2258187231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682114c640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f681c102790 0x7f681c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682214e640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c106f10 0x7f681c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682194d640 1 -- 172.21.15.7:0/2258187231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c106f10 msgr2=0x7f681c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682194d640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c106f10 0x7f681c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682194d640 1 -- 172.21.15.7:0/2258187231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f681c102790 msgr2=0x7f681c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682194d640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f681c102790 0x7f681c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682194d640 1 -- 172.21.15.7:0/2258187231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681c19d430 con 0x7f681c101de0 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682214e640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c106f10 0x7f681c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f682194d640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f681c101de0 0x7f681c198480 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6814006010 tx=0x7f68140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f681affd640 1 -- 172.21.15.7:0/2258187231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681403f030 con 0x7f681c101de0 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f681affd640 1 -- 172.21.15.7:0/2258187231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6814003ce0 con 0x7f681c101de0 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f6823bb1640 1 -- 172.21.15.7:0/2258187231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681c19d6c0 con 0x7f681c101de0 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f681affd640 1 -- 172.21.15.7:0/2258187231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814035070 con 0x7f681c101de0 2024-09-10T10:44:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.351+0000 7f6823bb1640 1 -- 172.21.15.7:0/2258187231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681c1a5000 con 0x7f681c101de0 2024-09-10T10:44:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.352+0000 7f6823bb1640 1 -- 172.21.15.7:0/2258187231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f681c1032c0 con 0x7f681c101de0 2024-09-10T10:44:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.354+0000 7f681affd640 1 -- 172.21.15.7:0/2258187231 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6814042030 con 0x7f681c101de0 2024-09-10T10:44:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.354+0000 7f681affd640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f68040a2b80 0x7f68040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.354+0000 7f682114c640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f68040a2b80 0x7f68040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.354+0000 7f681affd640 1 -- 172.21.15.7:0/2258187231 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f68140ef6b0 con 0x7f681c101de0 2024-09-10T10:44:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.355+0000 7f682114c640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f68040a2b80 0x7f68040a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f681c0fdd70 tx=0x7f681000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.358+0000 7f681affd640 1 -- 172.21.15.7:0/2258187231 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68140bca00 con 0x7f681c101de0 2024-09-10T10:44:09.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:09 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3896545624' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T10:44:09.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:09 smithi103 ceph-mon[35160]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T10:44:09.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:09 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3896545624' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T10:44:09.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:09 smithi133 ceph-mon[35282]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T10:44:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:09.536+0000 7f6823bb1640 1 -- 172.21.15.7:0/2258187231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f681c10a0f0 con 0x7f681c101de0 2024-09-10T10:44:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.012+0000 7f681affd640 1 -- 172.21.15.7:0/2258187231 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7f68140bc300 con 0x7f681c101de0 2024-09-10T10:44:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-10T10:44:10.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 -- 172.21.15.7:0/2258187231 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f68040a2b80 msgr2=0x7f68040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f68040a2b80 0x7f68040a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f681c0fdd70 tx=0x7f681000b390 comp rx=0 tx=0).stop 2024-09-10T10:44:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 -- 172.21.15.7:0/2258187231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f681c101de0 msgr2=0x7f681c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f681c101de0 0x7f681c198480 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6814006010 tx=0x7f68140042f0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 -- 172.21.15.7:0/2258187231 shutdown_connections 2024-09-10T10:44:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f68040a2b80 0x7f68040a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f681c106f10 0x7f681c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f681c102790 0x7f681c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 --2- 172.21.15.7:0/2258187231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f681c101de0 0x7f681c198480 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 -- 172.21.15.7:0/2258187231 >> 172.21.15.7:0/2258187231 conn(0x7f681c0fd8c0 msgr2=0x7f681c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.014+0000 7f6823bb1640 1 -- 172.21.15.7:0/2258187231 shutdown_connections 2024-09-10T10:44:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:10.015+0000 7f6823bb1640 1 -- 172.21.15.7:0/2258187231 wait complete. 2024-09-10T10:44:10.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:10 smithi007 ceph-mon[30873]: pgmap v269: 33 pgs: 11 creating+peering, 21 unknown, 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:10.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:10 smithi007 ceph-mon[30873]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T10:44:10.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:10 smithi007 ceph-mon[30873]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:44:10.315 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:10 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2258187231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T10:44:10.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:10 smithi103 ceph-mon[35160]: pgmap v269: 33 pgs: 11 creating+peering, 21 unknown, 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:10.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:10 smithi103 ceph-mon[35160]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T10:44:10.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:10 smithi103 ceph-mon[35160]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:44:10.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:10 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2258187231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T10:44:10.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:10 smithi133 ceph-mon[35282]: pgmap v269: 33 pgs: 11 creating+peering, 21 unknown, 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:10.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:10 smithi133 ceph-mon[35282]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T10:44:10.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:10 smithi133 ceph-mon[35282]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:44:10.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:10 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2258187231' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T10:44:10.645 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:44:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:11.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:11 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2258187231' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T10:44:11.320 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:11 smithi007 ceph-mon[30873]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T10:44:11.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.327+0000 7fee4112f640 1 -- 172.21.15.7:0/371318136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3c101240 msgr2=0x7fee3c101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.327+0000 7fee4112f640 1 --2- 172.21.15.7:0/371318136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3c101240 0x7fee3c101680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fee30009900 tx=0x7fee3002a410 comp rx=0 tx=0).stop 2024-09-10T10:44:11.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.327+0000 7fee4112f640 1 -- 172.21.15.7:0/371318136 shutdown_connections 2024-09-10T10:44:11.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.327+0000 7fee4112f640 1 --2- 172.21.15.7:0/371318136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3c101bc0 0x7fee3c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.327+0000 7fee4112f640 1 --2- 172.21.15.7:0/371318136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3c101240 0x7fee3c101680 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.327+0000 7fee4112f640 1 --2- 172.21.15.7:0/371318136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee3c105a90 0x7fee3c105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.327+0000 7fee4112f640 1 -- 172.21.15.7:0/371318136 >> 172.21.15.7:0/371318136 conn(0x7fee3c077210 msgr2=0x7fee3c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.327+0000 7fee4112f640 1 -- 172.21.15.7:0/371318136 shutdown_connections 2024-09-10T10:44:11.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.327+0000 7fee4112f640 1 -- 172.21.15.7:0/371318136 wait complete. 2024-09-10T10:44:11.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.328+0000 7fee4112f640 1 Processor -- start 2024-09-10T10:44:11.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.328+0000 7fee4112f640 1 -- start start 2024-09-10T10:44:11.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee4112f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3c101240 0x7fee3c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee4112f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3c101bc0 0x7fee3c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee4112f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee3c105a90 0x7fee3c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee4112f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c110fb0 con 0x7fee3c101240 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee4112f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c110e30 con 0x7fee3c105a90 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee4112f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c111130 con 0x7fee3c101bc0 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3c101240 0x7fee3c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3c101240 0x7fee3c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45414/0 (socket says 172.21.15.7:45414) 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3ad9d640 1 -- 172.21.15.7:0/2648031911 learned_addr learned my addr 172.21.15.7:0/2648031911 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3a59c640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3c101bc0 0x7fee3c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3b59e640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee3c105a90 0x7fee3c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3ad9d640 1 -- 172.21.15.7:0/2648031911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3c101bc0 msgr2=0x7fee3c19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3ad9d640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3c101bc0 0x7fee3c19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3ad9d640 1 -- 172.21.15.7:0/2648031911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee3c105a90 msgr2=0x7fee3c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3ad9d640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee3c105a90 0x7fee3c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3ad9d640 1 -- 172.21.15.7:0/2648031911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c19f680 con 0x7fee3c101240 2024-09-10T10:44:11.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3a59c640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3c101bc0 0x7fee3c19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:11.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3b59e640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee3c105a90 0x7fee3c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:11.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee3ad9d640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3c101240 0x7fee3c19a6d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00d950 tx=0x7fee2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.329+0000 7fee23fff640 1 -- 172.21.15.7:0/2648031911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c017070 con 0x7fee3c101240 2024-09-10T10:44:11.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.330+0000 7fee23fff640 1 -- 172.21.15.7:0/2648031911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee2c009c30 con 0x7fee3c101240 2024-09-10T10:44:11.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.330+0000 7fee4112f640 1 -- 172.21.15.7:0/2648031911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3c19f970 con 0x7fee3c101240 2024-09-10T10:44:11.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.330+0000 7fee23fff640 1 -- 172.21.15.7:0/2648031911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00f6b0 con 0x7fee3c101240 2024-09-10T10:44:11.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.330+0000 7fee4112f640 1 -- 172.21.15.7:0/2648031911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3c1a7250 con 0x7fee3c101240 2024-09-10T10:44:11.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.331+0000 7fee4112f640 1 -- 172.21.15.7:0/2648031911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3c101680 con 0x7fee3c101240 2024-09-10T10:44:11.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.332+0000 7fee23fff640 1 -- 172.21.15.7:0/2648031911 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fee2c0109d0 con 0x7fee3c101240 2024-09-10T10:44:11.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.332+0000 7fee23fff640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fee240a2b80 0x7fee240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.333+0000 7fee3a59c640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fee240a2b80 0x7fee240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.333+0000 7fee23fff640 1 -- 172.21.15.7:0/2648031911 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7fee2c0ceee0 con 0x7fee3c101240 2024-09-10T10:44:11.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.333+0000 7fee3a59c640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fee240a2b80 0x7fee240a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fee3c105700 tx=0x7fee30005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.337+0000 7fee23fff640 1 -- 172.21.15.7:0/2648031911 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee2c009de0 con 0x7fee3c101240 2024-09-10T10:44:11.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:11 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2258187231' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T10:44:11.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:11 smithi103 ceph-mon[35160]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T10:44:11.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:11 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2258187231' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T10:44:11.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:11 smithi133 ceph-mon[35282]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T10:44:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:11.546+0000 7fee4112f640 1 -- 172.21.15.7:0/2648031911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fee3c0fd740 con 0x7fee3c101240 2024-09-10T10:44:12.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.018+0000 7fee23fff640 1 -- 172.21.15.7:0/2648031911 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fee2c09bff0 con 0x7fee3c101240 2024-09-10T10:44:12.021 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T10:44:12.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 -- 172.21.15.7:0/2648031911 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fee240a2b80 msgr2=0x7fee240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fee240a2b80 0x7fee240a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fee3c105700 tx=0x7fee30005a20 comp rx=0 tx=0).stop 2024-09-10T10:44:12.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 -- 172.21.15.7:0/2648031911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3c101240 msgr2=0x7fee3c19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3c101240 0x7fee3c19a6d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00d950 tx=0x7fee2c00de10 comp rx=0 tx=0).stop 2024-09-10T10:44:12.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 -- 172.21.15.7:0/2648031911 shutdown_connections 2024-09-10T10:44:12.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fee240a2b80 0x7fee240a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee3c105a90 0x7fee3c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee3c101bc0 0x7fee3c19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 --2- 172.21.15.7:0/2648031911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee3c101240 0x7fee3c19a6d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 -- 172.21.15.7:0/2648031911 >> 172.21.15.7:0/2648031911 conn(0x7fee3c077210 msgr2=0x7fee3c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 -- 172.21.15.7:0/2648031911 shutdown_connections 2024-09-10T10:44:12.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:12.022+0000 7fee4112f640 1 -- 172.21.15.7:0/2648031911 wait complete. 2024-09-10T10:44:12.332 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:12 smithi007 ceph-mon[30873]: pgmap v272: 97 pgs: 14 active+clean, 2 creating+activating, 11 creating+peering, 70 unknown; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:12.332 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:12 smithi007 ceph-mon[30873]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T10:44:12.332 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:12 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2648031911' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T10:44:12.332 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:12 smithi007 ceph-mon[30873]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-10T10:44:12.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:12 smithi103 ceph-mon[35160]: pgmap v272: 97 pgs: 14 active+clean, 2 creating+activating, 11 creating+peering, 70 unknown; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:12.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:12 smithi103 ceph-mon[35160]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T10:44:12.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:12 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2648031911' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T10:44:12.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:12 smithi103 ceph-mon[35160]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-10T10:44:12.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:12 smithi133 ceph-mon[35282]: pgmap v272: 97 pgs: 14 active+clean, 2 creating+activating, 11 creating+peering, 70 unknown; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:44:12.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:12 smithi133 ceph-mon[35282]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T10:44:12.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:12 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2648031911' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T10:44:12.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:12 smithi133 ceph-mon[35282]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-10T10:44:12.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:44:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:13.277 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:13 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2648031911' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T10:44:13.277 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:13 smithi007 ceph-mon[30873]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T10:44:13.278 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:13 smithi007 ceph-mon[30873]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-10T10:44:13.278 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:13 smithi007 ceph-mon[30873]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-10T10:44:13.326 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:13 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2648031911' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T10:44:13.326 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:13 smithi103 ceph-mon[35160]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T10:44:13.326 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:13 smithi103 ceph-mon[35160]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-10T10:44:13.326 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:13 smithi103 ceph-mon[35160]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-10T10:44:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.356+0000 7fd7d1185640 1 -- 172.21.15.7:0/2604230779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7cc102ef0 msgr2=0x7fd7cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.356+0000 7fd7d1185640 1 --2- 172.21.15.7:0/2604230779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7cc102ef0 0x7fd7cc10da40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009ca0 tx=0x7fd7c402a950 comp rx=0 tx=0).stop 2024-09-10T10:44:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.356+0000 7fd7d1185640 1 -- 172.21.15.7:0/2604230779 shutdown_connections 2024-09-10T10:44:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.356+0000 7fd7d1185640 1 --2- 172.21.15.7:0/2604230779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7cc102ef0 0x7fd7cc10da40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.356+0000 7fd7d1185640 1 --2- 172.21.15.7:0/2604230779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7cc102570 0x7fd7cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.356+0000 7fd7d1185640 1 --2- 172.21.15.7:0/2604230779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc100e20 0x7fd7cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.356+0000 7fd7d1185640 1 -- 172.21.15.7:0/2604230779 >> 172.21.15.7:0/2604230779 conn(0x7fd7cc0fc7d0 msgr2=0x7fd7cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.356+0000 7fd7d1185640 1 -- 172.21.15.7:0/2604230779 shutdown_connections 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.356+0000 7fd7d1185640 1 -- 172.21.15.7:0/2604230779 wait complete. 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.357+0000 7fd7d1185640 1 Processor -- start 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.357+0000 7fd7d1185640 1 -- start start 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.357+0000 7fd7d1185640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc100e20 0x7fd7cc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.357+0000 7fd7d1185640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7cc102570 0x7fd7cc19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.357+0000 7fd7d1185640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7cc102ef0 0x7fd7cc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.357+0000 7fd7d1185640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc110f80 con 0x7fd7cc102ef0 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.357+0000 7fd7d1185640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc110e00 con 0x7fd7cc102570 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.357+0000 7fd7d1185640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd7cc111100 con 0x7fd7cc100e20 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7cb59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7cc102ef0 0x7fd7cc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7cb59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7cc102ef0 0x7fd7cc19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45434/0 (socket says 172.21.15.7:45434) 2024-09-10T10:44:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7cb59e640 1 -- 172.21.15.7:0/3975275233 learned_addr learned my addr 172.21.15.7:0/3975275233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7cad9d640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc100e20 0x7fd7cc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7ca59c640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7cc102570 0x7fd7cc19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7cb59e640 1 -- 172.21.15.7:0/3975275233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc100e20 msgr2=0x7fd7cc19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7cb59e640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc100e20 0x7fd7cc19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7cb59e640 1 -- 172.21.15.7:0/3975275233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7cc102570 msgr2=0x7fd7cc19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7cb59e640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7cc102570 0x7fd7cc19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7cb59e640 1 -- 172.21.15.7:0/3975275233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7cc19f680 con 0x7fd7cc102ef0 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7cb59e640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7cc102ef0 0x7fd7cc19efa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4004af0 tx=0x7fd7c402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7b3fff640 1 -- 172.21.15.7:0/3975275233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4034040 con 0x7fd7cc102ef0 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7d1185640 1 -- 172.21.15.7:0/3975275233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7cc19f910 con 0x7fd7cc102ef0 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7b3fff640 1 -- 172.21.15.7:0/3975275233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c4030920 con 0x7fd7cc102ef0 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7b3fff640 1 -- 172.21.15.7:0/3975275233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4004820 con 0x7fd7cc102ef0 2024-09-10T10:44:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.358+0000 7fd7d1185640 1 -- 172.21.15.7:0/3975275233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7cc1a7250 con 0x7fd7cc102ef0 2024-09-10T10:44:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.360+0000 7fd7d1185640 1 -- 172.21.15.7:0/3975275233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7cc101200 con 0x7fd7cc102ef0 2024-09-10T10:44:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.361+0000 7fd7b3fff640 1 -- 172.21.15.7:0/3975275233 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd7c4037020 con 0x7fd7cc102ef0 2024-09-10T10:44:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.362+0000 7fd7b3fff640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd7b40a2b80 0x7fd7b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.362+0000 7fd7cad9d640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd7b40a2b80 0x7fd7b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.362+0000 7fd7b3fff640 1 -- 172.21.15.7:0/3975275233 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fd7c40efdc0 con 0x7fd7cc102ef0 2024-09-10T10:44:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.362+0000 7fd7cad9d640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd7b40a2b80 0x7fd7b40a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00bbe0 tx=0x7fd7bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:13.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.366+0000 7fd7b3fff640 1 -- 172.21.15.7:0/3975275233 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7c40bcdd0 con 0x7fd7cc102ef0 2024-09-10T10:44:13.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:13 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2648031911' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T10:44:13.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:13 smithi133 ceph-mon[35282]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T10:44:13.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:13 smithi133 ceph-mon[35282]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-10T10:44:13.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:13 smithi133 ceph-mon[35282]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-10T10:44:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.541+0000 7fd7d1185640 1 -- 172.21.15.7:0/3975275233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7cc1a7420 con 0x7fd7cc102ef0 2024-09-10T10:44:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.543+0000 7fd7b3fff640 1 -- 172.21.15.7:0/3975275233 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22195 (secure 0 0 0) 0x7fd7c40a70e0 con 0x7fd7cc102ef0 2024-09-10T10:44:13.546 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:13.546 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:36:33.117443+0000","modified":"2024-09-10T10:44:11.548991+0000","last_up_change":"2024-09-10T10:43:23.913978+0000","last_in_change":"2024-09-10T10:42:59.733844+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:39:37.104114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:07.441548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:09.970273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9239e99-3a62-4883-b2f1-8964ebb54492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6803","nonce":704452390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6805","nonce":704452390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6809","nonce":704452390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6807","nonce":704452390}]},"public_addr":"172.21.15.7:6803/704452390","cluster_addr":"172.21.15.7:6805/704452390","heartbeat_back_addr":"172.21.15.7:6809/704452390","heartbeat_front_addr":"172.21.15.7:6807/704452390","state":["exists","up"]},{"osd":1,"uuid":"d750ff16-48c8-421d-b46b-49f0d2b7f4e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6801","nonce":2409158189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6803","nonce":2409158189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6807","nonce":2409158189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6805","nonce":2409158189}]},"public_addr":"172.21.15.103:6801/2409158189","cluster_addr":"172.21.15.103:6803/2409158189","heartbeat_back_addr":"172.21.15.103:6807/2409158189","heartbeat_front_addr":"172.21.15.103:6805/2409158189","state":["exists","up"]},{"osd":2,"uuid":"1ecb070a-a2f1-499a-99e7-f1967ebb52ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6801","nonce":1657349858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6803","nonce":1657349858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6807","nonce":1657349858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6805","nonce":1657349858}]},"public_addr":"172.21.15.133:6801/1657349858","cluster_addr":"172.21.15.133:6803/1657349858","heartbeat_back_addr":"172.21.15.133:6807/1657349858","heartbeat_front_addr":"172.21.15.133:6805/1657349858","state":["exists","up"]},{"osd":3,"uuid":"8e53d79e-038f-4a7f-9ac9-16b60f47bb4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6811","nonce":871781144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6813","nonce":871781144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6817","nonce":871781144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6815","nonce":871781144}]},"public_addr":"172.21.15.7:6811/871781144","cluster_addr":"172.21.15.7:6813/871781144","heartbeat_back_addr":"172.21.15.7:6817/871781144","heartbeat_front_addr":"172.21.15.7:6815/871781144","state":["exists","up"]},{"osd":4,"uuid":"e4f04c13-594c-43dd-9a98-d2138061b1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6809","nonce":1937243043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6811","nonce":1937243043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6815","nonce":1937243043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6813","nonce":1937243043}]},"public_addr":"172.21.15.103:6809/1937243043","cluster_addr":"172.21.15.103:6811/1937243043","heartbeat_back_addr":"172.21.15.103:6815/1937243043","heartbeat_front_addr":"172.21.15.103:6813/1937243043","state":["exists","up"]},{"osd":5,"uuid":"73143170-5816-407d-ae03-dcf896057681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6809","nonce":577882774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6811","nonce":577882774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6815","nonce":577882774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6813","nonce":577882774}]},"public_addr":"172.21.15.133:6809/577882774","cluster_addr":"172.21.15.133:6811/577882774","heartbeat_back_addr":"172.21.15.133:6815/577882774","heartbeat_front_addr":"172.21.15.133:6813/577882774","state":["exists","up"]},{"osd":6,"uuid":"3a5904d5-36de-47b0-97fa-5a10b24d0c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6819","nonce":2646192734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6821","nonce":2646192734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6825","nonce":2646192734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6823","nonce":2646192734}]},"public_addr":"172.21.15.7:6819/2646192734","cluster_addr":"172.21.15.7:6821/2646192734","heartbeat_back_addr":"172.21.15.7:6825/2646192734","heartbeat_front_addr":"172.21.15.7:6823/2646192734","state":["exists","up"]},{"osd":7,"uuid":"91e79d42-decd-4405-b9f2-9480f7edfca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6817","nonce":3858867236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6819","nonce":3858867236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6823","nonce":3858867236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6821","nonce":3858867236}]},"public_addr":"172.21.15.103:6817/3858867236","cluster_addr":"172.21.15.103:6819/3858867236","heartbeat_back_addr":"172.21.15.103:6823/3858867236","heartbeat_front_addr":"172.21.15.103:6821/3858867236","state":["exists","up"]},{"osd":8,"uuid":"64373754-e81a-430c-967d-4b4f289f3473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6817","nonce":131428321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6819","nonce":131428321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6823","nonce":131428321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6821","nonce":131428321}]},"public_addr":"172.21.15.133:6817/131428321","cluster_addr":"172.21.15.133:6819/131428321","heartbeat_back_addr":"172.21.15.133:6823/131428321","heartbeat_front_addr":"172.21.15.133:6821/131428321","state":["exists","up"]},{"osd":9,"uuid":"91ebe2c9-bda3-4596-ac76-5678c71f850b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6827","nonce":945633240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6829","nonce":945633240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6833","nonce":945633240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6831","nonce":945633240}]},"public_addr":"172.21.15.7:6827/945633240","cluster_addr":"172.21.15.7:6829/945633240","heartbeat_back_addr":"172.21.15.7:6833/945633240","heartbeat_front_addr":"172.21.15.7:6831/945633240","state":["exists","up"]},{"osd":10,"uuid":"2f65568f-89d1-4a67-a222-8e263d34c710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6825","nonce":907835659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6827","nonce":907835659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6831","nonce":907835659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6829","nonce":907835659}]},"public_addr":"172.21.15.103:6825/907835659","cluster_addr":"172.21.15.103:6827/907835659","heartbeat_back_addr":"172.21.15.103:6831/907835659","heartbeat_front_addr":"172.21.15.103:6829/907835659","state":["exists","up"]},{"osd":11,"uuid":"4b49839b-32e9-4105-abee-a253f1aa3f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6825","nonce":2538184038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6827","nonce":2538184038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6831","nonce":2538184038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6829","nonce":2538184038}]},"public_addr":"172.21.15.133:6825/2538184038","cluster_addr":"172.21.15.133:6827/2538184038","heartbeat_back_addr":"172.21.15.133:6831/2538184038","heartbeat_front_addr":"172.21.15.133:6829/2538184038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:44.541279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:10.103733+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:35.290268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:01.070763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:25.666941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:50.925733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:16.486537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:41.464155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:06.610089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:31.994080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:57.000696+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:21.398669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3394895925":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6800/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/262340568":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/3768226338":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/258851660":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3411487770":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/11850255":"2024-09-11T10:36:55.407956+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 -- 172.21.15.7:0/3975275233 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd7b40a2b80 msgr2=0x7fd7b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd7b40a2b80 0x7fd7b40a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00bbe0 tx=0x7fd7bc00b390 comp rx=0 tx=0).stop 2024-09-10T10:44:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 -- 172.21.15.7:0/3975275233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7cc102ef0 msgr2=0x7fd7cc19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7cc102ef0 0x7fd7cc19efa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4004af0 tx=0x7fd7c402f5b0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 -- 172.21.15.7:0/3975275233 shutdown_connections 2024-09-10T10:44:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd7b40a2b80 0x7fd7b40a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7cc102ef0 0x7fd7cc19efa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7cc102570 0x7fd7cc19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 --2- 172.21.15.7:0/3975275233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc100e20 0x7fd7cc19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 -- 172.21.15.7:0/3975275233 >> 172.21.15.7:0/3975275233 conn(0x7fd7cc0fc7d0 msgr2=0x7fd7cc069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 -- 172.21.15.7:0/3975275233 shutdown_connections 2024-09-10T10:44:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:13.546+0000 7fd7d1185640 1 -- 172.21.15.7:0/3975275233 wait complete. 2024-09-10T10:44:13.833 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 10:44:13 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-i[66074]: 2024-09-10T10:44:13.576+0000 7f4689229640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:44:14.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:44:14.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:14 smithi103 ceph-mon[35160]: pgmap v275: 97 pgs: 46 active+clean, 2 creating+activating, 11 creating+peering, 38 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-10T10:44:14.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:14 smithi103 ceph-mon[35160]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T10:44:14.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:14 smithi103 ceph-mon[35160]: Cluster is now healthy 2024-09-10T10:44:14.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:14 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] up:active 2024-09-10T10:44:14.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:14 smithi103 ceph-mon[35160]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:14.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:14 smithi103 ceph-mon[35160]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:14.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:14 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3975275233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:14.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:14 smithi133 ceph-mon[35282]: pgmap v275: 97 pgs: 46 active+clean, 2 creating+activating, 11 creating+peering, 38 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-10T10:44:14.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:14 smithi133 ceph-mon[35282]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T10:44:14.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:14 smithi133 ceph-mon[35282]: Cluster is now healthy 2024-09-10T10:44:14.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:14 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] up:active 2024-09-10T10:44:14.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:14 smithi133 ceph-mon[35282]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:14.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:14 smithi133 ceph-mon[35282]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:14.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:14 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3975275233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:14.342 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:14.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:14 smithi007 ceph-mon[30873]: pgmap v275: 97 pgs: 46 active+clean, 2 creating+activating, 11 creating+peering, 38 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-10T10:44:14.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:14 smithi007 ceph-mon[30873]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T10:44:14.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:14 smithi007 ceph-mon[30873]: Cluster is now healthy 2024-09-10T10:44:14.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:14 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] up:active 2024-09-10T10:44:14.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:14 smithi007 ceph-mon[30873]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:14.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:14 smithi007 ceph-mon[30873]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:14.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:14 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3975275233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.798+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1327784143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4101de0 msgr2=0x7fc5a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.798+0000 7fc5ac05f640 1 --2- 172.21.15.7:0/1327784143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4101de0 0x7fc5a41021c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009900 tx=0x7fc5a002a410 comp rx=0 tx=0).stop 2024-09-10T10:44:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.798+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1327784143 shutdown_connections 2024-09-10T10:44:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.798+0000 7fc5ac05f640 1 --2- 172.21.15.7:0/1327784143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a4106f10 0x7fc5a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.798+0000 7fc5ac05f640 1 --2- 172.21.15.7:0/1327784143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a4102790 0x7fc5a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.798+0000 7fc5ac05f640 1 --2- 172.21.15.7:0/1327784143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4101de0 0x7fc5a41021c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.798+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1327784143 >> 172.21.15.7:0/1327784143 conn(0x7fc5a40fd880 msgr2=0x7fc5a40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.798+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1327784143 shutdown_connections 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.798+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1327784143 wait complete. 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5ac05f640 1 Processor -- start 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5ac05f640 1 -- start start 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5ac05f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a4101de0 0x7fc5a4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5ac05f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a4102790 0x7fc5a4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5ac05f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4106f10 0x7fc5a419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5ac05f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4111f70 con 0x7fc5a4106f10 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5ac05f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4111df0 con 0x7fc5a4102790 2024-09-10T10:44:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5ac05f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41120f0 con 0x7fc5a4101de0 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5aa5fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4106f10 0x7fc5a419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5aa5fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4106f10 0x7fc5a419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45442/0 (socket says 172.21.15.7:45442) 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5aa5fc640 1 -- 172.21.15.7:0/1619817039 learned_addr learned my addr 172.21.15.7:0/1619817039 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5a95fa640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a4102790 0x7fc5a4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5a9dfb640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a4101de0 0x7fc5a4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5aa5fc640 1 -- 172.21.15.7:0/1619817039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a4101de0 msgr2=0x7fc5a4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5aa5fc640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a4101de0 0x7fc5a4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5aa5fc640 1 -- 172.21.15.7:0/1619817039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a4102790 msgr2=0x7fc5a4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5aa5fc640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a4102790 0x7fc5a4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.799+0000 7fc5aa5fc640 1 -- 172.21.15.7:0/1619817039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a419d3c0 con 0x7fc5a4106f10 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.800+0000 7fc5aa5fc640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4106f10 0x7fc5a419cce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00eaa0 tx=0x7fc59c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.800+0000 7fc59affd640 1 -- 172.21.15.7:0/1619817039 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00cdb0 con 0x7fc5a4106f10 2024-09-10T10:44:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.800+0000 7fc59affd640 1 -- 172.21.15.7:0/1619817039 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c005020 con 0x7fc5a4106f10 2024-09-10T10:44:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.800+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1619817039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a419d6b0 con 0x7fc5a4106f10 2024-09-10T10:44:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.800+0000 7fc59affd640 1 -- 172.21.15.7:0/1619817039 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0116b0 con 0x7fc5a4106f10 2024-09-10T10:44:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.800+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1619817039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a41a4f90 con 0x7fc5a4106f10 2024-09-10T10:44:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.801+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1619817039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a41021c0 con 0x7fc5a4106f10 2024-09-10T10:44:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.802+0000 7fc59affd640 1 -- 172.21.15.7:0/1619817039 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc59c005560 con 0x7fc5a4106f10 2024-09-10T10:44:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.802+0000 7fc59affd640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc5900a2c00 0x7fc5900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.803+0000 7fc5a9dfb640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc5900a2c00 0x7fc5900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.803+0000 7fc59affd640 1 -- 172.21.15.7:0/1619817039 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fc59c0cf4a0 con 0x7fc5a4106f10 2024-09-10T10:44:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.803+0000 7fc5a9dfb640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc5900a2c00 0x7fc5900a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00096c0 tx=0x7fc5a0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.806+0000 7fc59affd640 1 -- 172.21.15.7:0/1619817039 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc59c016030 con 0x7fc5a4106f10 2024-09-10T10:44:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.982+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1619817039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5a4109b60 con 0x7fc5a4106f10 2024-09-10T10:44:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.983+0000 7fc59affd640 1 -- 172.21.15.7:0/1619817039 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22195 (secure 0 0 0) 0x7fc59c09c5d0 con 0x7fc5a4106f10 2024-09-10T10:44:14.985 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:14.986 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:36:33.117443+0000","modified":"2024-09-10T10:44:11.548991+0000","last_up_change":"2024-09-10T10:43:23.913978+0000","last_in_change":"2024-09-10T10:42:59.733844+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:39:37.104114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:07.441548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:09.970273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9239e99-3a62-4883-b2f1-8964ebb54492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6803","nonce":704452390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6805","nonce":704452390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6809","nonce":704452390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6807","nonce":704452390}]},"public_addr":"172.21.15.7:6803/704452390","cluster_addr":"172.21.15.7:6805/704452390","heartbeat_back_addr":"172.21.15.7:6809/704452390","heartbeat_front_addr":"172.21.15.7:6807/704452390","state":["exists","up"]},{"osd":1,"uuid":"d750ff16-48c8-421d-b46b-49f0d2b7f4e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6801","nonce":2409158189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6803","nonce":2409158189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6807","nonce":2409158189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6805","nonce":2409158189}]},"public_addr":"172.21.15.103:6801/2409158189","cluster_addr":"172.21.15.103:6803/2409158189","heartbeat_back_addr":"172.21.15.103:6807/2409158189","heartbeat_front_addr":"172.21.15.103:6805/2409158189","state":["exists","up"]},{"osd":2,"uuid":"1ecb070a-a2f1-499a-99e7-f1967ebb52ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6801","nonce":1657349858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6803","nonce":1657349858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6807","nonce":1657349858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6805","nonce":1657349858}]},"public_addr":"172.21.15.133:6801/1657349858","cluster_addr":"172.21.15.133:6803/1657349858","heartbeat_back_addr":"172.21.15.133:6807/1657349858","heartbeat_front_addr":"172.21.15.133:6805/1657349858","state":["exists","up"]},{"osd":3,"uuid":"8e53d79e-038f-4a7f-9ac9-16b60f47bb4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6811","nonce":871781144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6813","nonce":871781144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6817","nonce":871781144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6815","nonce":871781144}]},"public_addr":"172.21.15.7:6811/871781144","cluster_addr":"172.21.15.7:6813/871781144","heartbeat_back_addr":"172.21.15.7:6817/871781144","heartbeat_front_addr":"172.21.15.7:6815/871781144","state":["exists","up"]},{"osd":4,"uuid":"e4f04c13-594c-43dd-9a98-d2138061b1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6809","nonce":1937243043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6811","nonce":1937243043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6815","nonce":1937243043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6813","nonce":1937243043}]},"public_addr":"172.21.15.103:6809/1937243043","cluster_addr":"172.21.15.103:6811/1937243043","heartbeat_back_addr":"172.21.15.103:6815/1937243043","heartbeat_front_addr":"172.21.15.103:6813/1937243043","state":["exists","up"]},{"osd":5,"uuid":"73143170-5816-407d-ae03-dcf896057681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6809","nonce":577882774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6811","nonce":577882774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6815","nonce":577882774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6813","nonce":577882774}]},"public_addr":"172.21.15.133:6809/577882774","cluster_addr":"172.21.15.133:6811/577882774","heartbeat_back_addr":"172.21.15.133:6815/577882774","heartbeat_front_addr":"172.21.15.133:6813/577882774","state":["exists","up"]},{"osd":6,"uuid":"3a5904d5-36de-47b0-97fa-5a10b24d0c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6819","nonce":2646192734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6821","nonce":2646192734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6825","nonce":2646192734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6823","nonce":2646192734}]},"public_addr":"172.21.15.7:6819/2646192734","cluster_addr":"172.21.15.7:6821/2646192734","heartbeat_back_addr":"172.21.15.7:6825/2646192734","heartbeat_front_addr":"172.21.15.7:6823/2646192734","state":["exists","up"]},{"osd":7,"uuid":"91e79d42-decd-4405-b9f2-9480f7edfca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6817","nonce":3858867236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6819","nonce":3858867236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6823","nonce":3858867236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6821","nonce":3858867236}]},"public_addr":"172.21.15.103:6817/3858867236","cluster_addr":"172.21.15.103:6819/3858867236","heartbeat_back_addr":"172.21.15.103:6823/3858867236","heartbeat_front_addr":"172.21.15.103:6821/3858867236","state":["exists","up"]},{"osd":8,"uuid":"64373754-e81a-430c-967d-4b4f289f3473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6817","nonce":131428321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6819","nonce":131428321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6823","nonce":131428321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6821","nonce":131428321}]},"public_addr":"172.21.15.133:6817/131428321","cluster_addr":"172.21.15.133:6819/131428321","heartbeat_back_addr":"172.21.15.133:6823/131428321","heartbeat_front_addr":"172.21.15.133:6821/131428321","state":["exists","up"]},{"osd":9,"uuid":"91ebe2c9-bda3-4596-ac76-5678c71f850b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6827","nonce":945633240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6829","nonce":945633240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6833","nonce":945633240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6831","nonce":945633240}]},"public_addr":"172.21.15.7:6827/945633240","cluster_addr":"172.21.15.7:6829/945633240","heartbeat_back_addr":"172.21.15.7:6833/945633240","heartbeat_front_addr":"172.21.15.7:6831/945633240","state":["exists","up"]},{"osd":10,"uuid":"2f65568f-89d1-4a67-a222-8e263d34c710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6825","nonce":907835659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6827","nonce":907835659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6831","nonce":907835659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6829","nonce":907835659}]},"public_addr":"172.21.15.103:6825/907835659","cluster_addr":"172.21.15.103:6827/907835659","heartbeat_back_addr":"172.21.15.103:6831/907835659","heartbeat_front_addr":"172.21.15.103:6829/907835659","state":["exists","up"]},{"osd":11,"uuid":"4b49839b-32e9-4105-abee-a253f1aa3f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6825","nonce":2538184038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6827","nonce":2538184038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6831","nonce":2538184038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6829","nonce":2538184038}]},"public_addr":"172.21.15.133:6825/2538184038","cluster_addr":"172.21.15.133:6827/2538184038","heartbeat_back_addr":"172.21.15.133:6831/2538184038","heartbeat_front_addr":"172.21.15.133:6829/2538184038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:44.541279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:10.103733+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:35.290268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:01.070763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:25.666941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:50.925733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:16.486537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:41.464155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:06.610089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:31.994080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:57.000696+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:21.398669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3394895925":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6800/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/262340568":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/3768226338":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/258851660":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3411487770":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/11850255":"2024-09-11T10:36:55.407956+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1619817039 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc5900a2c00 msgr2=0x7fc5900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc5900a2c00 0x7fc5900a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00096c0 tx=0x7fc5a0000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1619817039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4106f10 msgr2=0x7fc5a419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4106f10 0x7fc5a419cce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00eaa0 tx=0x7fc59c00ef60 comp rx=0 tx=0).stop 2024-09-10T10:44:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1619817039 shutdown_connections 2024-09-10T10:44:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc5900a2c00 0x7fc5900a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4106f10 0x7fc5a419cce0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a4102790 0x7fc5a4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 --2- 172.21.15.7:0/1619817039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5a4101de0 0x7fc5a4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1619817039 >> 172.21.15.7:0/1619817039 conn(0x7fc5a40fd880 msgr2=0x7fc5a40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1619817039 shutdown_connections 2024-09-10T10:44:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:14.986+0000 7fc5ac05f640 1 -- 172.21.15.7:0/1619817039 wait complete. 2024-09-10T10:44:15.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:15 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1619817039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:15.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:15 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1619817039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:15.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:15 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1619817039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:15.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:44:15.828 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:16.243 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:16 smithi007 ceph-mon[30873]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-10T10:44:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.326+0000 7f7073f99640 1 -- 172.21.15.7:0/72140481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706c103760 msgr2=0x7f706c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.326+0000 7f7073f99640 1 --2- 172.21.15.7:0/72140481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706c103760 0x7f706c107ba0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7060009930 tx=0x7f706002a420 comp rx=0 tx=0).stop 2024-09-10T10:44:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.326+0000 7f7073f99640 1 -- 172.21.15.7:0/72140481 shutdown_connections 2024-09-10T10:44:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.326+0000 7f7073f99640 1 --2- 172.21.15.7:0/72140481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f706c1082e0 0x7f706c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.326+0000 7f7073f99640 1 --2- 172.21.15.7:0/72140481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706c103760 0x7f706c107ba0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.326+0000 7f7073f99640 1 --2- 172.21.15.7:0/72140481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f706c102db0 0x7f706c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.326+0000 7f7073f99640 1 -- 172.21.15.7:0/72140481 >> 172.21.15.7:0/72140481 conn(0x7f706c0fc7d0 msgr2=0x7f706c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.326+0000 7f7073f99640 1 -- 172.21.15.7:0/72140481 shutdown_connections 2024-09-10T10:44:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.326+0000 7f7073f99640 1 -- 172.21.15.7:0/72140481 wait complete. 2024-09-10T10:44:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7073f99640 1 Processor -- start 2024-09-10T10:44:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7073f99640 1 -- start start 2024-09-10T10:44:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7073f99640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f706c102db0 0x7f706c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7073f99640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706c103760 0x7f706c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7073f99640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f706c1082e0 0x7f706c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7073f99640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f706c112150 con 0x7f706c103760 2024-09-10T10:44:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7073f99640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f706c111fd0 con 0x7f706c1082e0 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7073f99640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f706c1122d0 con 0x7f706c102db0 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7071534640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706c103760 0x7f706c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7071534640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706c103760 0x7f706c198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45470/0 (socket says 172.21.15.7:45470) 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7071534640 1 -- 172.21.15.7:0/3363905330 learned_addr learned my addr 172.21.15.7:0/3363905330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7071d35640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f706c102db0 0x7f706c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.327+0000 7f7072536640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f706c1082e0 0x7f706c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f7071534640 1 -- 172.21.15.7:0/3363905330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f706c102db0 msgr2=0x7f706c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f7071534640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f706c102db0 0x7f706c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f7071534640 1 -- 172.21.15.7:0/3363905330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f706c1082e0 msgr2=0x7f706c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f7071534640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f706c1082e0 0x7f706c19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f7071534640 1 -- 172.21.15.7:0/3363905330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706c19d700 con 0x7f706c103760 2024-09-10T10:44:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f7072536640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f706c1082e0 0x7f706c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f7071534640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706c103760 0x7f706c198c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7060009230 tx=0x7f7060003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f706affd640 1 -- 172.21.15.7:0/3363905330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706003f070 con 0x7f706c103760 2024-09-10T10:44:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f7073f99640 1 -- 172.21.15.7:0/3363905330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706c19d990 con 0x7f706c103760 2024-09-10T10:44:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f706affd640 1 -- 172.21.15.7:0/3363905330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7060031e30 con 0x7f706c103760 2024-09-10T10:44:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f706affd640 1 -- 172.21.15.7:0/3363905330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060035030 con 0x7f706c103760 2024-09-10T10:44:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.328+0000 7f7073f99640 1 -- 172.21.15.7:0/3363905330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f706c1a52d0 con 0x7f706c103760 2024-09-10T10:44:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.329+0000 7f7073f99640 1 -- 172.21.15.7:0/3363905330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f706c103210 con 0x7f706c103760 2024-09-10T10:44:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.330+0000 7f706affd640 1 -- 172.21.15.7:0/3363905330 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7060042070 con 0x7f706c103760 2024-09-10T10:44:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.331+0000 7f706affd640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70540a2b80 0x7f70540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.331+0000 7f7071d35640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70540a2b80 0x7f70540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.331+0000 7f706affd640 1 -- 172.21.15.7:0/3363905330 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f70600efee0 con 0x7f706c103760 2024-09-10T10:44:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.331+0000 7f7071d35640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70540a2b80 0x7f70540a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7064009890 tx=0x7f7064008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.334+0000 7f706affd640 1 -- 172.21.15.7:0/3363905330 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70600bcef0 con 0x7f706c103760 2024-09-10T10:44:16.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:16 smithi103 ceph-mon[35160]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-10T10:44:16.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:16 smithi133 ceph-mon[35282]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-10T10:44:16.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:16.546+0000 7f7073f99640 1 -- 172.21.15.7:0/3363905330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f706c199ab0 con 0x7f706c103760 2024-09-10T10:44:17.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.043+0000 7f706affd640 1 -- 172.21.15.7:0/3363905330 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f706c199ab0 con 0x7f706c103760 2024-09-10T10:44:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 -- 172.21.15.7:0/3363905330 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70540a2b80 msgr2=0x7f70540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70540a2b80 0x7f70540a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7064009890 tx=0x7f7064008040 comp rx=0 tx=0).stop 2024-09-10T10:44:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 -- 172.21.15.7:0/3363905330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706c103760 msgr2=0x7f706c198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706c103760 0x7f706c198c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7060009230 tx=0x7f7060003c10 comp rx=0 tx=0).stop 2024-09-10T10:44:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 -- 172.21.15.7:0/3363905330 shutdown_connections 2024-09-10T10:44:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f70540a2b80 0x7f70540a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f706c1082e0 0x7f706c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f706c103760 0x7f706c198c90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 --2- 172.21.15.7:0/3363905330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f706c102db0 0x7f706c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 -- 172.21.15.7:0/3363905330 >> 172.21.15.7:0/3363905330 conn(0x7f706c0fc7d0 msgr2=0x7f706c105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:17.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 -- 172.21.15.7:0/3363905330 shutdown_connections 2024-09-10T10:44:17.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:17.046+0000 7f7073f99640 1 -- 172.21.15.7:0/3363905330 wait complete. 2024-09-10T10:44:17.329 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:17 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3363905330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T10:44:17.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:17 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3363905330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T10:44:17.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:17 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3363905330' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T10:44:17.622 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-10T10:44:17.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-10T10:44:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:18.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:18 smithi007 ceph-mon[30873]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-10T10:44:18.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:18 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3363905330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T10:44:18.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:18 smithi007 ceph-mon[30873]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.375+0000 7efc2436b640 1 -- 172.21.15.7:0/2889957975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc1c102ef0 msgr2=0x7efc1c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.375+0000 7efc2436b640 1 --2- 172.21.15.7:0/2889957975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc1c102ef0 0x7efc1c10da40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc14009ca0 tx=0x7efc1402a950 comp rx=0 tx=0).stop 2024-09-10T10:44:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.375+0000 7efc2436b640 1 -- 172.21.15.7:0/2889957975 shutdown_connections 2024-09-10T10:44:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.375+0000 7efc2436b640 1 --2- 172.21.15.7:0/2889957975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc1c102ef0 0x7efc1c10da40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.375+0000 7efc2436b640 1 --2- 172.21.15.7:0/2889957975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1c102570 0x7efc1c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.375+0000 7efc2436b640 1 --2- 172.21.15.7:0/2889957975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c100e20 0x7efc1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.375+0000 7efc2436b640 1 -- 172.21.15.7:0/2889957975 >> 172.21.15.7:0/2889957975 conn(0x7efc1c0fc7d0 msgr2=0x7efc1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.375+0000 7efc2436b640 1 -- 172.21.15.7:0/2889957975 shutdown_connections 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.375+0000 7efc2436b640 1 -- 172.21.15.7:0/2889957975 wait complete. 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.375+0000 7efc2436b640 1 Processor -- start 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc2436b640 1 -- start start 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc2436b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1c100e20 0x7efc1c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc2436b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc1c102570 0x7efc1c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc2436b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c102ef0 0x7efc1c1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc2436b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c110f80 con 0x7efc1c102570 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc2436b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c110e00 con 0x7efc1c102ef0 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc2436b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c111100 con 0x7efc1c100e20 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc21906640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc1c102570 0x7efc1c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc21906640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc1c102570 0x7efc1c1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45482/0 (socket says 172.21.15.7:45482) 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc21906640 1 -- 172.21.15.7:0/3212422824 learned_addr learned my addr 172.21.15.7:0/3212422824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc22107640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1c100e20 0x7efc1c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc22908640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c102ef0 0x7efc1c1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc21906640 1 -- 172.21.15.7:0/3212422824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1c100e20 msgr2=0x7efc1c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc21906640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1c100e20 0x7efc1c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc21906640 1 -- 172.21.15.7:0/3212422824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c102ef0 msgr2=0x7efc1c1a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc21906640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c102ef0 0x7efc1c1a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc21906640 1 -- 172.21.15.7:0/3212422824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1c1a4930 con 0x7efc1c102570 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc22107640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1c100e20 0x7efc1c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc22908640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c102ef0 0x7efc1c1a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.376+0000 7efc21906640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc1c102570 0x7efc1c1998b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00c970 tx=0x7efc0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.377+0000 7efc137fe640 1 -- 172.21.15.7:0/3212422824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c007bb0 con 0x7efc1c102570 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.377+0000 7efc137fe640 1 -- 172.21.15.7:0/3212422824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc0c007d50 con 0x7efc1c102570 2024-09-10T10:44:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.377+0000 7efc2436b640 1 -- 172.21.15.7:0/3212422824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1c1a4b00 con 0x7efc1c102570 2024-09-10T10:44:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.377+0000 7efc137fe640 1 -- 172.21.15.7:0/3212422824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00f450 con 0x7efc1c102570 2024-09-10T10:44:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.377+0000 7efc2436b640 1 -- 172.21.15.7:0/3212422824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1c1a5010 con 0x7efc1c102570 2024-09-10T10:44:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.378+0000 7efc2436b640 1 -- 172.21.15.7:0/3212422824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc1c101200 con 0x7efc1c102570 2024-09-10T10:44:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.379+0000 7efc137fe640 1 -- 172.21.15.7:0/3212422824 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7efc0c004fe0 con 0x7efc1c102570 2024-09-10T10:44:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.380+0000 7efc137fe640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efc080a2c00 0x7efc080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.380+0000 7efc22107640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efc080a2c00 0x7efc080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.380+0000 7efc137fe640 1 -- 172.21.15.7:0/3212422824 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7efc0c0cf0d0 con 0x7efc1c102570 2024-09-10T10:44:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.380+0000 7efc22107640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efc080a2c00 0x7efc080a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc180058c0 tx=0x7efc18005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:18.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.383+0000 7efc137fe640 1 -- 172.21.15.7:0/3212422824 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc0c014030 con 0x7efc1c102570 2024-09-10T10:44:18.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:18 smithi103 ceph-mon[35160]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-10T10:44:18.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:18 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3363905330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T10:44:18.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:18 smithi103 ceph-mon[35160]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:18.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:18 smithi133 ceph-mon[35282]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-10T10:44:18.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:18 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3363905330' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T10:44:18.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:18 smithi133 ceph-mon[35282]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:18.599+0000 7efc2436b640 1 -- 172.21.15.7:0/3212422824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7efc1c1a5240 con 0x7efc1c102570 2024-09-10T10:44:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.050+0000 7efc137fe640 1 -- 172.21.15.7:0/3212422824 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7efc0c09c160 con 0x7efc1c102570 2024-09-10T10:44:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 -- 172.21.15.7:0/3212422824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efc080a2c00 msgr2=0x7efc080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efc080a2c00 0x7efc080a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc180058c0 tx=0x7efc18005850 comp rx=0 tx=0).stop 2024-09-10T10:44:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 -- 172.21.15.7:0/3212422824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc1c102570 msgr2=0x7efc1c1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc1c102570 0x7efc1c1998b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00c970 tx=0x7efc0c00ce30 comp rx=0 tx=0).stop 2024-09-10T10:44:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 -- 172.21.15.7:0/3212422824 shutdown_connections 2024-09-10T10:44:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efc080a2c00 0x7efc080a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc1c102ef0 0x7efc1c1a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc1c102570 0x7efc1c1998b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 --2- 172.21.15.7:0/3212422824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc1c100e20 0x7efc1c199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 -- 172.21.15.7:0/3212422824 >> 172.21.15.7:0/3212422824 conn(0x7efc1c0fc7d0 msgr2=0x7efc1c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 -- 172.21.15.7:0/3212422824 shutdown_connections 2024-09-10T10:44:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:19.054+0000 7efc2436b640 1 -- 172.21.15.7:0/3212422824 wait complete. 2024-09-10T10:44:19.338 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:19 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3212422824' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T10:44:19.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:19 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3212422824' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T10:44:19.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:19 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3212422824' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-10T10:44:19.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T10:44:19.917 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:20.300 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:20 smithi007 ceph-mon[30873]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T10:44:20.300 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:20 smithi007 ceph-mon[30873]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T10:44:20.300 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:20 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3212422824' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T10:44:20.301 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:20 smithi007 ceph-mon[30873]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T10:44:20.301 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:20 smithi007 ceph-mon[30873]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:20.301 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:20 smithi007 ceph-mon[30873]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-10T10:44:20.301 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:20 smithi007 ceph-mon[30873]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-10T10:44:20.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.384+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/3486994456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a38106a80 msgr2=0x7f3a38106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.384+0000 7f3a3d0d5640 1 --2- 172.21.15.7:0/3486994456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a38106a80 0x7f3a38106e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009930 tx=0x7f3a2802a420 comp rx=0 tx=0).stop 2024-09-10T10:44:20.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.384+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/3486994456 shutdown_connections 2024-09-10T10:44:20.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.384+0000 7f3a3d0d5640 1 --2- 172.21.15.7:0/3486994456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a38101420 0x7f3a3810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.384+0000 7f3a3d0d5640 1 --2- 172.21.15.7:0/3486994456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a38100a80 0x7f3a38100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.384+0000 7f3a3d0d5640 1 --2- 172.21.15.7:0/3486994456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a38106a80 0x7f3a38106e60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.384+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/3486994456 >> 172.21.15.7:0/3486994456 conn(0x7f3a380fc7a0 msgr2=0x7f3a380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:20.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.384+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/3486994456 shutdown_connections 2024-09-10T10:44:20.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.384+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/3486994456 wait complete. 2024-09-10T10:44:20.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3d0d5640 1 Processor -- start 2024-09-10T10:44:20.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3d0d5640 1 -- start start 2024-09-10T10:44:20.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3d0d5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a38100a80 0x7f3a381970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3d0d5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a38101420 0x7f3a38197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3d0d5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a38106a80 0x7f3a38197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3d0d5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38110e80 con 0x7f3a38106a80 2024-09-10T10:44:20.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3d0d5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38110d00 con 0x7f3a38100a80 2024-09-10T10:44:20.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3d0d5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38111000 con 0x7f3a38101420 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a38106a80 0x7f3a38197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a38106a80 0x7f3a38197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44796/0 (socket says 172.21.15.7:44796) 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a3759e640 1 -- 172.21.15.7:0/2328075824 learned_addr learned my addr 172.21.15.7:0/2328075824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a2e59c640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a38101420 0x7f3a38197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.385+0000 7f3a36d9d640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a38100a80 0x7f3a381970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a3759e640 1 -- 172.21.15.7:0/2328075824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a38101420 msgr2=0x7f3a38197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a3759e640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a38101420 0x7f3a38197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a3759e640 1 -- 172.21.15.7:0/2328075824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a38100a80 msgr2=0x7f3a381970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a3759e640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a38100a80 0x7f3a381970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a3759e640 1 -- 172.21.15.7:0/2328075824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a381a6950 con 0x7f3a38106a80 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a36d9d640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a38100a80 0x7f3a381970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a3759e640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a38106a80 0x7f3a38197b60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009fd0 tx=0x7f3a3000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:20.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a34d99640 1 -- 172.21.15.7:0/2328075824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30017070 con 0x7f3a38106a80 2024-09-10T10:44:20.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/2328075824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a381a6c40 con 0x7f3a38106a80 2024-09-10T10:44:20.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a34d99640 1 -- 172.21.15.7:0/2328075824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a30003e50 con 0x7f3a38106a80 2024-09-10T10:44:20.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a34d99640 1 -- 172.21.15.7:0/2328075824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300106c0 con 0x7f3a38106a80 2024-09-10T10:44:20.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.386+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/2328075824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a381a7180 con 0x7f3a38106a80 2024-09-10T10:44:20.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.388+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/2328075824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a38100ee0 con 0x7f3a38106a80 2024-09-10T10:44:20.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.389+0000 7f3a34d99640 1 -- 172.21.15.7:0/2328075824 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3a30003970 con 0x7f3a38106a80 2024-09-10T10:44:20.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.389+0000 7f3a34d99640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3a240a2b80 0x7f3a240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.389+0000 7f3a34d99640 1 -- 172.21.15.7:0/2328075824 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f3a300ce8e0 con 0x7f3a38106a80 2024-09-10T10:44:20.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.389+0000 7f3a36d9d640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3a240a2b80 0x7f3a240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.390+0000 7f3a36d9d640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3a240a2b80 0x7f3a240a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a28000c00 tx=0x7f3a28005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:20.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.394+0000 7f3a34d99640 1 -- 172.21.15.7:0/2328075824 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a3009b990 con 0x7f3a38106a80 2024-09-10T10:44:20.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:20 smithi103 ceph-mon[35160]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T10:44:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:20 smithi103 ceph-mon[35160]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T10:44:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:20 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3212422824' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T10:44:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:20 smithi103 ceph-mon[35160]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T10:44:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:20 smithi103 ceph-mon[35160]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:20 smithi103 ceph-mon[35160]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-10T10:44:20.469 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:20 smithi103 ceph-mon[35160]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-10T10:44:20.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:20 smithi133 ceph-mon[35282]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T10:44:20.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:20 smithi133 ceph-mon[35282]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T10:44:20.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:20 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3212422824' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-10T10:44:20.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:20 smithi133 ceph-mon[35282]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T10:44:20.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:20 smithi133 ceph-mon[35282]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-10T10:44:20.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:20 smithi133 ceph-mon[35282]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-10T10:44:20.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:20 smithi133 ceph-mon[35282]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-10T10:44:20.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:20.605+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/2328075824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f3a3802d7c0 con 0x7f3a38106a80 2024-09-10T10:44:21.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.068+0000 7f3a34d99640 1 -- 172.21.15.7:0/2328075824 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f3a3009b290 con 0x7f3a38106a80 2024-09-10T10:44:21.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/2328075824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3a240a2b80 msgr2=0x7f3a240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:21.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3a240a2b80 0x7f3a240a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3a28000c00 tx=0x7f3a28005a20 comp rx=0 tx=0).stop 2024-09-10T10:44:21.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/2328075824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a38106a80 msgr2=0x7f3a38197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:21.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a38106a80 0x7f3a38197b60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009fd0 tx=0x7f3a3000b260 comp rx=0 tx=0).stop 2024-09-10T10:44:21.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/2328075824 shutdown_connections 2024-09-10T10:44:21.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f3a240a2b80 0x7f3a240a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a38106a80 0x7f3a38197b60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a38101420 0x7f3a38197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 --2- 172.21.15.7:0/2328075824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a38100a80 0x7f3a381970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:21.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/2328075824 >> 172.21.15.7:0/2328075824 conn(0x7f3a380fc7a0 msgr2=0x7f3a380fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:21.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/2328075824 shutdown_connections 2024-09-10T10:44:21.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:21.071+0000 7f3a3d0d5640 1 -- 172.21.15.7:0/2328075824 wait complete. 2024-09-10T10:44:21.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] up:active 2024-09-10T10:44:21.388 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: Cluster is now healthy 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2328075824' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-10T10:44:21.389 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-10T10:44:21.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-10T10:44:21.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-10T10:44:21.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:21 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-10T10:44:21.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] up:active 2024-09-10T10:44:21.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T10:44:21.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: Cluster is now healthy 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2328075824' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-10T10:44:21.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:21 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-10T10:44:21.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] up:active 2024-09-10T10:44:21.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T10:44:21.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T10:44:21.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: Cluster is now healthy 2024-09-10T10:44:21.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-10T10:44:21.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-10T10:44:21.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-10T10:44:21.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2328075824' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T10:44:21.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-10T10:44:21.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-09-10T10:44:21.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-10T10:44:21.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-10T10:44:21.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-10T10:44:21.477 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-10T10:44:21.477 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-10T10:44:21.477 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:21 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-10T10:44:21.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T10:44:21.987 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:22.392 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-10T10:44:22.392 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2328075824' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T10:44:22.392 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-10T10:44:22.392 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-09-10T10:44:22.392 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-10T10:44:22.392 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-10T10:44:22.392 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-10T10:44:22.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-10T10:44:22.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-10T10:44:22.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-10T10:44:22.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T10:44:22.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] up:active 2024-09-10T10:44:22.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-10T10:44:22.393 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:22 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:44:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-10T10:44:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2328075824' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T10:44:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-10T10:44:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-09-10T10:44:22.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-10T10:44:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-10T10:44:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-10T10:44:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-10T10:44:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-10T10:44:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-10T10:44:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T10:44:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] up:active 2024-09-10T10:44:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-10T10:44:22.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:22 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:44:22.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2328075824' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T10:44:22.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] up:active 2024-09-10T10:44:22.477 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-09-10T10:44:22.477 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:22 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.476+0000 7f38109e8640 1 -- 172.21.15.7:0/597202531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f380c103660 msgr2=0x7f380c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.476+0000 7f38109e8640 1 --2- 172.21.15.7:0/597202531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f380c103660 0x7f380c10bb50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3800009900 tx=0x7f380002a480 comp rx=0 tx=0).stop 2024-09-10T10:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.477+0000 7f38109e8640 1 -- 172.21.15.7:0/597202531 shutdown_connections 2024-09-10T10:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.477+0000 7f38109e8640 1 --2- 172.21.15.7:0/597202531 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f380c10c170 0x7f380c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.477+0000 7f38109e8640 1 --2- 172.21.15.7:0/597202531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f380c103660 0x7f380c10bb50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.477+0000 7f38109e8640 1 --2- 172.21.15.7:0/597202531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c102d40 0x7f380c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.477+0000 7f38109e8640 1 -- 172.21.15.7:0/597202531 >> 172.21.15.7:0/597202531 conn(0x7f380c0fc7a0 msgr2=0x7f380c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.477+0000 7f38109e8640 1 -- 172.21.15.7:0/597202531 shutdown_connections 2024-09-10T10:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.477+0000 7f38109e8640 1 -- 172.21.15.7:0/597202531 wait complete. 2024-09-10T10:44:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f38109e8640 1 Processor -- start 2024-09-10T10:44:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f38109e8640 1 -- start start 2024-09-10T10:44:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f38109e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f380c102d40 0x7f380c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f38109e8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f380c103660 0x7f380c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f38109e8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c10c170 0x7f380c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f38109e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f380c111fd0 con 0x7f380c102d40 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f38109e8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f380c111e50 con 0x7f380c103660 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f38109e8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f380c112150 con 0x7f380c10c170 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f380a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f380c102d40 0x7f380c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f380a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f380c102d40 0x7f380c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44814/0 (socket says 172.21.15.7:44814) 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f380a59c640 1 -- 172.21.15.7:0/4193153297 learned_addr learned my addr 172.21.15.7:0/4193153297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f380ad9d640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c10c170 0x7f380c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f3809d9b640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f380c103660 0x7f380c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f380a59c640 1 -- 172.21.15.7:0/4193153297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c10c170 msgr2=0x7f380c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f380a59c640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c10c170 0x7f380c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f380a59c640 1 -- 172.21.15.7:0/4193153297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f380c103660 msgr2=0x7f380c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f380a59c640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f380c103660 0x7f380c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f380a59c640 1 -- 172.21.15.7:0/4193153297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c19d450 con 0x7f380c102d40 2024-09-10T10:44:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.478+0000 7f380ad9d640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c10c170 0x7f380c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.479+0000 7f380a59c640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f380c102d40 0x7f380c1984a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00d770 tx=0x7f37fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.479+0000 7f37fb7fe640 1 -- 172.21.15.7:0/4193153297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc017070 con 0x7f380c102d40 2024-09-10T10:44:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.479+0000 7f38109e8640 1 -- 172.21.15.7:0/4193153297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c19d740 con 0x7f380c102d40 2024-09-10T10:44:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.479+0000 7f37fb7fe640 1 -- 172.21.15.7:0/4193153297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37fc005500 con 0x7f380c102d40 2024-09-10T10:44:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.479+0000 7f37fb7fe640 1 -- 172.21.15.7:0/4193153297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc004e00 con 0x7f380c102d40 2024-09-10T10:44:22.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.479+0000 7f38109e8640 1 -- 172.21.15.7:0/4193153297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c1a5020 con 0x7f380c102d40 2024-09-10T10:44:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.480+0000 7f38109e8640 1 -- 172.21.15.7:0/4193153297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c103120 con 0x7f380c102d40 2024-09-10T10:44:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.481+0000 7f37fb7fe640 1 -- 172.21.15.7:0/4193153297 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f37fc004fa0 con 0x7f380c102d40 2024-09-10T10:44:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.482+0000 7f37fb7fe640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37f40a2b80 0x7f37f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.482+0000 7f3809d9b640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37f40a2b80 0x7f37f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.482+0000 7f37fb7fe640 1 -- 172.21.15.7:0/4193153297 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f37fc0cef90 con 0x7f380c102d40 2024-09-10T10:44:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.482+0000 7f3809d9b640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37f40a2b80 0x7f37f40a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38000096c0 tx=0x7f3800000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:22.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.485+0000 7f37fb7fe640 1 -- 172.21.15.7:0/4193153297 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37fc09bf00 con 0x7f380c102d40 2024-09-10T10:44:22.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:22.695+0000 7f38109e8640 1 -- 172.21.15.7:0/4193153297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f380c1071e0 con 0x7f380c102d40 2024-09-10T10:44:23.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.080+0000 7f37fb7fe640 1 -- 172.21.15.7:0/4193153297 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f37fc09b800 con 0x7f380c102d40 2024-09-10T10:44:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.082+0000 7f38109e8640 1 -- 172.21.15.7:0/4193153297 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37f40a2b80 msgr2=0x7f37f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.082+0000 7f38109e8640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37f40a2b80 0x7f37f40a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38000096c0 tx=0x7f3800000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.082+0000 7f38109e8640 1 -- 172.21.15.7:0/4193153297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f380c102d40 msgr2=0x7f380c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.082+0000 7f38109e8640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f380c102d40 0x7f380c1984a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00d770 tx=0x7f37fc00dc30 comp rx=0 tx=0).stop 2024-09-10T10:44:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.083+0000 7f38109e8640 1 -- 172.21.15.7:0/4193153297 shutdown_connections 2024-09-10T10:44:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.083+0000 7f38109e8640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37f40a2b80 0x7f37f40a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.083+0000 7f38109e8640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f380c10c170 0x7f380c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.083+0000 7f38109e8640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f380c103660 0x7f380c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.083+0000 7f38109e8640 1 --2- 172.21.15.7:0/4193153297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f380c102d40 0x7f380c1984a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:23.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.083+0000 7f38109e8640 1 -- 172.21.15.7:0/4193153297 >> 172.21.15.7:0/4193153297 conn(0x7f380c0fc7a0 msgr2=0x7f380c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:23.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.083+0000 7f38109e8640 1 -- 172.21.15.7:0/4193153297 shutdown_connections 2024-09-10T10:44:23.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:23.083+0000 7f38109e8640 1 -- 172.21.15.7:0/4193153297 wait complete. 2024-09-10T10:44:23.379 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:23 smithi007 ceph-mon[30873]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T10:44:23.379 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:23 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4193153297' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T10:44:23.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:23 smithi103 ceph-mon[35160]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T10:44:23.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:23 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4193153297' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T10:44:23.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:23 smithi133 ceph-mon[35282]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T10:44:23.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:23 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4193153297' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T10:44:23.718 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-10T10:44:23.718 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T10:44:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:24.333 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:24 smithi007 ceph-mon[30873]: pgmap v282: 97 pgs: 1 peering, 96 active+clean; 580 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:44:24.333 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:24 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4193153297' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T10:44:24.333 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:24 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:44:24.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:24 smithi103 ceph-mon[35160]: pgmap v282: 97 pgs: 1 peering, 96 active+clean; 580 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:44:24.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:24 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4193153297' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T10:44:24.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:24 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:44:24.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.419+0000 7f73ac73d640 1 -- 172.21.15.7:0/3791029021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a41036e0 msgr2=0x7f73a4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.419+0000 7f73ac73d640 1 --2- 172.21.15.7:0/3791029021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a41036e0 0x7f73a4107b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7398009930 tx=0x7f739802a420 comp rx=0 tx=0).stop 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.419+0000 7f73ac73d640 1 -- 172.21.15.7:0/3791029021 shutdown_connections 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.419+0000 7f73ac73d640 1 --2- 172.21.15.7:0/3791029021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a41082a0 0x7f73a410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.419+0000 7f73ac73d640 1 --2- 172.21.15.7:0/3791029021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a41036e0 0x7f73a4107b90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.419+0000 7f73ac73d640 1 --2- 172.21.15.7:0/3791029021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a4102d30 0x7f73a4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.419+0000 7f73ac73d640 1 -- 172.21.15.7:0/3791029021 >> 172.21.15.7:0/3791029021 conn(0x7f73a40fc7d0 msgr2=0x7f73a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.419+0000 7f73ac73d640 1 -- 172.21.15.7:0/3791029021 shutdown_connections 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.419+0000 7f73ac73d640 1 -- 172.21.15.7:0/3791029021 wait complete. 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73ac73d640 1 Processor -- start 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73ac73d640 1 -- start start 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73ac73d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a4102d30 0x7f73a4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73ac73d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a41036e0 0x7f73a4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73ac73d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a41082a0 0x7f73a419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73ac73d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41120c0 con 0x7f73a41036e0 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73ac73d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4111f40 con 0x7f73a41082a0 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73ac73d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4112240 con 0x7f73a4102d30 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73a9cd8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a41036e0 0x7f73a4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73a9cd8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a41036e0 0x7f73a4198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44824/0 (socket says 172.21.15.7:44824) 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73a9cd8640 1 -- 172.21.15.7:0/3380545432 learned_addr learned my addr 172.21.15.7:0/3380545432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73aacda640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a41082a0 0x7f73a419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73aa4d9640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a4102d30 0x7f73a4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73a9cd8640 1 -- 172.21.15.7:0/3380545432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a4102d30 msgr2=0x7f73a4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73a9cd8640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a4102d30 0x7f73a4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73a9cd8640 1 -- 172.21.15.7:0/3380545432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a41082a0 msgr2=0x7f73a419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73a9cd8640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a41082a0 0x7f73a419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.420+0000 7f73a9cd8640 1 -- 172.21.15.7:0/3380545432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a419d6e0 con 0x7f73a41036e0 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.421+0000 7f73a9cd8640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a41036e0 0x7f73a4198c70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7398005940 tx=0x7f73980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.421+0000 7f73977fe640 1 -- 172.21.15.7:0/3380545432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739803f030 con 0x7f73a41036e0 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.421+0000 7f73977fe640 1 -- 172.21.15.7:0/3380545432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7398003ce0 con 0x7f73a41036e0 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.421+0000 7f73ac73d640 1 -- 172.21.15.7:0/3380545432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a419d970 con 0x7f73a41036e0 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.421+0000 7f73977fe640 1 -- 172.21.15.7:0/3380545432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398035070 con 0x7f73a41036e0 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.421+0000 7f73ac73d640 1 -- 172.21.15.7:0/3380545432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a41a52b0 con 0x7f73a41036e0 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.422+0000 7f73ac73d640 1 -- 172.21.15.7:0/3380545432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a4103110 con 0x7f73a41036e0 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.423+0000 7f73977fe640 1 -- 172.21.15.7:0/3380545432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7398042030 con 0x7f73a41036e0 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.424+0000 7f73977fe640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f73900a2b80 0x7f73900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.424+0000 7f73aa4d9640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f73900a2b80 0x7f73900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:24.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.424+0000 7f73977fe640 1 -- 172.21.15.7:0/3380545432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f73980ef8b0 con 0x7f73a41036e0 2024-09-10T10:44:24.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.424+0000 7f73aa4d9640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f73900a2b80 0x7f73900a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005c40 tx=0x7f73a000a780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:24.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.427+0000 7f73977fe640 1 -- 172.21.15.7:0/3380545432 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73980bc7c0 con 0x7f73a41036e0 2024-09-10T10:44:24.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:24 smithi133 ceph-mon[35282]: pgmap v282: 97 pgs: 1 peering, 96 active+clean; 580 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:44:24.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:24 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4193153297' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T10:44:24.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:24 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:44:24.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.626+0000 7f73ac73d640 1 -- 172.21.15.7:0/3380545432 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f73a4062630 con 0x7f73900a2b80 2024-09-10T10:44:24.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.646+0000 7f73977fe640 1 -- 172.21.15.7:0/3380545432 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f73a4062630 con 0x7f73900a2b80 2024-09-10T10:44:24.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 -- 172.21.15.7:0/3380545432 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f73900a2b80 msgr2=0x7f73900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f73900a2b80 0x7f73900a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005c40 tx=0x7f73a000a780 comp rx=0 tx=0).stop 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 -- 172.21.15.7:0/3380545432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a41036e0 msgr2=0x7f73a4198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a41036e0 0x7f73a4198c70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7398005940 tx=0x7f73980042f0 comp rx=0 tx=0).stop 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 -- 172.21.15.7:0/3380545432 shutdown_connections 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f73900a2b80 0x7f73900a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73a41082a0 0x7f73a419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73a41036e0 0x7f73a4198c70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 --2- 172.21.15.7:0/3380545432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73a4102d30 0x7f73a4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 -- 172.21.15.7:0/3380545432 >> 172.21.15.7:0/3380545432 conn(0x7f73a40fc7d0 msgr2=0x7f73a4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 -- 172.21.15.7:0/3380545432 shutdown_connections 2024-09-10T10:44:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:24.648+0000 7f73ac73d640 1 -- 172.21.15.7:0/3380545432 wait complete. 2024-09-10T10:44:24.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:44:24 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:44:24.631+0000 7fe00b6bc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:44:24.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:44:24 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:44:24.631+0000 7fe00b6bc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:44:24.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:44:24 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:44:24.631+0000 7fe00b6bc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:44:24.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:44:24 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:44:24.631+0000 7fe00b6bc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:44:24.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:44:24 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:44:24.631+0000 7fe00b6bc640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:44:25.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T10:44:25.555 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.010+0000 7fd1f6241640 1 -- 172.21.15.7:0/888532714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1f01036e0 msgr2=0x7fd1f0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.010+0000 7fd1f6241640 1 --2- 172.21.15.7:0/888532714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1f01036e0 0x7fd1f0107b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009930 tx=0x7fd1e402a420 comp rx=0 tx=0).stop 2024-09-10T10:44:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.010+0000 7fd1f6241640 1 -- 172.21.15.7:0/888532714 shutdown_connections 2024-09-10T10:44:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.010+0000 7fd1f6241640 1 --2- 172.21.15.7:0/888532714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f01082a0 0x7fd1f010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.010+0000 7fd1f6241640 1 --2- 172.21.15.7:0/888532714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1f01036e0 0x7fd1f0107b90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.010+0000 7fd1f6241640 1 --2- 172.21.15.7:0/888532714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f0102d30 0x7fd1f0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.010+0000 7fd1f6241640 1 -- 172.21.15.7:0/888532714 >> 172.21.15.7:0/888532714 conn(0x7fd1f00fc7d0 msgr2=0x7fd1f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.010+0000 7fd1f6241640 1 -- 172.21.15.7:0/888532714 shutdown_connections 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.010+0000 7fd1f6241640 1 -- 172.21.15.7:0/888532714 wait complete. 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1f6241640 1 Processor -- start 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1f6241640 1 -- start start 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1f6241640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1f0102d30 0x7fd1f0198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1f6241640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f01036e0 0x7fd1f0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1f6241640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f01082a0 0x7fd1f019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1f6241640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f01120c0 con 0x7fd1f0102d30 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1f6241640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0111f40 con 0x7fd1f01036e0 2024-09-10T10:44:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1f6241640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0112240 con 0x7fd1f01082a0 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1f0102d30 0x7fd1f0198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1f0102d30 0x7fd1f0198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44866/0 (socket says 172.21.15.7:44866) 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1ef7fe640 1 -- 172.21.15.7:0/216346573 learned_addr learned my addr 172.21.15.7:0/216346573 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1effff640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f01082a0 0x7fd1f019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1eeffd640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f01036e0 0x7fd1f0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1ef7fe640 1 -- 172.21.15.7:0/216346573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f01082a0 msgr2=0x7fd1f019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1ef7fe640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f01082a0 0x7fd1f019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1ef7fe640 1 -- 172.21.15.7:0/216346573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f01036e0 msgr2=0x7fd1f0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.011+0000 7fd1ef7fe640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f01036e0 0x7fd1f0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.012+0000 7fd1ef7fe640 1 -- 172.21.15.7:0/216346573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f019d510 con 0x7fd1f0102d30 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.012+0000 7fd1ef7fe640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1f0102d30 0x7fd1f0198560 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c920 tx=0x7fd1e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.012+0000 7fd1ecff9640 1 -- 172.21.15.7:0/216346573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0007c70 con 0x7fd1f0102d30 2024-09-10T10:44:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.012+0000 7fd1f6241640 1 -- 172.21.15.7:0/216346573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f019d800 con 0x7fd1f0102d30 2024-09-10T10:44:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.012+0000 7fd1ecff9640 1 -- 172.21.15.7:0/216346573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e0007e10 con 0x7fd1f0102d30 2024-09-10T10:44:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.012+0000 7fd1ecff9640 1 -- 172.21.15.7:0/216346573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0018890 con 0x7fd1f0102d30 2024-09-10T10:44:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.012+0000 7fd1f6241640 1 -- 172.21.15.7:0/216346573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f01a50e0 con 0x7fd1f0102d30 2024-09-10T10:44:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.013+0000 7fd1f6241640 1 -- 172.21.15.7:0/216346573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f0103110 con 0x7fd1f0102d30 2024-09-10T10:44:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.014+0000 7fd1ecff9640 1 -- 172.21.15.7:0/216346573 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd1e0018a50 con 0x7fd1f0102d30 2024-09-10T10:44:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.015+0000 7fd1ecff9640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1d80a2c10 0x7fd1d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.015+0000 7fd1eeffd640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1d80a2c10 0x7fd1d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.015+0000 7fd1ecff9640 1 -- 172.21.15.7:0/216346573 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fd1e00cf5e0 con 0x7fd1f0102d30 2024-09-10T10:44:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.016+0000 7fd1eeffd640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1d80a2c10 0x7fd1d80a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009900 tx=0x7fd1e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:26.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.019+0000 7fd1ecff9640 1 -- 172.21.15.7:0/216346573 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1e0014030 con 0x7fd1f0102d30 2024-09-10T10:44:26.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:26 smithi007 ceph-mon[30873]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:26.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:26 smithi007 ceph-mon[30873]: pgmap v283: 97 pgs: 1 peering, 96 active+clean; 582 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:26.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:26 smithi007 ceph-mon[30873]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T10:44:26.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.222+0000 7fd1f6241640 1 -- 172.21.15.7:0/216346573 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd1f0107420 con 0x7fd1d80a2c10 2024-09-10T10:44:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.245+0000 7fd1ecff9640 1 -- 172.21.15.7:0/216346573 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd1f0107420 con 0x7fd1d80a2c10 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 -- 172.21.15.7:0/216346573 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1d80a2c10 msgr2=0x7fd1d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1d80a2c10 0x7fd1d80a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009900 tx=0x7fd1e4000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 -- 172.21.15.7:0/216346573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1f0102d30 msgr2=0x7fd1f0198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1f0102d30 0x7fd1f0198560 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c920 tx=0x7fd1e000cde0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 -- 172.21.15.7:0/216346573 shutdown_connections 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd1d80a2c10 0x7fd1d80a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1f01082a0 0x7fd1f019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1f01036e0 0x7fd1f0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 --2- 172.21.15.7:0/216346573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1f0102d30 0x7fd1f0198560 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 -- 172.21.15.7:0/216346573 >> 172.21.15.7:0/216346573 conn(0x7fd1f00fc7d0 msgr2=0x7fd1f0105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 -- 172.21.15.7:0/216346573 shutdown_connections 2024-09-10T10:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:26.247+0000 7fd1f6241640 1 -- 172.21.15.7:0/216346573 wait complete. 2024-09-10T10:44:26.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:26 smithi103 ceph-mon[35160]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:26.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:26 smithi103 ceph-mon[35160]: pgmap v283: 97 pgs: 1 peering, 96 active+clean; 582 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:26.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:26 smithi103 ceph-mon[35160]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T10:44:26.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:26 smithi133 ceph-mon[35282]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:26.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:26 smithi133 ceph-mon[35282]: pgmap v283: 97 pgs: 1 peering, 96 active+clean; 582 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:26.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:26 smithi133 ceph-mon[35282]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T10:44:26.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T10:44:27.125 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:27.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:27 smithi007 ceph-mon[30873]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:27.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:27 smithi103 ceph-mon[35160]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:27.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:27 smithi133 ceph-mon[35282]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.561+0000 7facfd82f640 1 -- 172.21.15.7:0/2386140394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf8106f50 msgr2=0x7facf810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.561+0000 7facfd82f640 1 --2- 172.21.15.7:0/2386140394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf8106f50 0x7facf810eaf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7facf0009cd0 tx=0x7facf002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.561+0000 7facfd82f640 1 -- 172.21.15.7:0/2386140394 shutdown_connections 2024-09-10T10:44:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.561+0000 7facfd82f640 1 --2- 172.21.15.7:0/2386140394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf8106f50 0x7facf810eaf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.561+0000 7facfd82f640 1 --2- 172.21.15.7:0/2386140394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf8100d00 0x7facf8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:27.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.561+0000 7facfd82f640 1 --2- 172.21.15.7:0/2386140394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf8100350 0x7facf8100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:27.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.561+0000 7facfd82f640 1 -- 172.21.15.7:0/2386140394 >> 172.21.15.7:0/2386140394 conn(0x7facf8077210 msgr2=0x7facf80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:27.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.561+0000 7facfd82f640 1 -- 172.21.15.7:0/2386140394 shutdown_connections 2024-09-10T10:44:27.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.561+0000 7facfd82f640 1 -- 172.21.15.7:0/2386140394 wait complete. 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.561+0000 7facfd82f640 1 Processor -- start 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facfd82f640 1 -- start start 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facfd82f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf8100350 0x7facf81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facfd82f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf8100d00 0x7facf81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facfd82f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf8106f50 0x7facf819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facfd82f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7facf8112000 con 0x7facf8106f50 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facfd82f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7facf8111e80 con 0x7facf8100350 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facfd82f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7facf8112180 con 0x7facf8100d00 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facf77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf8106f50 0x7facf819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facf77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf8106f50 0x7facf819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44884/0 (socket says 172.21.15.7:44884) 2024-09-10T10:44:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facf77fe640 1 -- 172.21.15.7:0/3650170053 learned_addr learned my addr 172.21.15.7:0/3650170053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facf67fc640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf8100d00 0x7facf81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.562+0000 7facf6ffd640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf8100350 0x7facf81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facf77fe640 1 -- 172.21.15.7:0/3650170053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf8100d00 msgr2=0x7facf81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facf77fe640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf8100d00 0x7facf81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facf77fe640 1 -- 172.21.15.7:0/3650170053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf8100350 msgr2=0x7facf81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facf77fe640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf8100350 0x7facf81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facf77fe640 1 -- 172.21.15.7:0/3650170053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf819d450 con 0x7facf8106f50 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facf77fe640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf8106f50 0x7facf819cd70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7facf0006010 tx=0x7facf0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facfc82d640 1 -- 172.21.15.7:0/3650170053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0004450 con 0x7facf8106f50 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facfd82f640 1 -- 172.21.15.7:0/3650170053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf819d6e0 con 0x7facf8106f50 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facfc82d640 1 -- 172.21.15.7:0/3650170053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facf0003e10 con 0x7facf8106f50 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facfc82d640 1 -- 172.21.15.7:0/3650170053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0046480 con 0x7facf8106f50 2024-09-10T10:44:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.563+0000 7facfd82f640 1 -- 172.21.15.7:0/3650170053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf81a5020 con 0x7facf8106f50 2024-09-10T10:44:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.564+0000 7facfd82f640 1 -- 172.21.15.7:0/3650170053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf8100730 con 0x7facf8106f50 2024-09-10T10:44:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.565+0000 7facfc82d640 1 -- 172.21.15.7:0/3650170053 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7facf00045f0 con 0x7facf8106f50 2024-09-10T10:44:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.566+0000 7facfc82d640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7face00a2c10 0x7face00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:27.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.566+0000 7facf6ffd640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7face00a2c10 0x7face00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:27.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.566+0000 7facfc82d640 1 -- 172.21.15.7:0/3650170053 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7facf002e030 con 0x7facf8106f50 2024-09-10T10:44:27.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.566+0000 7facf6ffd640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7face00a2c10 0x7face00a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7face8009990 tx=0x7face8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:27.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.569+0000 7facfc82d640 1 -- 172.21.15.7:0/3650170053 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7facf004b060 con 0x7facf8106f50 2024-09-10T10:44:27.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.771+0000 7facfd82f640 1 -- 172.21.15.7:0/3650170053 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7facf8062630 con 0x7face00a2c10 2024-09-10T10:44:27.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.776+0000 7facfc82d640 1 -- 172.21.15.7:0/3650170053 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7facf8062630 con 0x7face00a2c10 2024-09-10T10:44:27.777 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_0/b63505fb-ddc0-4ac8-b34e-076cbe2e75b4 2024-09-10T10:44:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 -- 172.21.15.7:0/3650170053 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7face00a2c10 msgr2=0x7face00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7face00a2c10 0x7face00a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7face8009990 tx=0x7face8009110 comp rx=0 tx=0).stop 2024-09-10T10:44:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 -- 172.21.15.7:0/3650170053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf8106f50 msgr2=0x7facf819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf8106f50 0x7facf819cd70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7facf0006010 tx=0x7facf0003c20 comp rx=0 tx=0).stop 2024-09-10T10:44:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 -- 172.21.15.7:0/3650170053 shutdown_connections 2024-09-10T10:44:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7face00a2c10 0x7face00a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facf8106f50 0x7facf819cd70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facf8100d00 0x7facf81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 --2- 172.21.15.7:0/3650170053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf8100350 0x7facf81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 -- 172.21.15.7:0/3650170053 >> 172.21.15.7:0/3650170053 conn(0x7facf8077210 msgr2=0x7facf8105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 -- 172.21.15.7:0/3650170053 shutdown_connections 2024-09-10T10:44:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:27.778+0000 7facfd82f640 1 -- 172.21.15.7:0/3650170053 wait complete. 2024-09-10T10:44:28.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:28 smithi007 ceph-mon[30873]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:28.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:28 smithi103 ceph-mon[35160]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:28.443 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T10:44:28.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:28 smithi133 ceph-mon[35282]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:28.704 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:29.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.160+0000 7ff533066640 1 -- 172.21.15.7:0/675051509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52c100a80 msgr2=0x7ff52c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.160+0000 7ff533066640 1 --2- 172.21.15.7:0/675051509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52c100a80 0x7ff52c100ee0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009930 tx=0x7ff51c02a420 comp rx=0 tx=0).stop 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.160+0000 7ff533066640 1 -- 172.21.15.7:0/675051509 shutdown_connections 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.160+0000 7ff533066640 1 --2- 172.21.15.7:0/675051509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff52c101420 0x7ff52c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.160+0000 7ff533066640 1 --2- 172.21.15.7:0/675051509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52c100a80 0x7ff52c100ee0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.160+0000 7ff533066640 1 --2- 172.21.15.7:0/675051509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c106a80 0x7ff52c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.160+0000 7ff533066640 1 -- 172.21.15.7:0/675051509 >> 172.21.15.7:0/675051509 conn(0x7ff52c0fc7a0 msgr2=0x7ff52c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.160+0000 7ff533066640 1 -- 172.21.15.7:0/675051509 shutdown_connections 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.160+0000 7ff533066640 1 -- 172.21.15.7:0/675051509 wait complete. 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff533066640 1 Processor -- start 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff533066640 1 -- start start 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff533066640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c100a80 0x7ff52c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff533066640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff52c101420 0x7ff52c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff533066640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52c106a80 0x7ff52c1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff533066640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c110e80 con 0x7ff52c106a80 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff533066640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c110d00 con 0x7ff52c101420 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff533066640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c111000 con 0x7ff52c100a80 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff531603640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52c106a80 0x7ff52c1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff531603640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52c106a80 0x7ff52c1a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44904/0 (socket says 172.21.15.7:44904) 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff531603640 1 -- 172.21.15.7:0/3579750280 learned_addr learned my addr 172.21.15.7:0/3579750280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.161+0000 7ff523fff640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff52c101420 0x7ff52c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff530e02640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c100a80 0x7ff52c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff531603640 1 -- 172.21.15.7:0/3579750280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c100a80 msgr2=0x7ff52c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff531603640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c100a80 0x7ff52c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff531603640 1 -- 172.21.15.7:0/3579750280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff52c101420 msgr2=0x7ff52c199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff531603640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff52c101420 0x7ff52c199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff531603640 1 -- 172.21.15.7:0/3579750280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c1a48b0 con 0x7ff52c106a80 2024-09-10T10:44:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff530e02640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c100a80 0x7ff52c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff531603640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52c106a80 0x7ff52c1a4370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff52400c970 tx=0x7ff52400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff521ffb640 1 -- 172.21.15.7:0/3579750280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524007bb0 con 0x7ff52c106a80 2024-09-10T10:44:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff521ffb640 1 -- 172.21.15.7:0/3579750280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff524007d50 con 0x7ff52c106a80 2024-09-10T10:44:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff521ffb640 1 -- 172.21.15.7:0/3579750280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52400f450 con 0x7ff52c106a80 2024-09-10T10:44:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff533066640 1 -- 172.21.15.7:0/3579750280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52c1a4a80 con 0x7ff52c106a80 2024-09-10T10:44:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.162+0000 7ff533066640 1 -- 172.21.15.7:0/3579750280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52c1a4f40 con 0x7ff52c106a80 2024-09-10T10:44:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.163+0000 7ff533066640 1 -- 172.21.15.7:0/3579750280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff52c100ee0 con 0x7ff52c106a80 2024-09-10T10:44:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.164+0000 7ff521ffb640 1 -- 172.21.15.7:0/3579750280 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff524004f60 con 0x7ff52c106a80 2024-09-10T10:44:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.165+0000 7ff521ffb640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff5140a2c90 0x7ff5140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.165+0000 7ff530e02640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff5140a2c90 0x7ff5140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.165+0000 7ff521ffb640 1 -- 172.21.15.7:0/3579750280 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7ff5240cee70 con 0x7ff52c106a80 2024-09-10T10:44:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.166+0000 7ff530e02640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff5140a2c90 0x7ff5140a5150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff528005fd0 tx=0x7ff5280082b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.169+0000 7ff521ffb640 1 -- 172.21.15.7:0/3579750280 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff524014030 con 0x7ff52c106a80 2024-09-10T10:44:29.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.369+0000 7ff533066640 1 -- 172.21.15.7:0/3579750280 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff52c109080 con 0x7ff5140a2c90 2024-09-10T10:44:29.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:29 smithi007 ceph-mon[30873]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:29.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.395+0000 7ff521ffb640 1 -- 172.21.15.7:0/3579750280 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff528008930 con 0x7ff5140a2c90 2024-09-10T10:44:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 -- 172.21.15.7:0/3579750280 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff5140a2c90 msgr2=0x7ff5140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff5140a2c90 0x7ff5140a5150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff528005fd0 tx=0x7ff5280082b0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 -- 172.21.15.7:0/3579750280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52c106a80 msgr2=0x7ff52c1a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52c106a80 0x7ff52c1a4370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff52400c970 tx=0x7ff52400ce30 comp rx=0 tx=0).stop 2024-09-10T10:44:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 -- 172.21.15.7:0/3579750280 shutdown_connections 2024-09-10T10:44:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7ff5140a2c90 0x7ff5140a5150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff52c106a80 0x7ff52c1a4370 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff52c101420 0x7ff52c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 --2- 172.21.15.7:0/3579750280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff52c100a80 0x7ff52c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 -- 172.21.15.7:0/3579750280 >> 172.21.15.7:0/3579750280 conn(0x7ff52c0fc7a0 msgr2=0x7ff52c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 -- 172.21.15.7:0/3579750280 shutdown_connections 2024-09-10T10:44:29.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:29.397+0000 7ff533066640 1 -- 172.21.15.7:0/3579750280 wait complete. 2024-09-10T10:44:29.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:29 smithi103 ceph-mon[35160]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:29.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:29 smithi133 ceph-mon[35282]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:29.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T10:44:30.227 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:30.347 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:30 smithi007 ceph-mon[30873]: pgmap v285: 97 pgs: 97 active+clean; 606 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.6 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:30.348 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:30 smithi007 ceph-mon[30873]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:30.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:30 smithi103 ceph-mon[35160]: pgmap v285: 97 pgs: 97 active+clean; 606 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.6 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:30.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:30 smithi103 ceph-mon[35160]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:30.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:30 smithi133 ceph-mon[35282]: pgmap v285: 97 pgs: 97 active+clean; 606 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.6 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:30.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:30 smithi133 ceph-mon[35282]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:30.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.691+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1624173658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd64106f70 msgr2=0x7fcd6410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.691+0000 7fcd69d7a640 1 --2- 172.21.15.7:0/1624173658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd64106f70 0x7fcd6410eab0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009ca0 tx=0x7fcd5c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.691+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1624173658 shutdown_connections 2024-09-10T10:44:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.691+0000 7fcd69d7a640 1 --2- 172.21.15.7:0/1624173658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd64106f70 0x7fcd6410eab0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.691+0000 7fcd69d7a640 1 --2- 172.21.15.7:0/1624173658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd64100eb0 0x7fcd64106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.691+0000 7fcd69d7a640 1 --2- 172.21.15.7:0/1624173658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd64100500 0x7fcd641008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.691+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1624173658 >> 172.21.15.7:0/1624173658 conn(0x7fcd640fc5c0 msgr2=0x7fcd640fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.691+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1624173658 shutdown_connections 2024-09-10T10:44:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.691+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1624173658 wait complete. 2024-09-10T10:44:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.692+0000 7fcd69d7a640 1 Processor -- start 2024-09-10T10:44:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.692+0000 7fcd69d7a640 1 -- start start 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.692+0000 7fcd69d7a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd64100500 0x7fcd641984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.692+0000 7fcd69d7a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd64100eb0 0x7fcd641989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.692+0000 7fcd69d7a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd64106f70 0x7fcd6419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.692+0000 7fcd69d7a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64112010 con 0x7fcd64106f70 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.692+0000 7fcd69d7a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64111e90 con 0x7fcd64100eb0 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.692+0000 7fcd69d7a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcd64112190 con 0x7fcd64100500 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd63fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd64106f70 0x7fcd6419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd63fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd64106f70 0x7fcd6419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49634/0 (socket says 172.21.15.7:49634) 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd63fff640 1 -- 172.21.15.7:0/1808432836 learned_addr learned my addr 172.21.15.7:0/1808432836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd62ffd640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd64100eb0 0x7fcd641989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd637fe640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd64100500 0x7fcd641984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd63fff640 1 -- 172.21.15.7:0/1808432836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd64100500 msgr2=0x7fcd641984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd63fff640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd64100500 0x7fcd641984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd63fff640 1 -- 172.21.15.7:0/1808432836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd64100eb0 msgr2=0x7fcd641989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd63fff640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd64100eb0 0x7fcd641989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd63fff640 1 -- 172.21.15.7:0/1808432836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6419d460 con 0x7fcd64106f70 2024-09-10T10:44:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd63fff640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd64106f70 0x7fcd6419cd80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c031a20 tx=0x7fcd5c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd60ff9640 1 -- 172.21.15.7:0/1808432836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c003b70 con 0x7fcd64106f70 2024-09-10T10:44:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1808432836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6419d6f0 con 0x7fcd64106f70 2024-09-10T10:44:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd60ff9640 1 -- 172.21.15.7:0/1808432836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5c003d10 con 0x7fcd64106f70 2024-09-10T10:44:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.693+0000 7fcd60ff9640 1 -- 172.21.15.7:0/1808432836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c02fc80 con 0x7fcd64106f70 2024-09-10T10:44:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.694+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1808432836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd641a5030 con 0x7fcd64106f70 2024-09-10T10:44:30.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.695+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1808432836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd641008e0 con 0x7fcd64106f70 2024-09-10T10:44:30.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.696+0000 7fcd60ff9640 1 -- 172.21.15.7:0/1808432836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fcd5c0046a0 con 0x7fcd64106f70 2024-09-10T10:44:30.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.696+0000 7fcd60ff9640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fcd4c0a2c10 0x7fcd4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.697+0000 7fcd637fe640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fcd4c0a2c10 0x7fcd4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.697+0000 7fcd60ff9640 1 -- 172.21.15.7:0/1808432836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fcd5c0f02c0 con 0x7fcd64106f70 2024-09-10T10:44:30.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.697+0000 7fcd637fe640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fcd4c0a2c10 0x7fcd4c0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcd64069fc0 tx=0x7fcd54009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:30.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.701+0000 7fcd60ff9640 1 -- 172.21.15.7:0/1808432836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd5c0bd210 con 0x7fcd64106f70 2024-09-10T10:44:30.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.906+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1808432836 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fcd64062630 con 0x7fcd4c0a2c10 2024-09-10T10:44:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.911+0000 7fcd60ff9640 1 -- 172.21.15.7:0/1808432836 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fcd64062630 con 0x7fcd4c0a2c10 2024-09-10T10:44:30.913 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_1/46e373e7-0ae7-4360-8dac-ecfc0d930e13 2024-09-10T10:44:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1808432836 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fcd4c0a2c10 msgr2=0x7fcd4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fcd4c0a2c10 0x7fcd4c0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcd64069fc0 tx=0x7fcd54009110 comp rx=0 tx=0).stop 2024-09-10T10:44:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1808432836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd64106f70 msgr2=0x7fcd6419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd64106f70 0x7fcd6419cd80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c031a20 tx=0x7fcd5c031a50 comp rx=0 tx=0).stop 2024-09-10T10:44:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1808432836 shutdown_connections 2024-09-10T10:44:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fcd4c0a2c10 0x7fcd4c0a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd64106f70 0x7fcd6419cd80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcd64100eb0 0x7fcd641989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 --2- 172.21.15.7:0/1808432836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd64100500 0x7fcd641984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1808432836 >> 172.21.15.7:0/1808432836 conn(0x7fcd640fc5c0 msgr2=0x7fcd64105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1808432836 shutdown_connections 2024-09-10T10:44:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:30.914+0000 7fcd69d7a640 1 -- 172.21.15.7:0/1808432836 wait complete. 2024-09-10T10:44:31.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:44:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:32.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:32 smithi007 ceph-mon[30873]: pgmap v286: 97 pgs: 97 active+clean; 606 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.8 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:32.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:32 smithi007 ceph-mon[30873]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:32.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.228+0000 7f815678c640 1 -- 172.21.15.7:0/3001656793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8150101de0 msgr2=0x7f81501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.228+0000 7f815678c640 1 --2- 172.21.15.7:0/3001656793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8150101de0 0x7f81501021c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8140009900 tx=0x7f814002a410 comp rx=0 tx=0).stop 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.228+0000 7f815678c640 1 -- 172.21.15.7:0/3001656793 shutdown_connections 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.228+0000 7f815678c640 1 --2- 172.21.15.7:0/3001656793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8150106f10 0x7f815010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.228+0000 7f815678c640 1 --2- 172.21.15.7:0/3001656793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8150102790 0x7f81501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.228+0000 7f815678c640 1 --2- 172.21.15.7:0/3001656793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8150101de0 0x7f81501021c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.228+0000 7f815678c640 1 -- 172.21.15.7:0/3001656793 >> 172.21.15.7:0/3001656793 conn(0x7f81500fd8c0 msgr2=0x7f81500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.228+0000 7f815678c640 1 -- 172.21.15.7:0/3001656793 shutdown_connections 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.228+0000 7f815678c640 1 -- 172.21.15.7:0/3001656793 wait complete. 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.228+0000 7f815678c640 1 Processor -- start 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f815678c640 1 -- start start 2024-09-10T10:44:32.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f815678c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8150101de0 0x7f81500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:32.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f815678c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8150102790 0x7f8150074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:32.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f815678c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8150106f10 0x7f8150074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:32.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f815678c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8150111fe0 con 0x7f8150101de0 2024-09-10T10:44:32.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f815678c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8150111e60 con 0x7f8150106f10 2024-09-10T10:44:32.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f815678c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8150112160 con 0x7f8150102790 2024-09-10T10:44:32.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f814ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8150101de0 0x7f81500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:32.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f814ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8150101de0 0x7f81500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49658/0 (socket says 172.21.15.7:49658) 2024-09-10T10:44:32.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f814ffff640 1 -- 172.21.15.7:0/3564915680 learned_addr learned my addr 172.21.15.7:0/3564915680 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:32.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f814ffff640 1 -- 172.21.15.7:0/3564915680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8150102790 msgr2=0x7f8150074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f814ffff640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8150102790 0x7f8150074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f814ffff640 1 -- 172.21.15.7:0/3564915680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8150106f10 msgr2=0x7f8150074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f8154d29640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8150106f10 0x7f8150074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:32.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f814ffff640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8150106f10 0x7f8150074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f814ffff640 1 -- 172.21.15.7:0/3564915680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81501a6920 con 0x7f8150101de0 2024-09-10T10:44:32.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f8154d29640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8150106f10 0x7f8150074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:32.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.229+0000 7f814ffff640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8150101de0 0x7f81500744f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8140004770 tx=0x7f8140003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:32.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.230+0000 7f814dffb640 1 -- 172.21.15.7:0/3564915680 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814003f070 con 0x7f8150101de0 2024-09-10T10:44:32.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.230+0000 7f814dffb640 1 -- 172.21.15.7:0/3564915680 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8140031ca0 con 0x7f8150101de0 2024-09-10T10:44:32.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.230+0000 7f815678c640 1 -- 172.21.15.7:0/3564915680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81501a6bb0 con 0x7f8150101de0 2024-09-10T10:44:32.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.230+0000 7f814dffb640 1 -- 172.21.15.7:0/3564915680 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140035030 con 0x7f8150101de0 2024-09-10T10:44:32.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.230+0000 7f815678c640 1 -- 172.21.15.7:0/3564915680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81501a7090 con 0x7f8150101de0 2024-09-10T10:44:32.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.231+0000 7f815678c640 1 -- 172.21.15.7:0/3564915680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81501021c0 con 0x7f8150101de0 2024-09-10T10:44:32.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.232+0000 7f814dffb640 1 -- 172.21.15.7:0/3564915680 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8140042070 con 0x7f8150101de0 2024-09-10T10:44:32.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.233+0000 7f814dffb640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f813c0a2c10 0x7f813c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:32.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.233+0000 7f8147fff640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f813c0a2c10 0x7f813c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:32.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.233+0000 7f814dffb640 1 -- 172.21.15.7:0/3564915680 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f81400efa60 con 0x7f8150101de0 2024-09-10T10:44:32.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.233+0000 7f8147fff640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f813c0a2c10 0x7f813c0a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8138005fd0 tx=0x7f8138005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:32.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.237+0000 7f814dffb640 1 -- 172.21.15.7:0/3564915680 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81400bc870 con 0x7f8150101de0 2024-09-10T10:44:32.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:32 smithi103 ceph-mon[35160]: pgmap v286: 97 pgs: 97 active+clean; 606 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.8 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:32.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:32 smithi103 ceph-mon[35160]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:32.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.447+0000 7f815678c640 1 -- 172.21.15.7:0/3564915680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8150109b60 con 0x7f8150101de0 2024-09-10T10:44:32.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.448+0000 7f814dffb640 1 -- 172.21.15.7:0/3564915680 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7f81400bc170 con 0x7f8150101de0 2024-09-10T10:44:32.449 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:32.450 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"btime":"2024-09-10T10:44:23:076442+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1984917528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1984917528},{"type":"v1","addr":"172.21.15.7:6839","nonce":1984917528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1887416712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1887416712},{"type":"v1","addr":"172.21.15.7:6841","nonce":1887416712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:44:11.549269+0000","modified":"2024-09-10T10:44:22.697096+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/2537001809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2537001809},{"type":"v1","addr":"172.21.15.103:6839","nonce":2537001809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24481,24482,24499]},"id":1}]} 2024-09-10T10:44:32.450 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-09-10T10:44:32.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 -- 172.21.15.7:0/3564915680 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f813c0a2c10 msgr2=0x7f813c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f813c0a2c10 0x7f813c0a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8138005fd0 tx=0x7f8138005850 comp rx=0 tx=0).stop 2024-09-10T10:44:32.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 -- 172.21.15.7:0/3564915680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8150101de0 msgr2=0x7f81500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8150101de0 0x7f81500744f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8140004770 tx=0x7f8140003c10 comp rx=0 tx=0).stop 2024-09-10T10:44:32.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 -- 172.21.15.7:0/3564915680 shutdown_connections 2024-09-10T10:44:32.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f813c0a2c10 0x7f813c0a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8150106f10 0x7f8150074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8150102790 0x7f8150074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 --2- 172.21.15.7:0/3564915680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8150101de0 0x7f81500744f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 -- 172.21.15.7:0/3564915680 >> 172.21.15.7:0/3564915680 conn(0x7f81500fd8c0 msgr2=0x7f81500fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:32.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 -- 172.21.15.7:0/3564915680 shutdown_connections 2024-09-10T10:44:32.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:32.450+0000 7f815678c640 1 -- 172.21.15.7:0/3564915680 wait complete. 2024-09-10T10:44:32.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:32 smithi133 ceph-mon[35282]: pgmap v286: 97 pgs: 97 active+clean; 606 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.8 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:32.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:32 smithi133 ceph-mon[35282]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:44:33.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:33 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3564915680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:33.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:44:33.407 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:33.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:33 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3564915680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:33.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:33 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3564915680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:44:33.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.827+0000 7f4e154b5640 1 -- 172.21.15.7:0/1096973113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e1010e480 msgr2=0x7f4e10110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.827+0000 7f4e154b5640 1 --2- 172.21.15.7:0/1096973113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e1010e480 0x7f4e10110870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009ca0 tx=0x7f4e0802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.827+0000 7f4e154b5640 1 -- 172.21.15.7:0/1096973113 shutdown_connections 2024-09-10T10:44:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.827+0000 7f4e154b5640 1 --2- 172.21.15.7:0/1096973113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e1010e480 0x7f4e10110870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.827+0000 7f4e154b5640 1 --2- 172.21.15.7:0/1096973113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e10101280 0x7f4e1010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.827+0000 7f4e154b5640 1 --2- 172.21.15.7:0/1096973113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e101008d0 0x7f4e10100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.827+0000 7f4e154b5640 1 -- 172.21.15.7:0/1096973113 >> 172.21.15.7:0/1096973113 conn(0x7f4e100fc7a0 msgr2=0x7f4e100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.827+0000 7f4e154b5640 1 -- 172.21.15.7:0/1096973113 shutdown_connections 2024-09-10T10:44:33.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.827+0000 7f4e154b5640 1 -- 172.21.15.7:0/1096973113 wait complete. 2024-09-10T10:44:33.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e154b5640 1 Processor -- start 2024-09-10T10:44:33.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e154b5640 1 -- start start 2024-09-10T10:44:33.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e154b5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e101008d0 0x7f4e1019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e154b5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e10101280 0x7f4e1019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e154b5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e1010e480 0x7f4e1019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e154b5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e101141b0 con 0x7f4e101008d0 2024-09-10T10:44:33.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e154b5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e10114030 con 0x7f4e1010e480 2024-09-10T10:44:33.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e154b5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4e10114330 con 0x7f4e10101280 2024-09-10T10:44:33.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e0effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e101008d0 0x7f4e1019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e0effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e101008d0 0x7f4e1019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49668/0 (socket says 172.21.15.7:49668) 2024-09-10T10:44:33.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e0effd640 1 -- 172.21.15.7:0/4196229562 learned_addr learned my addr 172.21.15.7:0/4196229562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:33.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e07fff640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e10101280 0x7f4e1019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.828+0000 7f4e0f7fe640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e1010e480 0x7f4e1019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e0effd640 1 -- 172.21.15.7:0/4196229562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e10101280 msgr2=0x7f4e1019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e0effd640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e10101280 0x7f4e1019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e0effd640 1 -- 172.21.15.7:0/4196229562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e1010e480 msgr2=0x7f4e1019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e0effd640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e1010e480 0x7f4e1019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e0effd640 1 -- 172.21.15.7:0/4196229562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1019f640 con 0x7f4e101008d0 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e0effd640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e101008d0 0x7f4e1019a690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000c920 tx=0x7f4e0000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e0cff9640 1 -- 172.21.15.7:0/4196229562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00007c70 con 0x7f4e101008d0 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e154b5640 1 -- 172.21.15.7:0/4196229562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1019f930 con 0x7f4e101008d0 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e0cff9640 1 -- 172.21.15.7:0/4196229562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e00007e10 con 0x7f4e101008d0 2024-09-10T10:44:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e0cff9640 1 -- 172.21.15.7:0/4196229562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00018890 con 0x7f4e101008d0 2024-09-10T10:44:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.829+0000 7f4e154b5640 1 -- 172.21.15.7:0/4196229562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e101a7210 con 0x7f4e101008d0 2024-09-10T10:44:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.830+0000 7f4e154b5640 1 -- 172.21.15.7:0/4196229562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e10100cb0 con 0x7f4e101008d0 2024-09-10T10:44:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.831+0000 7f4e0cff9640 1 -- 172.21.15.7:0/4196229562 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4e00018a70 con 0x7f4e101008d0 2024-09-10T10:44:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.832+0000 7f4e0cff9640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4dfc0a2c10 0x7f4dfc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.832+0000 7f4e07fff640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4dfc0a2c10 0x7f4dfc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.832+0000 7f4e0cff9640 1 -- 172.21.15.7:0/4196229562 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f4e000cf260 con 0x7f4e101008d0 2024-09-10T10:44:33.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.833+0000 7f4e07fff640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4dfc0a2c10 0x7f4dfc0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4e101021f0 tx=0x7f4df800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:33.836+0000 7f4e0cff9640 1 -- 172.21.15.7:0/4196229562 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e00014030 con 0x7f4e101008d0 2024-09-10T10:44:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.014+0000 7f4e154b5640 1 -- 172.21.15.7:0/4196229562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e1019b9f0 con 0x7f4e101008d0 2024-09-10T10:44:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.015+0000 7f4e0cff9640 1 -- 172.21.15.7:0/4196229562 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22646 (secure 0 0 0) 0x7f4e0009c070 con 0x7f4e101008d0 2024-09-10T10:44:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:34.017 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:36:33.117443+0000","modified":"2024-09-10T10:44:22.070795+0000","last_up_change":"2024-09-10T10:43:23.913978+0000","last_in_change":"2024-09-10T10:42:59.733844+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:39:37.104114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:07.441548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:09.970273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9239e99-3a62-4883-b2f1-8964ebb54492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6803","nonce":704452390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6805","nonce":704452390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6809","nonce":704452390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6807","nonce":704452390}]},"public_addr":"172.21.15.7:6803/704452390","cluster_addr":"172.21.15.7:6805/704452390","heartbeat_back_addr":"172.21.15.7:6809/704452390","heartbeat_front_addr":"172.21.15.7:6807/704452390","state":["exists","up"]},{"osd":1,"uuid":"d750ff16-48c8-421d-b46b-49f0d2b7f4e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6801","nonce":2409158189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6803","nonce":2409158189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6807","nonce":2409158189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6805","nonce":2409158189}]},"public_addr":"172.21.15.103:6801/2409158189","cluster_addr":"172.21.15.103:6803/2409158189","heartbeat_back_addr":"172.21.15.103:6807/2409158189","heartbeat_front_addr":"172.21.15.103:6805/2409158189","state":["exists","up"]},{"osd":2,"uuid":"1ecb070a-a2f1-499a-99e7-f1967ebb52ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6801","nonce":1657349858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6803","nonce":1657349858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6807","nonce":1657349858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6805","nonce":1657349858}]},"public_addr":"172.21.15.133:6801/1657349858","cluster_addr":"172.21.15.133:6803/1657349858","heartbeat_back_addr":"172.21.15.133:6807/1657349858","heartbeat_front_addr":"172.21.15.133:6805/1657349858","state":["exists","up"]},{"osd":3,"uuid":"8e53d79e-038f-4a7f-9ac9-16b60f47bb4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6811","nonce":871781144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6813","nonce":871781144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6817","nonce":871781144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6815","nonce":871781144}]},"public_addr":"172.21.15.7:6811/871781144","cluster_addr":"172.21.15.7:6813/871781144","heartbeat_back_addr":"172.21.15.7:6817/871781144","heartbeat_front_addr":"172.21.15.7:6815/871781144","state":["exists","up"]},{"osd":4,"uuid":"e4f04c13-594c-43dd-9a98-d2138061b1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6809","nonce":1937243043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6811","nonce":1937243043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6815","nonce":1937243043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6813","nonce":1937243043}]},"public_addr":"172.21.15.103:6809/1937243043","cluster_addr":"172.21.15.103:6811/1937243043","heartbeat_back_addr":"172.21.15.103:6815/1937243043","heartbeat_front_addr":"172.21.15.103:6813/1937243043","state":["exists","up"]},{"osd":5,"uuid":"73143170-5816-407d-ae03-dcf896057681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6809","nonce":577882774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6811","nonce":577882774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6815","nonce":577882774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6813","nonce":577882774}]},"public_addr":"172.21.15.133:6809/577882774","cluster_addr":"172.21.15.133:6811/577882774","heartbeat_back_addr":"172.21.15.133:6815/577882774","heartbeat_front_addr":"172.21.15.133:6813/577882774","state":["exists","up"]},{"osd":6,"uuid":"3a5904d5-36de-47b0-97fa-5a10b24d0c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6819","nonce":2646192734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6821","nonce":2646192734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6825","nonce":2646192734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6823","nonce":2646192734}]},"public_addr":"172.21.15.7:6819/2646192734","cluster_addr":"172.21.15.7:6821/2646192734","heartbeat_back_addr":"172.21.15.7:6825/2646192734","heartbeat_front_addr":"172.21.15.7:6823/2646192734","state":["exists","up"]},{"osd":7,"uuid":"91e79d42-decd-4405-b9f2-9480f7edfca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6817","nonce":3858867236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6819","nonce":3858867236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6823","nonce":3858867236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6821","nonce":3858867236}]},"public_addr":"172.21.15.103:6817/3858867236","cluster_addr":"172.21.15.103:6819/3858867236","heartbeat_back_addr":"172.21.15.103:6823/3858867236","heartbeat_front_addr":"172.21.15.103:6821/3858867236","state":["exists","up"]},{"osd":8,"uuid":"64373754-e81a-430c-967d-4b4f289f3473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6817","nonce":131428321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6819","nonce":131428321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6823","nonce":131428321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6821","nonce":131428321}]},"public_addr":"172.21.15.133:6817/131428321","cluster_addr":"172.21.15.133:6819/131428321","heartbeat_back_addr":"172.21.15.133:6823/131428321","heartbeat_front_addr":"172.21.15.133:6821/131428321","state":["exists","up"]},{"osd":9,"uuid":"91ebe2c9-bda3-4596-ac76-5678c71f850b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6827","nonce":945633240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6829","nonce":945633240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6833","nonce":945633240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6831","nonce":945633240}]},"public_addr":"172.21.15.7:6827/945633240","cluster_addr":"172.21.15.7:6829/945633240","heartbeat_back_addr":"172.21.15.7:6833/945633240","heartbeat_front_addr":"172.21.15.7:6831/945633240","state":["exists","up"]},{"osd":10,"uuid":"2f65568f-89d1-4a67-a222-8e263d34c710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6825","nonce":907835659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6827","nonce":907835659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6831","nonce":907835659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6829","nonce":907835659}]},"public_addr":"172.21.15.103:6825/907835659","cluster_addr":"172.21.15.103:6827/907835659","heartbeat_back_addr":"172.21.15.103:6831/907835659","heartbeat_front_addr":"172.21.15.103:6829/907835659","state":["exists","up"]},{"osd":11,"uuid":"4b49839b-32e9-4105-abee-a253f1aa3f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6825","nonce":2538184038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6827","nonce":2538184038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6831","nonce":2538184038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6829","nonce":2538184038}]},"public_addr":"172.21.15.133:6825/2538184038","cluster_addr":"172.21.15.133:6827/2538184038","heartbeat_back_addr":"172.21.15.133:6831/2538184038","heartbeat_front_addr":"172.21.15.133:6829/2538184038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:44.541279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:10.103733+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:35.290268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:01.070763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:25.666941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:50.925733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:16.486537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:41.464155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:06.610089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:31.994080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:57.000696+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:21.398669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3394895925":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6800/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/262340568":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/3768226338":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/258851660":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3411487770":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/11850255":"2024-09-11T10:36:55.407956+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 -- 172.21.15.7:0/4196229562 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4dfc0a2c10 msgr2=0x7f4dfc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4dfc0a2c10 0x7f4dfc0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4e101021f0 tx=0x7f4df800b390 comp rx=0 tx=0).stop 2024-09-10T10:44:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 -- 172.21.15.7:0/4196229562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e101008d0 msgr2=0x7f4e1019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e101008d0 0x7f4e1019a690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000c920 tx=0x7f4e0000cde0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 -- 172.21.15.7:0/4196229562 shutdown_connections 2024-09-10T10:44:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4dfc0a2c10 0x7f4dfc0a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e1010e480 0x7f4e1019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e10101280 0x7f4e1019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 --2- 172.21.15.7:0/4196229562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e101008d0 0x7f4e1019a690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 -- 172.21.15.7:0/4196229562 >> 172.21.15.7:0/4196229562 conn(0x7f4e100fc7a0 msgr2=0x7f4e100feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:34.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 -- 172.21.15.7:0/4196229562 shutdown_connections 2024-09-10T10:44:34.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:34.018+0000 7f4e154b5640 1 -- 172.21.15.7:0/4196229562 wait complete. 2024-09-10T10:44:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:34 smithi133 ceph-mon[35282]: pgmap v287: 97 pgs: 97 active+clean; 606 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.3 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:34.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:34 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4196229562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:34.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:34 smithi007 ceph-mon[30873]: pgmap v287: 97 pgs: 97 active+clean; 606 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.3 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:34.735 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:34 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4196229562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:34.764 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T10:44:34.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:44:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:34 smithi103 ceph-mon[35160]: pgmap v287: 97 pgs: 97 active+clean; 606 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.3 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:34 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4196229562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:35.021 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.453+0000 7fce2765f640 1 -- 172.21.15.7:0/1093704920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce20101de0 msgr2=0x7fce201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.453+0000 7fce2765f640 1 --2- 172.21.15.7:0/1093704920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce20101de0 0x7fce201021c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009900 tx=0x7fce1c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.453+0000 7fce2765f640 1 -- 172.21.15.7:0/1093704920 shutdown_connections 2024-09-10T10:44:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.453+0000 7fce2765f640 1 --2- 172.21.15.7:0/1093704920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce20106f10 0x7fce2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.453+0000 7fce2765f640 1 --2- 172.21.15.7:0/1093704920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce20102790 0x7fce201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.453+0000 7fce2765f640 1 --2- 172.21.15.7:0/1093704920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce20101de0 0x7fce201021c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.453+0000 7fce2765f640 1 -- 172.21.15.7:0/1093704920 >> 172.21.15.7:0/1093704920 conn(0x7fce200fd880 msgr2=0x7fce200ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.453+0000 7fce2765f640 1 -- 172.21.15.7:0/1093704920 shutdown_connections 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.453+0000 7fce2765f640 1 -- 172.21.15.7:0/1093704920 wait complete. 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.454+0000 7fce2765f640 1 Processor -- start 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.454+0000 7fce2765f640 1 -- start start 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.454+0000 7fce2765f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce20101de0 0x7fce20198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.454+0000 7fce2765f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce20102790 0x7fce20198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.454+0000 7fce2765f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce20106f10 0x7fce2019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.454+0000 7fce2765f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce20111f30 con 0x7fce20102790 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.454+0000 7fce2765f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce20111db0 con 0x7fce20106f10 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.454+0000 7fce2765f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fce201120b0 con 0x7fce20101de0 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.454+0000 7fce24bfa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce20102790 0x7fce20198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce24bfa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce20102790 0x7fce20198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49682/0 (socket says 172.21.15.7:49682) 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce24bfa640 1 -- 172.21.15.7:0/2043722385 learned_addr learned my addr 172.21.15.7:0/2043722385 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce25bfc640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce20106f10 0x7fce2019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce253fb640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce20101de0 0x7fce20198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce24bfa640 1 -- 172.21.15.7:0/2043722385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce20101de0 msgr2=0x7fce20198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce24bfa640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce20101de0 0x7fce20198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce24bfa640 1 -- 172.21.15.7:0/2043722385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce20106f10 msgr2=0x7fce2019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce24bfa640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce20106f10 0x7fce2019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce24bfa640 1 -- 172.21.15.7:0/2043722385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2019d2e0 con 0x7fce20102790 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce24bfa640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce20102790 0x7fce20198870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fce1000c920 tx=0x7fce1000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce167fc640 1 -- 172.21.15.7:0/2043722385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10007c70 con 0x7fce20102790 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce167fc640 1 -- 172.21.15.7:0/2043722385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce10007e10 con 0x7fce20102790 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce2765f640 1 -- 172.21.15.7:0/2043722385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2019d5d0 con 0x7fce20102790 2024-09-10T10:44:35.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce167fc640 1 -- 172.21.15.7:0/2043722385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10018890 con 0x7fce20102790 2024-09-10T10:44:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.455+0000 7fce2765f640 1 -- 172.21.15.7:0/2043722385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce201a4eb0 con 0x7fce20102790 2024-09-10T10:44:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.456+0000 7fce2765f640 1 -- 172.21.15.7:0/2043722385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce201021c0 con 0x7fce20102790 2024-09-10T10:44:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.457+0000 7fce167fc640 1 -- 172.21.15.7:0/2043722385 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fce10018a70 con 0x7fce20102790 2024-09-10T10:44:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.458+0000 7fce167fc640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fce0c0a2c90 0x7fce0c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.458+0000 7fce253fb640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fce0c0a2c90 0x7fce0c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.458+0000 7fce167fc640 1 -- 172.21.15.7:0/2043722385 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fce100cf260 con 0x7fce20102790 2024-09-10T10:44:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.459+0000 7fce253fb640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fce0c0a2c90 0x7fce0c0a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce1c000c00 tx=0x7fce1c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.463+0000 7fce167fc640 1 -- 172.21.15.7:0/2043722385 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce10014030 con 0x7fce20102790 2024-09-10T10:44:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.634+0000 7fce2765f640 1 -- 172.21.15.7:0/2043722385 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce2010a150 con 0x7fce0c0a2c90 2024-09-10T10:44:35.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.641+0000 7fce167fc640 1 -- 172.21.15.7:0/2043722385 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258716 (secure 0 0 0) 0x7fce2010a150 con 0x7fce0c0a2c90 2024-09-10T10:44:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:35.646 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T10:44:35.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 -- 172.21.15.7:0/2043722385 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fce0c0a2c90 msgr2=0x7fce0c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fce0c0a2c90 0x7fce0c0a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce1c000c00 tx=0x7fce1c0058d0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 -- 172.21.15.7:0/2043722385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce20102790 msgr2=0x7fce20198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce20102790 0x7fce20198870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fce1000c920 tx=0x7fce1000cde0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 -- 172.21.15.7:0/2043722385 shutdown_connections 2024-09-10T10:44:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fce0c0a2c90 0x7fce0c0a5150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce20106f10 0x7fce2019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce20102790 0x7fce20198870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 --2- 172.21.15.7:0/2043722385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce20101de0 0x7fce20198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 -- 172.21.15.7:0/2043722385 >> 172.21.15.7:0/2043722385 conn(0x7fce200fd880 msgr2=0x7fce200feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 -- 172.21.15.7:0/2043722385 shutdown_connections 2024-09-10T10:44:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:35.644+0000 7fce2765f640 1 -- 172.21.15.7:0/2043722385 wait complete. 2024-09-10T10:44:36.355 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-10T10:44:34.676340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646312,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":238,"num_read_kb":229,"num_write":340,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399104,"kb_used_data":12396,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542720,"statfs":{"total":1151940427776,"available":1151531745280,"internally_reserved":0,"allocated":12693504,"data_stored":7076230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":52862,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":100,"num_read_kb":108,"num_write":93,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002909"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078402+0000","last_change":"2024-09-10T10:44:11.040610+0000","last_active":"2024-09-10T10:44:22.078402+0000","last_peered":"2024-09-10T10:44:22.078402+0000","last_clean":"2024-09-10T10:44:22.078402+0000","last_became_active":"2024-09-10T10:44:11.040250+0000","last_became_peered":"2024-09-10T10:44:11.040250+0000","last_unstale":"2024-09-10T10:44:22.078402+0000","last_undegraded":"2024-09-10T10:44:22.078402+0000","last_fullsized":"2024-09-10T10:44:22.078402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:25.577181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079163+0000","last_change":"2024-09-10T10:44:11.041944+0000","last_active":"2024-09-10T10:44:22.079163+0000","last_peered":"2024-09-10T10:44:22.079163+0000","last_clean":"2024-09-10T10:44:22.079163+0000","last_became_active":"2024-09-10T10:44:11.041587+0000","last_became_peered":"2024-09-10T10:44:11.041587+0000","last_unstale":"2024-09-10T10:44:22.079163+0000","last_undegraded":"2024-09-10T10:44:22.079163+0000","last_fullsized":"2024-09-10T10:44:22.079163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:50.923879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079144+0000","last_change":"2024-09-10T10:44:11.038730+0000","last_active":"2024-09-10T10:44:22.079144+0000","last_peered":"2024-09-10T10:44:22.079144+0000","last_clean":"2024-09-10T10:44:22.079144+0000","last_became_active":"2024-09-10T10:44:11.038153+0000","last_became_peered":"2024-09-10T10:44:11.038153+0000","last_unstale":"2024-09-10T10:44:22.079144+0000","last_undegraded":"2024-09-10T10:44:22.079144+0000","last_fullsized":"2024-09-10T10:44:22.079144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:47.283582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092089+0000","last_change":"2024-09-10T10:44:11.038739+0000","last_active":"2024-09-10T10:44:22.092089+0000","last_peered":"2024-09-10T10:44:22.092089+0000","last_clean":"2024-09-10T10:44:22.092089+0000","last_became_active":"2024-09-10T10:44:11.038002+0000","last_became_peered":"2024-09-10T10:44:11.038002+0000","last_unstale":"2024-09-10T10:44:22.092089+0000","last_undegraded":"2024-09-10T10:44:22.092089+0000","last_fullsized":"2024-09-10T10:44:22.092089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:40.552127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083116+0000","last_change":"2024-09-10T10:44:11.037026+0000","last_active":"2024-09-10T10:44:22.083116+0000","last_peered":"2024-09-10T10:44:22.083116+0000","last_clean":"2024-09-10T10:44:22.083116+0000","last_became_active":"2024-09-10T10:44:11.036575+0000","last_became_peered":"2024-09-10T10:44:11.036575+0000","last_unstale":"2024-09-10T10:44:22.083116+0000","last_undegraded":"2024-09-10T10:44:22.083116+0000","last_fullsized":"2024-09-10T10:44:22.083116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:38.994882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082953+0000","last_change":"2024-09-10T10:44:11.036454+0000","last_active":"2024-09-10T10:44:22.082953+0000","last_peered":"2024-09-10T10:44:22.082953+0000","last_clean":"2024-09-10T10:44:22.082953+0000","last_became_active":"2024-09-10T10:44:11.034870+0000","last_became_peered":"2024-09-10T10:44:11.034870+0000","last_unstale":"2024-09-10T10:44:22.082953+0000","last_undegraded":"2024-09-10T10:44:22.082953+0000","last_fullsized":"2024-09-10T10:44:22.082953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:09.390465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081505+0000","last_change":"2024-09-10T10:44:11.042061+0000","last_active":"2024-09-10T10:44:22.081505+0000","last_peered":"2024-09-10T10:44:22.081505+0000","last_clean":"2024-09-10T10:44:22.081505+0000","last_became_active":"2024-09-10T10:44:11.041740+0000","last_became_peered":"2024-09-10T10:44:11.041740+0000","last_unstale":"2024-09-10T10:44:22.081505+0000","last_undegraded":"2024-09-10T10:44:22.081505+0000","last_fullsized":"2024-09-10T10:44:22.081505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:27.305586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153429+0000","last_change":"2024-09-10T10:44:11.041426+0000","last_active":"2024-09-10T10:44:22.153429+0000","last_peered":"2024-09-10T10:44:22.153429+0000","last_clean":"2024-09-10T10:44:22.153429+0000","last_became_active":"2024-09-10T10:44:11.041094+0000","last_became_peered":"2024-09-10T10:44:11.041094+0000","last_unstale":"2024-09-10T10:44:22.153429+0000","last_undegraded":"2024-09-10T10:44:22.153429+0000","last_fullsized":"2024-09-10T10:44:22.153429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:48.448394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085711+0000","last_change":"2024-09-10T10:44:11.035830+0000","last_active":"2024-09-10T10:44:22.085711+0000","last_peered":"2024-09-10T10:44:22.085711+0000","last_clean":"2024-09-10T10:44:22.085711+0000","last_became_active":"2024-09-10T10:44:11.034736+0000","last_became_peered":"2024-09-10T10:44:11.034736+0000","last_unstale":"2024-09-10T10:44:22.085711+0000","last_undegraded":"2024-09-10T10:44:22.085711+0000","last_fullsized":"2024-09-10T10:44:22.085711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:25.988368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082569+0000","last_change":"2024-09-10T10:44:11.040292+0000","last_active":"2024-09-10T10:44:22.082569+0000","last_peered":"2024-09-10T10:44:22.082569+0000","last_clean":"2024-09-10T10:44:22.082569+0000","last_became_active":"2024-09-10T10:44:11.039979+0000","last_became_peered":"2024-09-10T10:44:11.039979+0000","last_unstale":"2024-09-10T10:44:22.082569+0000","last_undegraded":"2024-09-10T10:44:22.082569+0000","last_fullsized":"2024-09-10T10:44:22.082569+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:02.539946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082219+0000","last_change":"2024-09-10T10:44:11.043683+0000","last_active":"2024-09-10T10:44:22.082219+0000","last_peered":"2024-09-10T10:44:22.082219+0000","last_clean":"2024-09-10T10:44:22.082219+0000","last_became_active":"2024-09-10T10:44:11.043038+0000","last_became_peered":"2024-09-10T10:44:11.043038+0000","last_unstale":"2024-09-10T10:44:22.082219+0000","last_undegraded":"2024-09-10T10:44:22.082219+0000","last_fullsized":"2024-09-10T10:44:22.082219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:25.548780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092135+0000","last_change":"2024-09-10T10:44:11.041776+0000","last_active":"2024-09-10T10:44:22.092135+0000","last_peered":"2024-09-10T10:44:22.092135+0000","last_clean":"2024-09-10T10:44:22.092135+0000","last_became_active":"2024-09-10T10:44:11.041450+0000","last_became_peered":"2024-09-10T10:44:11.041450+0000","last_unstale":"2024-09-10T10:44:22.092135+0000","last_undegraded":"2024-09-10T10:44:22.092135+0000","last_fullsized":"2024-09-10T10:44:22.092135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:01.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077361+0000","last_change":"2024-09-10T10:44:11.033973+0000","last_active":"2024-09-10T10:44:22.077361+0000","last_peered":"2024-09-10T10:44:22.077361+0000","last_clean":"2024-09-10T10:44:22.077361+0000","last_became_active":"2024-09-10T10:44:11.033649+0000","last_became_peered":"2024-09-10T10:44:11.033649+0000","last_unstale":"2024-09-10T10:44:22.077361+0000","last_undegraded":"2024-09-10T10:44:22.077361+0000","last_fullsized":"2024-09-10T10:44:22.077361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:16.671491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078786+0000","last_change":"2024-09-10T10:44:11.036931+0000","last_active":"2024-09-10T10:44:22.078786+0000","last_peered":"2024-09-10T10:44:22.078786+0000","last_clean":"2024-09-10T10:44:22.078786+0000","last_became_active":"2024-09-10T10:44:11.036574+0000","last_became_peered":"2024-09-10T10:44:11.036574+0000","last_unstale":"2024-09-10T10:44:22.078786+0000","last_undegraded":"2024-09-10T10:44:22.078786+0000","last_fullsized":"2024-09-10T10:44:22.078786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:57.908420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078875+0000","last_change":"2024-09-10T10:44:11.037701+0000","last_active":"2024-09-10T10:44:22.078875+0000","last_peered":"2024-09-10T10:44:22.078875+0000","last_clean":"2024-09-10T10:44:22.078875+0000","last_became_active":"2024-09-10T10:44:11.037417+0000","last_became_peered":"2024-09-10T10:44:11.037417+0000","last_unstale":"2024-09-10T10:44:22.078875+0000","last_undegraded":"2024-09-10T10:44:22.078875+0000","last_fullsized":"2024-09-10T10:44:22.078875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:35.584226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084980+0000","last_change":"2024-09-10T10:44:11.038545+0000","last_active":"2024-09-10T10:44:22.084980+0000","last_peered":"2024-09-10T10:44:22.084980+0000","last_clean":"2024-09-10T10:44:22.084980+0000","last_became_active":"2024-09-10T10:44:11.037574+0000","last_became_peered":"2024-09-10T10:44:11.037574+0000","last_unstale":"2024-09-10T10:44:22.084980+0000","last_undegraded":"2024-09-10T10:44:22.084980+0000","last_fullsized":"2024-09-10T10:44:22.084980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:13.471862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078510+0000","last_change":"2024-09-10T10:44:11.042231+0000","last_active":"2024-09-10T10:44:22.078510+0000","last_peered":"2024-09-10T10:44:22.078510+0000","last_clean":"2024-09-10T10:44:22.078510+0000","last_became_active":"2024-09-10T10:44:11.041764+0000","last_became_peered":"2024-09-10T10:44:11.041764+0000","last_unstale":"2024-09-10T10:44:22.078510+0000","last_undegraded":"2024-09-10T10:44:22.078510+0000","last_fullsized":"2024-09-10T10:44:22.078510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:38.301271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080361+0000","last_change":"2024-09-10T10:44:11.044492+0000","last_active":"2024-09-10T10:44:22.080361+0000","last_peered":"2024-09-10T10:44:22.080361+0000","last_clean":"2024-09-10T10:44:22.080361+0000","last_became_active":"2024-09-10T10:44:11.044135+0000","last_became_peered":"2024-09-10T10:44:11.044135+0000","last_unstale":"2024-09-10T10:44:22.080361+0000","last_undegraded":"2024-09-10T10:44:22.080361+0000","last_fullsized":"2024-09-10T10:44:22.080361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:47.054076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081417+0000","last_change":"2024-09-10T10:44:11.040349+0000","last_active":"2024-09-10T10:44:22.081417+0000","last_peered":"2024-09-10T10:44:22.081417+0000","last_clean":"2024-09-10T10:44:22.081417+0000","last_became_active":"2024-09-10T10:44:11.040051+0000","last_became_peered":"2024-09-10T10:44:11.040051+0000","last_unstale":"2024-09-10T10:44:22.081417+0000","last_undegraded":"2024-09-10T10:44:22.081417+0000","last_fullsized":"2024-09-10T10:44:22.081417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:33.777529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085576+0000","last_change":"2024-09-10T10:44:22.085576+0000","last_active":"2024-09-10T10:44:22.085576+0000","last_peered":"2024-09-10T10:44:22.085576+0000","last_clean":"2024-09-10T10:44:22.085576+0000","last_became_active":"2024-09-10T10:44:22.085135+0000","last_became_peered":"2024-09-10T10:44:22.085135+0000","last_unstale":"2024-09-10T10:44:22.085576+0000","last_undegraded":"2024-09-10T10:44:22.085576+0000","last_fullsized":"2024-09-10T10:44:22.085576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077458+0000","last_change":"2024-09-10T10:44:11.030324+0000","last_active":"2024-09-10T10:44:22.077458+0000","last_peered":"2024-09-10T10:44:22.077458+0000","last_clean":"2024-09-10T10:44:22.077458+0000","last_became_active":"2024-09-10T10:44:11.029652+0000","last_became_peered":"2024-09-10T10:44:11.029652+0000","last_unstale":"2024-09-10T10:44:22.077458+0000","last_undegraded":"2024-09-10T10:44:22.077458+0000","last_fullsized":"2024-09-10T10:44:22.077458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:51.332548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085112+0000","last_change":"2024-09-10T10:44:11.040637+0000","last_active":"2024-09-10T10:44:22.085112+0000","last_peered":"2024-09-10T10:44:22.085112+0000","last_clean":"2024-09-10T10:44:22.085112+0000","last_became_active":"2024-09-10T10:44:11.040345+0000","last_became_peered":"2024-09-10T10:44:11.040345+0000","last_unstale":"2024-09-10T10:44:22.085112+0000","last_undegraded":"2024-09-10T10:44:22.085112+0000","last_fullsized":"2024-09-10T10:44:22.085112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:18.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153476+0000","last_change":"2024-09-10T10:44:11.043157+0000","last_active":"2024-09-10T10:44:22.153476+0000","last_peered":"2024-09-10T10:44:22.153476+0000","last_clean":"2024-09-10T10:44:22.153476+0000","last_became_active":"2024-09-10T10:44:11.042823+0000","last_became_peered":"2024-09-10T10:44:11.042823+0000","last_unstale":"2024-09-10T10:44:22.153476+0000","last_undegraded":"2024-09-10T10:44:22.153476+0000","last_fullsized":"2024-09-10T10:44:22.153476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:23.988072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080356+0000","last_change":"2024-09-10T10:44:11.041918+0000","last_active":"2024-09-10T10:44:22.080356+0000","last_peered":"2024-09-10T10:44:22.080356+0000","last_clean":"2024-09-10T10:44:22.080356+0000","last_became_active":"2024-09-10T10:44:11.040111+0000","last_became_peered":"2024-09-10T10:44:11.040111+0000","last_unstale":"2024-09-10T10:44:22.080356+0000","last_undegraded":"2024-09-10T10:44:22.080356+0000","last_fullsized":"2024-09-10T10:44:22.080356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:24.999485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092270+0000","last_change":"2024-09-10T10:44:22.091959+0000","last_active":"2024-09-10T10:44:22.092270+0000","last_peered":"2024-09-10T10:44:22.092270+0000","last_clean":"2024-09-10T10:44:22.092270+0000","last_became_active":"2024-09-10T10:44:22.091707+0000","last_became_peered":"2024-09-10T10:44:22.091707+0000","last_unstale":"2024-09-10T10:44:22.092270+0000","last_undegraded":"2024-09-10T10:44:22.092270+0000","last_fullsized":"2024-09-10T10:44:22.092270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:38.454868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081432+0000","last_change":"2024-09-10T10:44:11.034339+0000","last_active":"2024-09-10T10:44:22.081432+0000","last_peered":"2024-09-10T10:44:22.081432+0000","last_clean":"2024-09-10T10:44:22.081432+0000","last_became_active":"2024-09-10T10:44:11.034022+0000","last_became_peered":"2024-09-10T10:44:11.034022+0000","last_unstale":"2024-09-10T10:44:22.081432+0000","last_undegraded":"2024-09-10T10:44:22.081432+0000","last_fullsized":"2024-09-10T10:44:22.081432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:54.122244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078068+0000","last_change":"2024-09-10T10:44:11.037871+0000","last_active":"2024-09-10T10:44:22.078068+0000","last_peered":"2024-09-10T10:44:22.078068+0000","last_clean":"2024-09-10T10:44:22.078068+0000","last_became_active":"2024-09-10T10:44:11.037372+0000","last_became_peered":"2024-09-10T10:44:11.037372+0000","last_unstale":"2024-09-10T10:44:22.078068+0000","last_undegraded":"2024-09-10T10:44:22.078068+0000","last_fullsized":"2024-09-10T10:44:22.078068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:22.296078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077383+0000","last_change":"2024-09-10T10:44:11.034826+0000","last_active":"2024-09-10T10:44:22.077383+0000","last_peered":"2024-09-10T10:44:22.077383+0000","last_clean":"2024-09-10T10:44:22.077383+0000","last_became_active":"2024-09-10T10:44:11.034520+0000","last_became_peered":"2024-09-10T10:44:11.034520+0000","last_unstale":"2024-09-10T10:44:22.077383+0000","last_undegraded":"2024-09-10T10:44:22.077383+0000","last_fullsized":"2024-09-10T10:44:22.077383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:13.131060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085136+0000","last_change":"2024-09-10T10:44:22.085136+0000","last_active":"2024-09-10T10:44:22.085136+0000","last_peered":"2024-09-10T10:44:22.085136+0000","last_clean":"2024-09-10T10:44:22.085136+0000","last_became_active":"2024-09-10T10:44:22.084718+0000","last_became_peered":"2024-09-10T10:44:22.084718+0000","last_unstale":"2024-09-10T10:44:22.085136+0000","last_undegraded":"2024-09-10T10:44:22.085136+0000","last_fullsized":"2024-09-10T10:44:22.085136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:31.165424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081062+0000","last_change":"2024-09-10T10:44:11.042336+0000","last_active":"2024-09-10T10:44:22.081062+0000","last_peered":"2024-09-10T10:44:22.081062+0000","last_clean":"2024-09-10T10:44:22.081062+0000","last_became_active":"2024-09-10T10:44:11.041127+0000","last_became_peered":"2024-09-10T10:44:11.041127+0000","last_unstale":"2024-09-10T10:44:22.081062+0000","last_undegraded":"2024-09-10T10:44:22.081062+0000","last_fullsized":"2024-09-10T10:44:22.081062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:50.349201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081158+0000","last_change":"2024-09-10T10:44:11.043188+0000","last_active":"2024-09-10T10:44:22.081158+0000","last_peered":"2024-09-10T10:44:22.081158+0000","last_clean":"2024-09-10T10:44:22.081158+0000","last_became_active":"2024-09-10T10:44:11.042735+0000","last_became_peered":"2024-09-10T10:44:11.042735+0000","last_unstale":"2024-09-10T10:44:22.081158+0000","last_undegraded":"2024-09-10T10:44:22.081158+0000","last_fullsized":"2024-09-10T10:44:22.081158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:50.090810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077961+0000","last_change":"2024-09-10T10:44:11.041339+0000","last_active":"2024-09-10T10:44:22.077961+0000","last_peered":"2024-09-10T10:44:22.077961+0000","last_clean":"2024-09-10T10:44:22.077961+0000","last_became_active":"2024-09-10T10:44:11.038011+0000","last_became_peered":"2024-09-10T10:44:11.038011+0000","last_unstale":"2024-09-10T10:44:22.077961+0000","last_undegraded":"2024-09-10T10:44:22.077961+0000","last_fullsized":"2024-09-10T10:44:22.077961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:47.521465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084488+0000","last_change":"2024-09-10T10:44:22.084488+0000","last_active":"2024-09-10T10:44:22.084488+0000","last_peered":"2024-09-10T10:44:22.084488+0000","last_clean":"2024-09-10T10:44:22.084488+0000","last_became_active":"2024-09-10T10:44:22.084062+0000","last_became_peered":"2024-09-10T10:44:22.084062+0000","last_unstale":"2024-09-10T10:44:22.084488+0000","last_undegraded":"2024-09-10T10:44:22.084488+0000","last_fullsized":"2024-09-10T10:44:22.084488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:57.739261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:32.103226+0000","last_change":"2024-09-10T10:44:09.024716+0000","last_active":"2024-09-10T10:44:32.103226+0000","last_peered":"2024-09-10T10:44:32.103226+0000","last_clean":"2024-09-10T10:44:32.103226+0000","last_became_active":"2024-09-10T10:44:09.023901+0000","last_became_peered":"2024-09-10T10:44:09.023901+0000","last_unstale":"2024-09-10T10:44:32.103226+0000","last_undegraded":"2024-09-10T10:44:32.103226+0000","last_fullsized":"2024-09-10T10:44:32.103226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:28.901896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080568+0000","last_change":"2024-09-10T10:44:11.044838+0000","last_active":"2024-09-10T10:44:22.080568+0000","last_peered":"2024-09-10T10:44:22.080568+0000","last_clean":"2024-09-10T10:44:22.080568+0000","last_became_active":"2024-09-10T10:44:11.044542+0000","last_became_peered":"2024-09-10T10:44:11.044542+0000","last_unstale":"2024-09-10T10:44:22.080568+0000","last_undegraded":"2024-09-10T10:44:22.080568+0000","last_fullsized":"2024-09-10T10:44:22.080568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:02.753914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082979+0000","last_change":"2024-09-10T10:44:09.025799+0000","last_active":"2024-09-10T10:44:22.082979+0000","last_peered":"2024-09-10T10:44:22.082979+0000","last_clean":"2024-09-10T10:44:22.082979+0000","last_became_active":"2024-09-10T10:44:09.025421+0000","last_became_peered":"2024-09-10T10:44:09.025421+0000","last_unstale":"2024-09-10T10:44:22.082979+0000","last_undegraded":"2024-09-10T10:44:22.082979+0000","last_fullsized":"2024-09-10T10:44:22.082979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:11.928663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077144+0000","last_change":"2024-09-10T10:44:09.017640+0000","last_active":"2024-09-10T10:44:22.077144+0000","last_peered":"2024-09-10T10:44:22.077144+0000","last_clean":"2024-09-10T10:44:22.077144+0000","last_became_active":"2024-09-10T10:44:09.017286+0000","last_became_peered":"2024-09-10T10:44:09.017286+0000","last_unstale":"2024-09-10T10:44:22.077144+0000","last_undegraded":"2024-09-10T10:44:22.077144+0000","last_fullsized":"2024-09-10T10:44:22.077144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:59.678607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153461+0000","last_change":"2024-09-10T10:44:11.038225+0000","last_active":"2024-09-10T10:44:22.153461+0000","last_peered":"2024-09-10T10:44:22.153461+0000","last_clean":"2024-09-10T10:44:22.153461+0000","last_became_active":"2024-09-10T10:44:11.037924+0000","last_became_peered":"2024-09-10T10:44:11.037924+0000","last_unstale":"2024-09-10T10:44:22.153461+0000","last_undegraded":"2024-09-10T10:44:22.153461+0000","last_fullsized":"2024-09-10T10:44:22.153461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:58.787166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080948+0000","last_change":"2024-09-10T10:44:09.024676+0000","last_active":"2024-09-10T10:44:22.080948+0000","last_peered":"2024-09-10T10:44:22.080948+0000","last_clean":"2024-09-10T10:44:22.080948+0000","last_became_active":"2024-09-10T10:44:09.023970+0000","last_became_peered":"2024-09-10T10:44:09.023970+0000","last_unstale":"2024-09-10T10:44:22.080948+0000","last_undegraded":"2024-09-10T10:44:22.080948+0000","last_fullsized":"2024-09-10T10:44:22.080948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:40.296260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080161+0000","last_change":"2024-09-10T10:44:11.040330+0000","last_active":"2024-09-10T10:44:22.080161+0000","last_peered":"2024-09-10T10:44:22.080161+0000","last_clean":"2024-09-10T10:44:22.080161+0000","last_became_active":"2024-09-10T10:44:11.040018+0000","last_became_peered":"2024-09-10T10:44:11.040018+0000","last_unstale":"2024-09-10T10:44:22.080161+0000","last_undegraded":"2024-09-10T10:44:22.080161+0000","last_fullsized":"2024-09-10T10:44:22.080161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:24.403070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:32.102696+0000","last_change":"2024-09-10T10:44:09.021488+0000","last_active":"2024-09-10T10:44:32.102696+0000","last_peered":"2024-09-10T10:44:32.102696+0000","last_clean":"2024-09-10T10:44:32.102696+0000","last_became_active":"2024-09-10T10:44:09.021088+0000","last_became_peered":"2024-09-10T10:44:09.021088+0000","last_unstale":"2024-09-10T10:44:32.102696+0000","last_undegraded":"2024-09-10T10:44:32.102696+0000","last_fullsized":"2024-09-10T10:44:32.102696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:37.990090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085498+0000","last_change":"2024-09-10T10:44:11.045142+0000","last_active":"2024-09-10T10:44:22.085498+0000","last_peered":"2024-09-10T10:44:22.085498+0000","last_clean":"2024-09-10T10:44:22.085498+0000","last_became_active":"2024-09-10T10:44:11.044770+0000","last_became_peered":"2024-09-10T10:44:11.044770+0000","last_unstale":"2024-09-10T10:44:22.085498+0000","last_undegraded":"2024-09-10T10:44:22.085498+0000","last_fullsized":"2024-09-10T10:44:22.085498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:10.731504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.096348+0000","last_change":"2024-09-10T10:44:22.096348+0000","last_active":"2024-09-10T10:44:22.096348+0000","last_peered":"2024-09-10T10:44:22.096348+0000","last_clean":"2024-09-10T10:44:22.096348+0000","last_became_active":"2024-09-10T10:44:22.092921+0000","last_became_peered":"2024-09-10T10:44:22.092921+0000","last_unstale":"2024-09-10T10:44:22.096348+0000","last_undegraded":"2024-09-10T10:44:22.096348+0000","last_fullsized":"2024-09-10T10:44:22.096348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:15.721849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079428+0000","last_change":"2024-09-10T10:44:11.034499+0000","last_active":"2024-09-10T10:44:22.079428+0000","last_peered":"2024-09-10T10:44:22.079428+0000","last_clean":"2024-09-10T10:44:22.079428+0000","last_became_active":"2024-09-10T10:44:11.034030+0000","last_became_peered":"2024-09-10T10:44:11.034030+0000","last_unstale":"2024-09-10T10:44:22.079428+0000","last_undegraded":"2024-09-10T10:44:22.079428+0000","last_fullsized":"2024-09-10T10:44:22.079428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:57.313000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080739+0000","last_change":"2024-09-10T10:44:09.142509+0000","last_active":"2024-09-10T10:44:22.080739+0000","last_peered":"2024-09-10T10:44:22.080739+0000","last_clean":"2024-09-10T10:44:22.080739+0000","last_became_active":"2024-09-10T10:44:09.142048+0000","last_became_peered":"2024-09-10T10:44:09.142048+0000","last_unstale":"2024-09-10T10:44:22.080739+0000","last_undegraded":"2024-09-10T10:44:22.080739+0000","last_fullsized":"2024-09-10T10:44:22.080739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:57.937241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085180+0000","last_change":"2024-09-10T10:44:11.034851+0000","last_active":"2024-09-10T10:44:22.085180+0000","last_peered":"2024-09-10T10:44:22.085180+0000","last_clean":"2024-09-10T10:44:22.085180+0000","last_became_active":"2024-09-10T10:44:11.034524+0000","last_became_peered":"2024-09-10T10:44:11.034524+0000","last_unstale":"2024-09-10T10:44:22.085180+0000","last_undegraded":"2024-09-10T10:44:22.085180+0000","last_fullsized":"2024-09-10T10:44:22.085180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:06.356958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'26","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:33.116162+0000","last_change":"2024-09-10T10:44:09.024313+0000","last_active":"2024-09-10T10:44:33.116162+0000","last_peered":"2024-09-10T10:44:33.116162+0000","last_clean":"2024-09-10T10:44:33.116162+0000","last_became_active":"2024-09-10T10:44:09.023933+0000","last_became_peered":"2024-09-10T10:44:09.023933+0000","last_unstale":"2024-09-10T10:44:33.116162+0000","last_undegraded":"2024-09-10T10:44:33.116162+0000","last_fullsized":"2024-09-10T10:44:33.116162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:37.522786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52100,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":65,"num_write":27,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083275+0000","last_change":"2024-09-10T10:44:11.039171+0000","last_active":"2024-09-10T10:44:22.083275+0000","last_peered":"2024-09-10T10:44:22.083275+0000","last_clean":"2024-09-10T10:44:22.083275+0000","last_became_active":"2024-09-10T10:44:11.038833+0000","last_became_peered":"2024-09-10T10:44:11.038833+0000","last_unstale":"2024-09-10T10:44:22.083275+0000","last_undegraded":"2024-09-10T10:44:22.083275+0000","last_fullsized":"2024-09-10T10:44:22.083275+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:37.115626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081645+0000","last_change":"2024-09-10T10:44:09.021368+0000","last_active":"2024-09-10T10:44:22.081645+0000","last_peered":"2024-09-10T10:44:22.081645+0000","last_clean":"2024-09-10T10:44:22.081645+0000","last_became_active":"2024-09-10T10:44:09.020805+0000","last_became_peered":"2024-09-10T10:44:09.020805+0000","last_unstale":"2024-09-10T10:44:22.081645+0000","last_undegraded":"2024-09-10T10:44:22.081645+0000","last_fullsized":"2024-09-10T10:44:22.081645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:20.408776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077226+0000","last_change":"2024-09-10T10:44:11.037539+0000","last_active":"2024-09-10T10:44:22.077226+0000","last_peered":"2024-09-10T10:44:22.077226+0000","last_clean":"2024-09-10T10:44:22.077226+0000","last_became_active":"2024-09-10T10:44:11.036607+0000","last_became_peered":"2024-09-10T10:44:11.036607+0000","last_unstale":"2024-09-10T10:44:22.077226+0000","last_undegraded":"2024-09-10T10:44:22.077226+0000","last_fullsized":"2024-09-10T10:44:22.077226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:01:46.029665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080495+0000","last_change":"2024-09-10T10:44:09.023766+0000","last_active":"2024-09-10T10:44:22.080495+0000","last_peered":"2024-09-10T10:44:22.080495+0000","last_clean":"2024-09-10T10:44:22.080495+0000","last_became_active":"2024-09-10T10:44:09.023422+0000","last_became_peered":"2024-09-10T10:44:09.023422+0000","last_unstale":"2024-09-10T10:44:22.080495+0000","last_undegraded":"2024-09-10T10:44:22.080495+0000","last_fullsized":"2024-09-10T10:44:22.080495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:56.009610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082817+0000","last_change":"2024-09-10T10:44:11.038846+0000","last_active":"2024-09-10T10:44:22.082817+0000","last_peered":"2024-09-10T10:44:22.082817+0000","last_clean":"2024-09-10T10:44:22.082817+0000","last_became_active":"2024-09-10T10:44:11.038477+0000","last_became_peered":"2024-09-10T10:44:11.038477+0000","last_unstale":"2024-09-10T10:44:22.082817+0000","last_undegraded":"2024-09-10T10:44:22.082817+0000","last_fullsized":"2024-09-10T10:44:22.082817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:26.255612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'3","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:32.105971+0000","last_change":"2024-09-10T10:44:09.023394+0000","last_active":"2024-09-10T10:44:32.105971+0000","last_peered":"2024-09-10T10:44:32.105971+0000","last_clean":"2024-09-10T10:44:32.105971+0000","last_became_active":"2024-09-10T10:44:09.023049+0000","last_became_peered":"2024-09-10T10:44:09.023049+0000","last_unstale":"2024-09-10T10:44:32.105971+0000","last_undegraded":"2024-09-10T10:44:32.105971+0000","last_fullsized":"2024-09-10T10:44:32.105971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:00.024671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081712+0000","last_change":"2024-09-10T10:44:11.041162+0000","last_active":"2024-09-10T10:44:22.081712+0000","last_peered":"2024-09-10T10:44:22.081712+0000","last_clean":"2024-09-10T10:44:22.081712+0000","last_became_active":"2024-09-10T10:44:11.040848+0000","last_became_peered":"2024-09-10T10:44:11.040848+0000","last_unstale":"2024-09-10T10:44:22.081712+0000","last_undegraded":"2024-09-10T10:44:22.081712+0000","last_fullsized":"2024-09-10T10:44:22.081712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:13.044447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082834+0000","last_change":"2024-09-10T10:44:09.142490+0000","last_active":"2024-09-10T10:44:22.082834+0000","last_peered":"2024-09-10T10:44:22.082834+0000","last_clean":"2024-09-10T10:44:22.082834+0000","last_became_active":"2024-09-10T10:44:09.141983+0000","last_became_peered":"2024-09-10T10:44:09.141983+0000","last_unstale":"2024-09-10T10:44:22.082834+0000","last_undegraded":"2024-09-10T10:44:22.082834+0000","last_fullsized":"2024-09-10T10:44:22.082834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:34.481241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080333+0000","last_change":"2024-09-10T10:44:11.042109+0000","last_active":"2024-09-10T10:44:22.080333+0000","last_peered":"2024-09-10T10:44:22.080333+0000","last_clean":"2024-09-10T10:44:22.080333+0000","last_became_active":"2024-09-10T10:44:11.041154+0000","last_became_peered":"2024-09-10T10:44:11.041154+0000","last_unstale":"2024-09-10T10:44:22.080333+0000","last_undegraded":"2024-09-10T10:44:22.080333+0000","last_fullsized":"2024-09-10T10:44:22.080333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:50.134773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083174+0000","last_change":"2024-09-10T10:44:09.019117+0000","last_active":"2024-09-10T10:44:22.083174+0000","last_peered":"2024-09-10T10:44:22.083174+0000","last_clean":"2024-09-10T10:44:22.083174+0000","last_became_active":"2024-09-10T10:44:09.018733+0000","last_became_peered":"2024-09-10T10:44:09.018733+0000","last_unstale":"2024-09-10T10:44:22.083174+0000","last_undegraded":"2024-09-10T10:44:22.083174+0000","last_fullsized":"2024-09-10T10:44:22.083174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:40.049342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078300+0000","last_change":"2024-09-10T10:44:11.040381+0000","last_active":"2024-09-10T10:44:22.078300+0000","last_peered":"2024-09-10T10:44:22.078300+0000","last_clean":"2024-09-10T10:44:22.078300+0000","last_became_active":"2024-09-10T10:44:11.040053+0000","last_became_peered":"2024-09-10T10:44:11.040053+0000","last_unstale":"2024-09-10T10:44:22.078300+0000","last_undegraded":"2024-09-10T10:44:22.078300+0000","last_fullsized":"2024-09-10T10:44:22.078300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:29.043171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077403+0000","last_change":"2024-09-10T10:44:09.018173+0000","last_active":"2024-09-10T10:44:22.077403+0000","last_peered":"2024-09-10T10:44:22.077403+0000","last_clean":"2024-09-10T10:44:22.077403+0000","last_became_active":"2024-09-10T10:44:09.017790+0000","last_became_peered":"2024-09-10T10:44:09.017790+0000","last_unstale":"2024-09-10T10:44:22.077403+0000","last_undegraded":"2024-09-10T10:44:22.077403+0000","last_fullsized":"2024-09-10T10:44:22.077403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:43.515112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082583+0000","last_change":"2024-09-10T10:44:11.035293+0000","last_active":"2024-09-10T10:44:22.082583+0000","last_peered":"2024-09-10T10:44:22.082583+0000","last_clean":"2024-09-10T10:44:22.082583+0000","last_became_active":"2024-09-10T10:44:11.034958+0000","last_became_peered":"2024-09-10T10:44:11.034958+0000","last_unstale":"2024-09-10T10:44:22.082583+0000","last_undegraded":"2024-09-10T10:44:22.082583+0000","last_fullsized":"2024-09-10T10:44:22.082583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:09.397229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080789+0000","last_change":"2024-09-10T10:44:09.021322+0000","last_active":"2024-09-10T10:44:22.080789+0000","last_peered":"2024-09-10T10:44:22.080789+0000","last_clean":"2024-09-10T10:44:22.080789+0000","last_became_active":"2024-09-10T10:44:09.021018+0000","last_became_peered":"2024-09-10T10:44:09.021018+0000","last_unstale":"2024-09-10T10:44:22.080789+0000","last_undegraded":"2024-09-10T10:44:22.080789+0000","last_fullsized":"2024-09-10T10:44:22.080789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:48.999125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082090+0000","last_change":"2024-09-10T10:44:11.040545+0000","last_active":"2024-09-10T10:44:22.082090+0000","last_peered":"2024-09-10T10:44:22.082090+0000","last_clean":"2024-09-10T10:44:22.082090+0000","last_became_active":"2024-09-10T10:44:11.040042+0000","last_became_peered":"2024-09-10T10:44:11.040042+0000","last_unstale":"2024-09-10T10:44:22.082090+0000","last_undegraded":"2024-09-10T10:44:22.082090+0000","last_fullsized":"2024-09-10T10:44:22.082090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:14.367771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081581+0000","last_change":"2024-09-10T10:44:09.142944+0000","last_active":"2024-09-10T10:44:22.081581+0000","last_peered":"2024-09-10T10:44:22.081581+0000","last_clean":"2024-09-10T10:44:22.081581+0000","last_became_active":"2024-09-10T10:44:09.142550+0000","last_became_peered":"2024-09-10T10:44:09.142550+0000","last_unstale":"2024-09-10T10:44:22.081581+0000","last_undegraded":"2024-09-10T10:44:22.081581+0000","last_fullsized":"2024-09-10T10:44:22.081581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:56.247251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082331+0000","last_change":"2024-09-10T10:44:11.038205+0000","last_active":"2024-09-10T10:44:22.082331+0000","last_peered":"2024-09-10T10:44:22.082331+0000","last_clean":"2024-09-10T10:44:22.082331+0000","last_became_active":"2024-09-10T10:44:11.037817+0000","last_became_peered":"2024-09-10T10:44:11.037817+0000","last_unstale":"2024-09-10T10:44:22.082331+0000","last_undegraded":"2024-09-10T10:44:22.082331+0000","last_fullsized":"2024-09-10T10:44:22.082331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:39.221897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":343,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081904+0000","last_change":"2024-09-10T10:43:01.771132+0000","last_active":"2024-09-10T10:44:22.081904+0000","last_peered":"2024-09-10T10:44:22.081904+0000","last_clean":"2024-09-10T10:44:22.081904+0000","last_became_active":"2024-09-10T10:43:01.770568+0000","last_became_peered":"2024-09-10T10:43:01.770568+0000","last_unstale":"2024-09-10T10:44:22.081904+0000","last_undegraded":"2024-09-10T10:44:22.081904+0000","last_fullsized":"2024-09-10T10:44:22.081904+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:39:37.547474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:39:37.547474+0000","last_clean_scrub_stamp":"2024-09-10T10:39:37.547474+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:09.699899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.091961+0000","last_change":"2024-09-10T10:44:09.021115+0000","last_active":"2024-09-10T10:44:22.091961+0000","last_peered":"2024-09-10T10:44:22.091961+0000","last_clean":"2024-09-10T10:44:22.091961+0000","last_became_active":"2024-09-10T10:44:09.020747+0000","last_became_peered":"2024-09-10T10:44:09.020747+0000","last_unstale":"2024-09-10T10:44:22.091961+0000","last_undegraded":"2024-09-10T10:44:22.091961+0000","last_fullsized":"2024-09-10T10:44:22.091961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:15.828535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153652+0000","last_change":"2024-09-10T10:44:11.046484+0000","last_active":"2024-09-10T10:44:22.153652+0000","last_peered":"2024-09-10T10:44:22.153652+0000","last_clean":"2024-09-10T10:44:22.153652+0000","last_became_active":"2024-09-10T10:44:11.046142+0000","last_became_peered":"2024-09-10T10:44:11.046142+0000","last_unstale":"2024-09-10T10:44:22.153652+0000","last_undegraded":"2024-09-10T10:44:22.153652+0000","last_fullsized":"2024-09-10T10:44:22.153652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:15.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.088392+0000","last_change":"2024-09-10T10:44:22.087926+0000","last_active":"2024-09-10T10:44:22.088392+0000","last_peered":"2024-09-10T10:44:22.088392+0000","last_clean":"2024-09-10T10:44:22.088392+0000","last_became_active":"2024-09-10T10:44:22.084248+0000","last_became_peered":"2024-09-10T10:44:22.084248+0000","last_unstale":"2024-09-10T10:44:22.088392+0000","last_undegraded":"2024-09-10T10:44:22.088392+0000","last_fullsized":"2024-09-10T10:44:22.088392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:15.774609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077128+0000","last_change":"2024-09-10T10:44:11.034105+0000","last_active":"2024-09-10T10:44:22.077128+0000","last_peered":"2024-09-10T10:44:22.077128+0000","last_clean":"2024-09-10T10:44:22.077128+0000","last_became_active":"2024-09-10T10:44:11.033792+0000","last_became_peered":"2024-09-10T10:44:11.033792+0000","last_unstale":"2024-09-10T10:44:22.077128+0000","last_undegraded":"2024-09-10T10:44:22.077128+0000","last_fullsized":"2024-09-10T10:44:22.077128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.131963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:33.106609+0000","last_change":"2024-09-10T10:44:09.142514+0000","last_active":"2024-09-10T10:44:33.106609+0000","last_peered":"2024-09-10T10:44:33.106609+0000","last_clean":"2024-09-10T10:44:33.106609+0000","last_became_active":"2024-09-10T10:44:09.142115+0000","last_became_peered":"2024-09-10T10:44:09.142115+0000","last_unstale":"2024-09-10T10:44:33.106609+0000","last_undegraded":"2024-09-10T10:44:33.106609+0000","last_fullsized":"2024-09-10T10:44:33.106609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:37.520105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080531+0000","last_change":"2024-09-10T10:44:11.040122+0000","last_active":"2024-09-10T10:44:22.080531+0000","last_peered":"2024-09-10T10:44:22.080531+0000","last_clean":"2024-09-10T10:44:22.080531+0000","last_became_active":"2024-09-10T10:44:11.039649+0000","last_became_peered":"2024-09-10T10:44:11.039649+0000","last_unstale":"2024-09-10T10:44:22.080531+0000","last_undegraded":"2024-09-10T10:44:22.080531+0000","last_fullsized":"2024-09-10T10:44:22.080531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:26.401981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083151+0000","last_change":"2024-09-10T10:44:09.024933+0000","last_active":"2024-09-10T10:44:22.083151+0000","last_peered":"2024-09-10T10:44:22.083151+0000","last_clean":"2024-09-10T10:44:22.083151+0000","last_became_active":"2024-09-10T10:44:09.024579+0000","last_became_peered":"2024-09-10T10:44:09.024579+0000","last_unstale":"2024-09-10T10:44:22.083151+0000","last_undegraded":"2024-09-10T10:44:22.083151+0000","last_fullsized":"2024-09-10T10:44:22.083151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:45.644932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084217+0000","last_change":"2024-09-10T10:44:22.084217+0000","last_active":"2024-09-10T10:44:22.084217+0000","last_peered":"2024-09-10T10:44:22.084217+0000","last_clean":"2024-09-10T10:44:22.084217+0000","last_became_active":"2024-09-10T10:44:22.083707+0000","last_became_peered":"2024-09-10T10:44:22.083707+0000","last_unstale":"2024-09-10T10:44:22.084217+0000","last_undegraded":"2024-09-10T10:44:22.084217+0000","last_fullsized":"2024-09-10T10:44:22.084217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:30.124690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080402+0000","last_change":"2024-09-10T10:44:09.142462+0000","last_active":"2024-09-10T10:44:22.080402+0000","last_peered":"2024-09-10T10:44:22.080402+0000","last_clean":"2024-09-10T10:44:22.080402+0000","last_became_active":"2024-09-10T10:44:09.142086+0000","last_became_peered":"2024-09-10T10:44:09.142086+0000","last_unstale":"2024-09-10T10:44:22.080402+0000","last_undegraded":"2024-09-10T10:44:22.080402+0000","last_fullsized":"2024-09-10T10:44:22.080402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:28.019899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082676+0000","last_change":"2024-09-10T10:44:11.036928+0000","last_active":"2024-09-10T10:44:22.082676+0000","last_peered":"2024-09-10T10:44:22.082676+0000","last_clean":"2024-09-10T10:44:22.082676+0000","last_became_active":"2024-09-10T10:44:11.036626+0000","last_became_peered":"2024-09-10T10:44:11.036626+0000","last_unstale":"2024-09-10T10:44:22.082676+0000","last_undegraded":"2024-09-10T10:44:22.082676+0000","last_fullsized":"2024-09-10T10:44:22.082676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:42.624867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083407+0000","last_change":"2024-09-10T10:44:09.142521+0000","last_active":"2024-09-10T10:44:22.083407+0000","last_peered":"2024-09-10T10:44:22.083407+0000","last_clean":"2024-09-10T10:44:22.083407+0000","last_became_active":"2024-09-10T10:44:09.142061+0000","last_became_peered":"2024-09-10T10:44:09.142061+0000","last_unstale":"2024-09-10T10:44:22.083407+0000","last_undegraded":"2024-09-10T10:44:22.083407+0000","last_fullsized":"2024-09-10T10:44:22.083407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:14.054388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077489+0000","last_change":"2024-09-10T10:44:11.031953+0000","last_active":"2024-09-10T10:44:22.077489+0000","last_peered":"2024-09-10T10:44:22.077489+0000","last_clean":"2024-09-10T10:44:22.077489+0000","last_became_active":"2024-09-10T10:44:11.031610+0000","last_became_peered":"2024-09-10T10:44:11.031610+0000","last_unstale":"2024-09-10T10:44:22.077489+0000","last_undegraded":"2024-09-10T10:44:22.077489+0000","last_fullsized":"2024-09-10T10:44:22.077489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:03.305651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:33.029846+0000","last_change":"2024-09-10T10:44:09.016752+0000","last_active":"2024-09-10T10:44:33.029846+0000","last_peered":"2024-09-10T10:44:33.029846+0000","last_clean":"2024-09-10T10:44:33.029846+0000","last_became_active":"2024-09-10T10:44:09.016464+0000","last_became_peered":"2024-09-10T10:44:09.016464+0000","last_unstale":"2024-09-10T10:44:33.029846+0000","last_undegraded":"2024-09-10T10:44:33.029846+0000","last_fullsized":"2024-09-10T10:44:33.029846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:25.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153747+0000","last_change":"2024-09-10T10:44:11.044793+0000","last_active":"2024-09-10T10:44:22.153747+0000","last_peered":"2024-09-10T10:44:22.153747+0000","last_clean":"2024-09-10T10:44:22.153747+0000","last_became_active":"2024-09-10T10:44:11.043611+0000","last_became_peered":"2024-09-10T10:44:11.043611+0000","last_unstale":"2024-09-10T10:44:22.153747+0000","last_undegraded":"2024-09-10T10:44:22.153747+0000","last_fullsized":"2024-09-10T10:44:22.153747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:20.235527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.091934+0000","last_change":"2024-09-10T10:44:09.017385+0000","last_active":"2024-09-10T10:44:22.091934+0000","last_peered":"2024-09-10T10:44:22.091934+0000","last_clean":"2024-09-10T10:44:22.091934+0000","last_became_active":"2024-09-10T10:44:09.017033+0000","last_became_peered":"2024-09-10T10:44:09.017033+0000","last_unstale":"2024-09-10T10:44:22.091934+0000","last_undegraded":"2024-09-10T10:44:22.091934+0000","last_fullsized":"2024-09-10T10:44:22.091934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:57.963765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079263+0000","last_change":"2024-09-10T10:44:11.039194+0000","last_active":"2024-09-10T10:44:22.079263+0000","last_peered":"2024-09-10T10:44:22.079263+0000","last_clean":"2024-09-10T10:44:22.079263+0000","last_became_active":"2024-09-10T10:44:11.038290+0000","last_became_peered":"2024-09-10T10:44:11.038290+0000","last_unstale":"2024-09-10T10:44:22.079263+0000","last_undegraded":"2024-09-10T10:44:22.079263+0000","last_fullsized":"2024-09-10T10:44:22.079263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:50.938124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085070+0000","last_change":"2024-09-10T10:44:09.142618+0000","last_active":"2024-09-10T10:44:22.085070+0000","last_peered":"2024-09-10T10:44:22.085070+0000","last_clean":"2024-09-10T10:44:22.085070+0000","last_became_active":"2024-09-10T10:44:09.142232+0000","last_became_peered":"2024-09-10T10:44:09.142232+0000","last_unstale":"2024-09-10T10:44:22.085070+0000","last_undegraded":"2024-09-10T10:44:22.085070+0000","last_fullsized":"2024-09-10T10:44:22.085070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:18.797080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082392+0000","last_change":"2024-09-10T10:44:11.039288+0000","last_active":"2024-09-10T10:44:22.082392+0000","last_peered":"2024-09-10T10:44:22.082392+0000","last_clean":"2024-09-10T10:44:22.082392+0000","last_became_active":"2024-09-10T10:44:11.038955+0000","last_became_peered":"2024-09-10T10:44:11.038955+0000","last_unstale":"2024-09-10T10:44:22.082392+0000","last_undegraded":"2024-09-10T10:44:22.082392+0000","last_fullsized":"2024-09-10T10:44:22.082392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:29.564274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153587+0000","last_change":"2024-09-10T10:44:09.020638+0000","last_active":"2024-09-10T10:44:22.153587+0000","last_peered":"2024-09-10T10:44:22.153587+0000","last_clean":"2024-09-10T10:44:22.153587+0000","last_became_active":"2024-09-10T10:44:09.020320+0000","last_became_peered":"2024-09-10T10:44:09.020320+0000","last_unstale":"2024-09-10T10:44:22.153587+0000","last_undegraded":"2024-09-10T10:44:22.153587+0000","last_fullsized":"2024-09-10T10:44:22.153587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:00.793392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078085+0000","last_change":"2024-09-10T10:44:11.034010+0000","last_active":"2024-09-10T10:44:22.078085+0000","last_peered":"2024-09-10T10:44:22.078085+0000","last_clean":"2024-09-10T10:44:22.078085+0000","last_became_active":"2024-09-10T10:44:11.033639+0000","last_became_peered":"2024-09-10T10:44:11.033639+0000","last_unstale":"2024-09-10T10:44:22.078085+0000","last_undegraded":"2024-09-10T10:44:22.078085+0000","last_fullsized":"2024-09-10T10:44:22.078085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:20.127131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081171+0000","last_change":"2024-09-10T10:44:09.019478+0000","last_active":"2024-09-10T10:44:22.081171+0000","last_peered":"2024-09-10T10:44:22.081171+0000","last_clean":"2024-09-10T10:44:22.081171+0000","last_became_active":"2024-09-10T10:44:09.019033+0000","last_became_peered":"2024-09-10T10:44:09.019033+0000","last_unstale":"2024-09-10T10:44:22.081171+0000","last_undegraded":"2024-09-10T10:44:22.081171+0000","last_fullsized":"2024-09-10T10:44:22.081171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:17.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153647+0000","last_change":"2024-09-10T10:44:11.036118+0000","last_active":"2024-09-10T10:44:22.153647+0000","last_peered":"2024-09-10T10:44:22.153647+0000","last_clean":"2024-09-10T10:44:22.153647+0000","last_became_active":"2024-09-10T10:44:11.035648+0000","last_became_peered":"2024-09-10T10:44:11.035648+0000","last_unstale":"2024-09-10T10:44:22.153647+0000","last_undegraded":"2024-09-10T10:44:22.153647+0000","last_fullsized":"2024-09-10T10:44:22.153647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:52.134207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.100063+0000","last_change":"2024-09-10T10:44:22.099643+0000","last_active":"2024-09-10T10:44:22.100063+0000","last_peered":"2024-09-10T10:44:22.100063+0000","last_clean":"2024-09-10T10:44:22.100063+0000","last_became_active":"2024-09-10T10:44:22.093961+0000","last_became_peered":"2024-09-10T10:44:22.093961+0000","last_unstale":"2024-09-10T10:44:22.100063+0000","last_undegraded":"2024-09-10T10:44:22.100063+0000","last_fullsized":"2024-09-10T10:44:22.100063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:11.958415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082997+0000","last_change":"2024-09-10T10:44:11.035067+0000","last_active":"2024-09-10T10:44:22.082997+0000","last_peered":"2024-09-10T10:44:22.082997+0000","last_clean":"2024-09-10T10:44:22.082997+0000","last_became_active":"2024-09-10T10:44:11.034756+0000","last_became_peered":"2024-09-10T10:44:11.034756+0000","last_unstale":"2024-09-10T10:44:22.082997+0000","last_undegraded":"2024-09-10T10:44:22.082997+0000","last_fullsized":"2024-09-10T10:44:22.082997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:20.425948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080613+0000","last_change":"2024-09-10T10:44:11.039774+0000","last_active":"2024-09-10T10:44:22.080613+0000","last_peered":"2024-09-10T10:44:22.080613+0000","last_clean":"2024-09-10T10:44:22.080613+0000","last_became_active":"2024-09-10T10:44:11.039433+0000","last_became_peered":"2024-09-10T10:44:11.039433+0000","last_unstale":"2024-09-10T10:44:22.080613+0000","last_undegraded":"2024-09-10T10:44:22.080613+0000","last_fullsized":"2024-09-10T10:44:22.080613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:17.743436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083623+0000","last_change":"2024-09-10T10:44:09.022130+0000","last_active":"2024-09-10T10:44:22.083623+0000","last_peered":"2024-09-10T10:44:22.083623+0000","last_clean":"2024-09-10T10:44:22.083623+0000","last_became_active":"2024-09-10T10:44:09.021769+0000","last_became_peered":"2024-09-10T10:44:09.021769+0000","last_unstale":"2024-09-10T10:44:22.083623+0000","last_undegraded":"2024-09-10T10:44:22.083623+0000","last_fullsized":"2024-09-10T10:44:22.083623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:03.215955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085154+0000","last_change":"2024-09-10T10:44:11.044147+0000","last_active":"2024-09-10T10:44:22.085154+0000","last_peered":"2024-09-10T10:44:22.085154+0000","last_clean":"2024-09-10T10:44:22.085154+0000","last_became_active":"2024-09-10T10:44:11.042545+0000","last_became_peered":"2024-09-10T10:44:11.042545+0000","last_unstale":"2024-09-10T10:44:22.085154+0000","last_undegraded":"2024-09-10T10:44:22.085154+0000","last_fullsized":"2024-09-10T10:44:22.085154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:59.360624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083361+0000","last_change":"2024-09-10T10:44:09.023396+0000","last_active":"2024-09-10T10:44:22.083361+0000","last_peered":"2024-09-10T10:44:22.083361+0000","last_clean":"2024-09-10T10:44:22.083361+0000","last_became_active":"2024-09-10T10:44:09.022876+0000","last_became_peered":"2024-09-10T10:44:09.022876+0000","last_unstale":"2024-09-10T10:44:22.083361+0000","last_undegraded":"2024-09-10T10:44:22.083361+0000","last_fullsized":"2024-09-10T10:44:22.083361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:01.901011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079435+0000","last_change":"2024-09-10T10:44:11.033338+0000","last_active":"2024-09-10T10:44:22.079435+0000","last_peered":"2024-09-10T10:44:22.079435+0000","last_clean":"2024-09-10T10:44:22.079435+0000","last_became_active":"2024-09-10T10:44:11.032941+0000","last_became_peered":"2024-09-10T10:44:11.032941+0000","last_unstale":"2024-09-10T10:44:22.079435+0000","last_undegraded":"2024-09-10T10:44:22.079435+0000","last_fullsized":"2024-09-10T10:44:22.079435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:13.787799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.154016+0000","last_change":"2024-09-10T10:44:09.143814+0000","last_active":"2024-09-10T10:44:22.154016+0000","last_peered":"2024-09-10T10:44:22.154016+0000","last_clean":"2024-09-10T10:44:22.154016+0000","last_became_active":"2024-09-10T10:44:09.141942+0000","last_became_peered":"2024-09-10T10:44:09.141942+0000","last_unstale":"2024-09-10T10:44:22.154016+0000","last_undegraded":"2024-09-10T10:44:22.154016+0000","last_fullsized":"2024-09-10T10:44:22.154016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:31.839457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153764+0000","last_change":"2024-09-10T10:44:11.041386+0000","last_active":"2024-09-10T10:44:22.153764+0000","last_peered":"2024-09-10T10:44:22.153764+0000","last_clean":"2024-09-10T10:44:22.153764+0000","last_became_active":"2024-09-10T10:44:11.041060+0000","last_became_peered":"2024-09-10T10:44:11.041060+0000","last_unstale":"2024-09-10T10:44:22.153764+0000","last_undegraded":"2024-09-10T10:44:22.153764+0000","last_fullsized":"2024-09-10T10:44:22.153764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:38.528977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080586+0000","last_change":"2024-09-10T10:44:09.023375+0000","last_active":"2024-09-10T10:44:22.080586+0000","last_peered":"2024-09-10T10:44:22.080586+0000","last_clean":"2024-09-10T10:44:22.080586+0000","last_became_active":"2024-09-10T10:44:09.023082+0000","last_became_peered":"2024-09-10T10:44:09.023082+0000","last_unstale":"2024-09-10T10:44:22.080586+0000","last_undegraded":"2024-09-10T10:44:22.080586+0000","last_fullsized":"2024-09-10T10:44:22.080586+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:51.685194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":55710,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":102,"num_read_kb":110,"num_write":157,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":145060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":880640,"data_stored":430529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841557,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1568768,"data_stored":1074110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005083,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":430765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":430685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495653,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1490944,"data_stored":1022224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626474,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40368,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704784,"statfs":{"total":95995035648,"available":95953698816,"internally_reserved":0,"allocated":884736,"data_stored":430154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855407,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1021908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986228,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":888832,"data_stored":430591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":430465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":472362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":471012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":905216,"data_stored":431425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":41065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:44:36.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:44:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:36.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:36 smithi007 ceph-mon[30873]: pgmap v288: 97 pgs: 97 active+clean; 631 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:36.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:36 smithi133 ceph-mon[35282]: pgmap v288: 97 pgs: 97 active+clean; 631 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:36 smithi103 ceph-mon[35160]: pgmap v288: 97 pgs: 97 active+clean; 631 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.0 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T10:44:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.100+0000 7fb869d1e640 1 -- 172.21.15.7:0/2715957628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8641082a0 msgr2=0x7fb86410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.100+0000 7fb869d1e640 1 --2- 172.21.15.7:0/2715957628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8641082a0 0x7fb86410ec40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009cd0 tx=0x7fb85c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.100+0000 7fb869d1e640 1 -- 172.21.15.7:0/2715957628 shutdown_connections 2024-09-10T10:44:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.100+0000 7fb869d1e640 1 --2- 172.21.15.7:0/2715957628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8641082a0 0x7fb86410ec40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.100+0000 7fb869d1e640 1 --2- 172.21.15.7:0/2715957628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8641036e0 0x7fb864107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.100+0000 7fb869d1e640 1 --2- 172.21.15.7:0/2715957628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb864102d30 0x7fb864103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.100+0000 7fb869d1e640 1 -- 172.21.15.7:0/2715957628 >> 172.21.15.7:0/2715957628 conn(0x7fb8640fc7d0 msgr2=0x7fb8640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.100+0000 7fb869d1e640 1 -- 172.21.15.7:0/2715957628 shutdown_connections 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.100+0000 7fb869d1e640 1 -- 172.21.15.7:0/2715957628 wait complete. 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb869d1e640 1 Processor -- start 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb869d1e640 1 -- start start 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb869d1e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb864102d30 0x7fb864198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb869d1e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8641036e0 0x7fb864198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb869d1e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8641082a0 0x7fb86419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb869d1e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641120a0 con 0x7fb8641082a0 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb869d1e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb864111f20 con 0x7fb864102d30 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb869d1e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb864112220 con 0x7fb8641036e0 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb863fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8641082a0 0x7fb86419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb863fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8641082a0 0x7fb86419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49698/0 (socket says 172.21.15.7:49698) 2024-09-10T10:44:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb863fff640 1 -- 172.21.15.7:0/84538417 learned_addr learned my addr 172.21.15.7:0/84538417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb85bfff640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8641036e0 0x7fb864198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb8637fe640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb864102d30 0x7fb864198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb863fff640 1 -- 172.21.15.7:0/84538417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8641036e0 msgr2=0x7fb864198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb863fff640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8641036e0 0x7fb864198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb863fff640 1 -- 172.21.15.7:0/84538417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb864102d30 msgr2=0x7fb864198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb863fff640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb864102d30 0x7fb864198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.101+0000 7fb863fff640 1 -- 172.21.15.7:0/84538417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86419d4f0 con 0x7fb8641082a0 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.102+0000 7fb863fff640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8641082a0 0x7fb86419ce10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005cc0 tx=0x7fb85c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.102+0000 7fb8617fa640 1 -- 172.21.15.7:0/84538417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c004040 con 0x7fb8641082a0 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.102+0000 7fb8617fa640 1 -- 172.21.15.7:0/84538417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb85c0046c0 con 0x7fb8641082a0 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.102+0000 7fb869d1e640 1 -- 172.21.15.7:0/84538417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86419d780 con 0x7fb8641082a0 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.102+0000 7fb8617fa640 1 -- 172.21.15.7:0/84538417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c030e00 con 0x7fb8641082a0 2024-09-10T10:44:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.102+0000 7fb869d1e640 1 -- 172.21.15.7:0/84538417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8641a50c0 con 0x7fb8641082a0 2024-09-10T10:44:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.103+0000 7fb869d1e640 1 -- 172.21.15.7:0/84538417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb864103110 con 0x7fb8641082a0 2024-09-10T10:44:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.104+0000 7fb8617fa640 1 -- 172.21.15.7:0/84538417 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb85c0305a0 con 0x7fb8641082a0 2024-09-10T10:44:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.105+0000 7fb8617fa640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb8500a2c90 0x7fb8500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.105+0000 7fb8637fe640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb8500a2c90 0x7fb8500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.105+0000 7fb8617fa640 1 -- 172.21.15.7:0/84538417 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fb85c02e030 con 0x7fb8641082a0 2024-09-10T10:44:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.106+0000 7fb8637fe640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb8500a2c90 0x7fb8500a5150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb854009b60 tx=0x7fb854005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:37.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.109+0000 7fb8617fa640 1 -- 172.21.15.7:0/84538417 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb85c04b060 con 0x7fb8641082a0 2024-09-10T10:44:37.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.282+0000 7fb869d1e640 1 -- 172.21.15.7:0/84538417 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb864107420 con 0x7fb8500a2c90 2024-09-10T10:44:37.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.289+0000 7fb8617fa640 1 -- 172.21.15.7:0/84538417 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258710 (secure 0 0 0) 0x7fb864107420 con 0x7fb8500a2c90 2024-09-10T10:44:37.293 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:37.294 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T10:44:37.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 -- 172.21.15.7:0/84538417 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb8500a2c90 msgr2=0x7fb8500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb8500a2c90 0x7fb8500a5150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb854009b60 tx=0x7fb854005f90 comp rx=0 tx=0).stop 2024-09-10T10:44:37.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 -- 172.21.15.7:0/84538417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8641082a0 msgr2=0x7fb86419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8641082a0 0x7fb86419ce10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005cc0 tx=0x7fb85c032f40 comp rx=0 tx=0).stop 2024-09-10T10:44:37.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 -- 172.21.15.7:0/84538417 shutdown_connections 2024-09-10T10:44:37.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb8500a2c90 0x7fb8500a5150 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8641082a0 0x7fb86419ce10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8641036e0 0x7fb864198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 --2- 172.21.15.7:0/84538417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb864102d30 0x7fb864198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 -- 172.21.15.7:0/84538417 >> 172.21.15.7:0/84538417 conn(0x7fb8640fc7d0 msgr2=0x7fb864105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 -- 172.21.15.7:0/84538417 shutdown_connections 2024-09-10T10:44:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:37.292+0000 7fb869d1e640 1 -- 172.21.15.7:0/84538417 wait complete. 2024-09-10T10:44:37.593 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:37 smithi007 ceph-mon[30873]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:37.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:37 smithi133 ceph-mon[35282]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:37 smithi103 ceph-mon[35160]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:38.017 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-10T10:44:36.676867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646312,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":238,"num_read_kb":229,"num_write":340,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399104,"kb_used_data":12396,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542720,"statfs":{"total":1151940427776,"available":1151531745280,"internally_reserved":0,"allocated":12693504,"data_stored":7076230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50722,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":81,"num_write":34,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003044"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078402+0000","last_change":"2024-09-10T10:44:11.040610+0000","last_active":"2024-09-10T10:44:22.078402+0000","last_peered":"2024-09-10T10:44:22.078402+0000","last_clean":"2024-09-10T10:44:22.078402+0000","last_became_active":"2024-09-10T10:44:11.040250+0000","last_became_peered":"2024-09-10T10:44:11.040250+0000","last_unstale":"2024-09-10T10:44:22.078402+0000","last_undegraded":"2024-09-10T10:44:22.078402+0000","last_fullsized":"2024-09-10T10:44:22.078402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:25.577181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079163+0000","last_change":"2024-09-10T10:44:11.041944+0000","last_active":"2024-09-10T10:44:22.079163+0000","last_peered":"2024-09-10T10:44:22.079163+0000","last_clean":"2024-09-10T10:44:22.079163+0000","last_became_active":"2024-09-10T10:44:11.041587+0000","last_became_peered":"2024-09-10T10:44:11.041587+0000","last_unstale":"2024-09-10T10:44:22.079163+0000","last_undegraded":"2024-09-10T10:44:22.079163+0000","last_fullsized":"2024-09-10T10:44:22.079163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:50.923879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079144+0000","last_change":"2024-09-10T10:44:11.038730+0000","last_active":"2024-09-10T10:44:22.079144+0000","last_peered":"2024-09-10T10:44:22.079144+0000","last_clean":"2024-09-10T10:44:22.079144+0000","last_became_active":"2024-09-10T10:44:11.038153+0000","last_became_peered":"2024-09-10T10:44:11.038153+0000","last_unstale":"2024-09-10T10:44:22.079144+0000","last_undegraded":"2024-09-10T10:44:22.079144+0000","last_fullsized":"2024-09-10T10:44:22.079144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:47.283582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092089+0000","last_change":"2024-09-10T10:44:11.038739+0000","last_active":"2024-09-10T10:44:22.092089+0000","last_peered":"2024-09-10T10:44:22.092089+0000","last_clean":"2024-09-10T10:44:22.092089+0000","last_became_active":"2024-09-10T10:44:11.038002+0000","last_became_peered":"2024-09-10T10:44:11.038002+0000","last_unstale":"2024-09-10T10:44:22.092089+0000","last_undegraded":"2024-09-10T10:44:22.092089+0000","last_fullsized":"2024-09-10T10:44:22.092089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:40.552127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083116+0000","last_change":"2024-09-10T10:44:11.037026+0000","last_active":"2024-09-10T10:44:22.083116+0000","last_peered":"2024-09-10T10:44:22.083116+0000","last_clean":"2024-09-10T10:44:22.083116+0000","last_became_active":"2024-09-10T10:44:11.036575+0000","last_became_peered":"2024-09-10T10:44:11.036575+0000","last_unstale":"2024-09-10T10:44:22.083116+0000","last_undegraded":"2024-09-10T10:44:22.083116+0000","last_fullsized":"2024-09-10T10:44:22.083116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:38.994882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082953+0000","last_change":"2024-09-10T10:44:11.036454+0000","last_active":"2024-09-10T10:44:22.082953+0000","last_peered":"2024-09-10T10:44:22.082953+0000","last_clean":"2024-09-10T10:44:22.082953+0000","last_became_active":"2024-09-10T10:44:11.034870+0000","last_became_peered":"2024-09-10T10:44:11.034870+0000","last_unstale":"2024-09-10T10:44:22.082953+0000","last_undegraded":"2024-09-10T10:44:22.082953+0000","last_fullsized":"2024-09-10T10:44:22.082953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:09.390465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081505+0000","last_change":"2024-09-10T10:44:11.042061+0000","last_active":"2024-09-10T10:44:22.081505+0000","last_peered":"2024-09-10T10:44:22.081505+0000","last_clean":"2024-09-10T10:44:22.081505+0000","last_became_active":"2024-09-10T10:44:11.041740+0000","last_became_peered":"2024-09-10T10:44:11.041740+0000","last_unstale":"2024-09-10T10:44:22.081505+0000","last_undegraded":"2024-09-10T10:44:22.081505+0000","last_fullsized":"2024-09-10T10:44:22.081505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:27.305586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153429+0000","last_change":"2024-09-10T10:44:11.041426+0000","last_active":"2024-09-10T10:44:22.153429+0000","last_peered":"2024-09-10T10:44:22.153429+0000","last_clean":"2024-09-10T10:44:22.153429+0000","last_became_active":"2024-09-10T10:44:11.041094+0000","last_became_peered":"2024-09-10T10:44:11.041094+0000","last_unstale":"2024-09-10T10:44:22.153429+0000","last_undegraded":"2024-09-10T10:44:22.153429+0000","last_fullsized":"2024-09-10T10:44:22.153429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:48.448394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085711+0000","last_change":"2024-09-10T10:44:11.035830+0000","last_active":"2024-09-10T10:44:22.085711+0000","last_peered":"2024-09-10T10:44:22.085711+0000","last_clean":"2024-09-10T10:44:22.085711+0000","last_became_active":"2024-09-10T10:44:11.034736+0000","last_became_peered":"2024-09-10T10:44:11.034736+0000","last_unstale":"2024-09-10T10:44:22.085711+0000","last_undegraded":"2024-09-10T10:44:22.085711+0000","last_fullsized":"2024-09-10T10:44:22.085711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:25.988368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082569+0000","last_change":"2024-09-10T10:44:11.040292+0000","last_active":"2024-09-10T10:44:22.082569+0000","last_peered":"2024-09-10T10:44:22.082569+0000","last_clean":"2024-09-10T10:44:22.082569+0000","last_became_active":"2024-09-10T10:44:11.039979+0000","last_became_peered":"2024-09-10T10:44:11.039979+0000","last_unstale":"2024-09-10T10:44:22.082569+0000","last_undegraded":"2024-09-10T10:44:22.082569+0000","last_fullsized":"2024-09-10T10:44:22.082569+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:02.539946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082219+0000","last_change":"2024-09-10T10:44:11.043683+0000","last_active":"2024-09-10T10:44:22.082219+0000","last_peered":"2024-09-10T10:44:22.082219+0000","last_clean":"2024-09-10T10:44:22.082219+0000","last_became_active":"2024-09-10T10:44:11.043038+0000","last_became_peered":"2024-09-10T10:44:11.043038+0000","last_unstale":"2024-09-10T10:44:22.082219+0000","last_undegraded":"2024-09-10T10:44:22.082219+0000","last_fullsized":"2024-09-10T10:44:22.082219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:25.548780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092135+0000","last_change":"2024-09-10T10:44:11.041776+0000","last_active":"2024-09-10T10:44:22.092135+0000","last_peered":"2024-09-10T10:44:22.092135+0000","last_clean":"2024-09-10T10:44:22.092135+0000","last_became_active":"2024-09-10T10:44:11.041450+0000","last_became_peered":"2024-09-10T10:44:11.041450+0000","last_unstale":"2024-09-10T10:44:22.092135+0000","last_undegraded":"2024-09-10T10:44:22.092135+0000","last_fullsized":"2024-09-10T10:44:22.092135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:01.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077361+0000","last_change":"2024-09-10T10:44:11.033973+0000","last_active":"2024-09-10T10:44:22.077361+0000","last_peered":"2024-09-10T10:44:22.077361+0000","last_clean":"2024-09-10T10:44:22.077361+0000","last_became_active":"2024-09-10T10:44:11.033649+0000","last_became_peered":"2024-09-10T10:44:11.033649+0000","last_unstale":"2024-09-10T10:44:22.077361+0000","last_undegraded":"2024-09-10T10:44:22.077361+0000","last_fullsized":"2024-09-10T10:44:22.077361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:16.671491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078786+0000","last_change":"2024-09-10T10:44:11.036931+0000","last_active":"2024-09-10T10:44:22.078786+0000","last_peered":"2024-09-10T10:44:22.078786+0000","last_clean":"2024-09-10T10:44:22.078786+0000","last_became_active":"2024-09-10T10:44:11.036574+0000","last_became_peered":"2024-09-10T10:44:11.036574+0000","last_unstale":"2024-09-10T10:44:22.078786+0000","last_undegraded":"2024-09-10T10:44:22.078786+0000","last_fullsized":"2024-09-10T10:44:22.078786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:57.908420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078875+0000","last_change":"2024-09-10T10:44:11.037701+0000","last_active":"2024-09-10T10:44:22.078875+0000","last_peered":"2024-09-10T10:44:22.078875+0000","last_clean":"2024-09-10T10:44:22.078875+0000","last_became_active":"2024-09-10T10:44:11.037417+0000","last_became_peered":"2024-09-10T10:44:11.037417+0000","last_unstale":"2024-09-10T10:44:22.078875+0000","last_undegraded":"2024-09-10T10:44:22.078875+0000","last_fullsized":"2024-09-10T10:44:22.078875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:35.584226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084980+0000","last_change":"2024-09-10T10:44:11.038545+0000","last_active":"2024-09-10T10:44:22.084980+0000","last_peered":"2024-09-10T10:44:22.084980+0000","last_clean":"2024-09-10T10:44:22.084980+0000","last_became_active":"2024-09-10T10:44:11.037574+0000","last_became_peered":"2024-09-10T10:44:11.037574+0000","last_unstale":"2024-09-10T10:44:22.084980+0000","last_undegraded":"2024-09-10T10:44:22.084980+0000","last_fullsized":"2024-09-10T10:44:22.084980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:13.471862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078510+0000","last_change":"2024-09-10T10:44:11.042231+0000","last_active":"2024-09-10T10:44:22.078510+0000","last_peered":"2024-09-10T10:44:22.078510+0000","last_clean":"2024-09-10T10:44:22.078510+0000","last_became_active":"2024-09-10T10:44:11.041764+0000","last_became_peered":"2024-09-10T10:44:11.041764+0000","last_unstale":"2024-09-10T10:44:22.078510+0000","last_undegraded":"2024-09-10T10:44:22.078510+0000","last_fullsized":"2024-09-10T10:44:22.078510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:38.301271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080361+0000","last_change":"2024-09-10T10:44:11.044492+0000","last_active":"2024-09-10T10:44:22.080361+0000","last_peered":"2024-09-10T10:44:22.080361+0000","last_clean":"2024-09-10T10:44:22.080361+0000","last_became_active":"2024-09-10T10:44:11.044135+0000","last_became_peered":"2024-09-10T10:44:11.044135+0000","last_unstale":"2024-09-10T10:44:22.080361+0000","last_undegraded":"2024-09-10T10:44:22.080361+0000","last_fullsized":"2024-09-10T10:44:22.080361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:47.054076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081417+0000","last_change":"2024-09-10T10:44:11.040349+0000","last_active":"2024-09-10T10:44:22.081417+0000","last_peered":"2024-09-10T10:44:22.081417+0000","last_clean":"2024-09-10T10:44:22.081417+0000","last_became_active":"2024-09-10T10:44:11.040051+0000","last_became_peered":"2024-09-10T10:44:11.040051+0000","last_unstale":"2024-09-10T10:44:22.081417+0000","last_undegraded":"2024-09-10T10:44:22.081417+0000","last_fullsized":"2024-09-10T10:44:22.081417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:33.777529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085576+0000","last_change":"2024-09-10T10:44:22.085576+0000","last_active":"2024-09-10T10:44:22.085576+0000","last_peered":"2024-09-10T10:44:22.085576+0000","last_clean":"2024-09-10T10:44:22.085576+0000","last_became_active":"2024-09-10T10:44:22.085135+0000","last_became_peered":"2024-09-10T10:44:22.085135+0000","last_unstale":"2024-09-10T10:44:22.085576+0000","last_undegraded":"2024-09-10T10:44:22.085576+0000","last_fullsized":"2024-09-10T10:44:22.085576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077458+0000","last_change":"2024-09-10T10:44:11.030324+0000","last_active":"2024-09-10T10:44:22.077458+0000","last_peered":"2024-09-10T10:44:22.077458+0000","last_clean":"2024-09-10T10:44:22.077458+0000","last_became_active":"2024-09-10T10:44:11.029652+0000","last_became_peered":"2024-09-10T10:44:11.029652+0000","last_unstale":"2024-09-10T10:44:22.077458+0000","last_undegraded":"2024-09-10T10:44:22.077458+0000","last_fullsized":"2024-09-10T10:44:22.077458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:51.332548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085112+0000","last_change":"2024-09-10T10:44:11.040637+0000","last_active":"2024-09-10T10:44:22.085112+0000","last_peered":"2024-09-10T10:44:22.085112+0000","last_clean":"2024-09-10T10:44:22.085112+0000","last_became_active":"2024-09-10T10:44:11.040345+0000","last_became_peered":"2024-09-10T10:44:11.040345+0000","last_unstale":"2024-09-10T10:44:22.085112+0000","last_undegraded":"2024-09-10T10:44:22.085112+0000","last_fullsized":"2024-09-10T10:44:22.085112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:18.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153476+0000","last_change":"2024-09-10T10:44:11.043157+0000","last_active":"2024-09-10T10:44:22.153476+0000","last_peered":"2024-09-10T10:44:22.153476+0000","last_clean":"2024-09-10T10:44:22.153476+0000","last_became_active":"2024-09-10T10:44:11.042823+0000","last_became_peered":"2024-09-10T10:44:11.042823+0000","last_unstale":"2024-09-10T10:44:22.153476+0000","last_undegraded":"2024-09-10T10:44:22.153476+0000","last_fullsized":"2024-09-10T10:44:22.153476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:23.988072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080356+0000","last_change":"2024-09-10T10:44:11.041918+0000","last_active":"2024-09-10T10:44:22.080356+0000","last_peered":"2024-09-10T10:44:22.080356+0000","last_clean":"2024-09-10T10:44:22.080356+0000","last_became_active":"2024-09-10T10:44:11.040111+0000","last_became_peered":"2024-09-10T10:44:11.040111+0000","last_unstale":"2024-09-10T10:44:22.080356+0000","last_undegraded":"2024-09-10T10:44:22.080356+0000","last_fullsized":"2024-09-10T10:44:22.080356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:24.999485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092270+0000","last_change":"2024-09-10T10:44:22.091959+0000","last_active":"2024-09-10T10:44:22.092270+0000","last_peered":"2024-09-10T10:44:22.092270+0000","last_clean":"2024-09-10T10:44:22.092270+0000","last_became_active":"2024-09-10T10:44:22.091707+0000","last_became_peered":"2024-09-10T10:44:22.091707+0000","last_unstale":"2024-09-10T10:44:22.092270+0000","last_undegraded":"2024-09-10T10:44:22.092270+0000","last_fullsized":"2024-09-10T10:44:22.092270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:38.454868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081432+0000","last_change":"2024-09-10T10:44:11.034339+0000","last_active":"2024-09-10T10:44:22.081432+0000","last_peered":"2024-09-10T10:44:22.081432+0000","last_clean":"2024-09-10T10:44:22.081432+0000","last_became_active":"2024-09-10T10:44:11.034022+0000","last_became_peered":"2024-09-10T10:44:11.034022+0000","last_unstale":"2024-09-10T10:44:22.081432+0000","last_undegraded":"2024-09-10T10:44:22.081432+0000","last_fullsized":"2024-09-10T10:44:22.081432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:54.122244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078068+0000","last_change":"2024-09-10T10:44:11.037871+0000","last_active":"2024-09-10T10:44:22.078068+0000","last_peered":"2024-09-10T10:44:22.078068+0000","last_clean":"2024-09-10T10:44:22.078068+0000","last_became_active":"2024-09-10T10:44:11.037372+0000","last_became_peered":"2024-09-10T10:44:11.037372+0000","last_unstale":"2024-09-10T10:44:22.078068+0000","last_undegraded":"2024-09-10T10:44:22.078068+0000","last_fullsized":"2024-09-10T10:44:22.078068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:22.296078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077383+0000","last_change":"2024-09-10T10:44:11.034826+0000","last_active":"2024-09-10T10:44:22.077383+0000","last_peered":"2024-09-10T10:44:22.077383+0000","last_clean":"2024-09-10T10:44:22.077383+0000","last_became_active":"2024-09-10T10:44:11.034520+0000","last_became_peered":"2024-09-10T10:44:11.034520+0000","last_unstale":"2024-09-10T10:44:22.077383+0000","last_undegraded":"2024-09-10T10:44:22.077383+0000","last_fullsized":"2024-09-10T10:44:22.077383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:13.131060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085136+0000","last_change":"2024-09-10T10:44:22.085136+0000","last_active":"2024-09-10T10:44:22.085136+0000","last_peered":"2024-09-10T10:44:22.085136+0000","last_clean":"2024-09-10T10:44:22.085136+0000","last_became_active":"2024-09-10T10:44:22.084718+0000","last_became_peered":"2024-09-10T10:44:22.084718+0000","last_unstale":"2024-09-10T10:44:22.085136+0000","last_undegraded":"2024-09-10T10:44:22.085136+0000","last_fullsized":"2024-09-10T10:44:22.085136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:31.165424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081062+0000","last_change":"2024-09-10T10:44:11.042336+0000","last_active":"2024-09-10T10:44:22.081062+0000","last_peered":"2024-09-10T10:44:22.081062+0000","last_clean":"2024-09-10T10:44:22.081062+0000","last_became_active":"2024-09-10T10:44:11.041127+0000","last_became_peered":"2024-09-10T10:44:11.041127+0000","last_unstale":"2024-09-10T10:44:22.081062+0000","last_undegraded":"2024-09-10T10:44:22.081062+0000","last_fullsized":"2024-09-10T10:44:22.081062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:50.349201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081158+0000","last_change":"2024-09-10T10:44:11.043188+0000","last_active":"2024-09-10T10:44:22.081158+0000","last_peered":"2024-09-10T10:44:22.081158+0000","last_clean":"2024-09-10T10:44:22.081158+0000","last_became_active":"2024-09-10T10:44:11.042735+0000","last_became_peered":"2024-09-10T10:44:11.042735+0000","last_unstale":"2024-09-10T10:44:22.081158+0000","last_undegraded":"2024-09-10T10:44:22.081158+0000","last_fullsized":"2024-09-10T10:44:22.081158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:50.090810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077961+0000","last_change":"2024-09-10T10:44:11.041339+0000","last_active":"2024-09-10T10:44:22.077961+0000","last_peered":"2024-09-10T10:44:22.077961+0000","last_clean":"2024-09-10T10:44:22.077961+0000","last_became_active":"2024-09-10T10:44:11.038011+0000","last_became_peered":"2024-09-10T10:44:11.038011+0000","last_unstale":"2024-09-10T10:44:22.077961+0000","last_undegraded":"2024-09-10T10:44:22.077961+0000","last_fullsized":"2024-09-10T10:44:22.077961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:47.521465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084488+0000","last_change":"2024-09-10T10:44:22.084488+0000","last_active":"2024-09-10T10:44:22.084488+0000","last_peered":"2024-09-10T10:44:22.084488+0000","last_clean":"2024-09-10T10:44:22.084488+0000","last_became_active":"2024-09-10T10:44:22.084062+0000","last_became_peered":"2024-09-10T10:44:22.084062+0000","last_unstale":"2024-09-10T10:44:22.084488+0000","last_undegraded":"2024-09-10T10:44:22.084488+0000","last_fullsized":"2024-09-10T10:44:22.084488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:57.739261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:32.103226+0000","last_change":"2024-09-10T10:44:09.024716+0000","last_active":"2024-09-10T10:44:32.103226+0000","last_peered":"2024-09-10T10:44:32.103226+0000","last_clean":"2024-09-10T10:44:32.103226+0000","last_became_active":"2024-09-10T10:44:09.023901+0000","last_became_peered":"2024-09-10T10:44:09.023901+0000","last_unstale":"2024-09-10T10:44:32.103226+0000","last_undegraded":"2024-09-10T10:44:32.103226+0000","last_fullsized":"2024-09-10T10:44:32.103226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:28.901896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080568+0000","last_change":"2024-09-10T10:44:11.044838+0000","last_active":"2024-09-10T10:44:22.080568+0000","last_peered":"2024-09-10T10:44:22.080568+0000","last_clean":"2024-09-10T10:44:22.080568+0000","last_became_active":"2024-09-10T10:44:11.044542+0000","last_became_peered":"2024-09-10T10:44:11.044542+0000","last_unstale":"2024-09-10T10:44:22.080568+0000","last_undegraded":"2024-09-10T10:44:22.080568+0000","last_fullsized":"2024-09-10T10:44:22.080568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:02.753914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082979+0000","last_change":"2024-09-10T10:44:09.025799+0000","last_active":"2024-09-10T10:44:22.082979+0000","last_peered":"2024-09-10T10:44:22.082979+0000","last_clean":"2024-09-10T10:44:22.082979+0000","last_became_active":"2024-09-10T10:44:09.025421+0000","last_became_peered":"2024-09-10T10:44:09.025421+0000","last_unstale":"2024-09-10T10:44:22.082979+0000","last_undegraded":"2024-09-10T10:44:22.082979+0000","last_fullsized":"2024-09-10T10:44:22.082979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:11.928663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077144+0000","last_change":"2024-09-10T10:44:09.017640+0000","last_active":"2024-09-10T10:44:22.077144+0000","last_peered":"2024-09-10T10:44:22.077144+0000","last_clean":"2024-09-10T10:44:22.077144+0000","last_became_active":"2024-09-10T10:44:09.017286+0000","last_became_peered":"2024-09-10T10:44:09.017286+0000","last_unstale":"2024-09-10T10:44:22.077144+0000","last_undegraded":"2024-09-10T10:44:22.077144+0000","last_fullsized":"2024-09-10T10:44:22.077144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:59.678607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153461+0000","last_change":"2024-09-10T10:44:11.038225+0000","last_active":"2024-09-10T10:44:22.153461+0000","last_peered":"2024-09-10T10:44:22.153461+0000","last_clean":"2024-09-10T10:44:22.153461+0000","last_became_active":"2024-09-10T10:44:11.037924+0000","last_became_peered":"2024-09-10T10:44:11.037924+0000","last_unstale":"2024-09-10T10:44:22.153461+0000","last_undegraded":"2024-09-10T10:44:22.153461+0000","last_fullsized":"2024-09-10T10:44:22.153461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:58.787166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080948+0000","last_change":"2024-09-10T10:44:09.024676+0000","last_active":"2024-09-10T10:44:22.080948+0000","last_peered":"2024-09-10T10:44:22.080948+0000","last_clean":"2024-09-10T10:44:22.080948+0000","last_became_active":"2024-09-10T10:44:09.023970+0000","last_became_peered":"2024-09-10T10:44:09.023970+0000","last_unstale":"2024-09-10T10:44:22.080948+0000","last_undegraded":"2024-09-10T10:44:22.080948+0000","last_fullsized":"2024-09-10T10:44:22.080948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:40.296260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080161+0000","last_change":"2024-09-10T10:44:11.040330+0000","last_active":"2024-09-10T10:44:22.080161+0000","last_peered":"2024-09-10T10:44:22.080161+0000","last_clean":"2024-09-10T10:44:22.080161+0000","last_became_active":"2024-09-10T10:44:11.040018+0000","last_became_peered":"2024-09-10T10:44:11.040018+0000","last_unstale":"2024-09-10T10:44:22.080161+0000","last_undegraded":"2024-09-10T10:44:22.080161+0000","last_fullsized":"2024-09-10T10:44:22.080161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:24.403070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:32.102696+0000","last_change":"2024-09-10T10:44:09.021488+0000","last_active":"2024-09-10T10:44:32.102696+0000","last_peered":"2024-09-10T10:44:32.102696+0000","last_clean":"2024-09-10T10:44:32.102696+0000","last_became_active":"2024-09-10T10:44:09.021088+0000","last_became_peered":"2024-09-10T10:44:09.021088+0000","last_unstale":"2024-09-10T10:44:32.102696+0000","last_undegraded":"2024-09-10T10:44:32.102696+0000","last_fullsized":"2024-09-10T10:44:32.102696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:37.990090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085498+0000","last_change":"2024-09-10T10:44:11.045142+0000","last_active":"2024-09-10T10:44:22.085498+0000","last_peered":"2024-09-10T10:44:22.085498+0000","last_clean":"2024-09-10T10:44:22.085498+0000","last_became_active":"2024-09-10T10:44:11.044770+0000","last_became_peered":"2024-09-10T10:44:11.044770+0000","last_unstale":"2024-09-10T10:44:22.085498+0000","last_undegraded":"2024-09-10T10:44:22.085498+0000","last_fullsized":"2024-09-10T10:44:22.085498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:10.731504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.096348+0000","last_change":"2024-09-10T10:44:22.096348+0000","last_active":"2024-09-10T10:44:22.096348+0000","last_peered":"2024-09-10T10:44:22.096348+0000","last_clean":"2024-09-10T10:44:22.096348+0000","last_became_active":"2024-09-10T10:44:22.092921+0000","last_became_peered":"2024-09-10T10:44:22.092921+0000","last_unstale":"2024-09-10T10:44:22.096348+0000","last_undegraded":"2024-09-10T10:44:22.096348+0000","last_fullsized":"2024-09-10T10:44:22.096348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:15.721849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079428+0000","last_change":"2024-09-10T10:44:11.034499+0000","last_active":"2024-09-10T10:44:22.079428+0000","last_peered":"2024-09-10T10:44:22.079428+0000","last_clean":"2024-09-10T10:44:22.079428+0000","last_became_active":"2024-09-10T10:44:11.034030+0000","last_became_peered":"2024-09-10T10:44:11.034030+0000","last_unstale":"2024-09-10T10:44:22.079428+0000","last_undegraded":"2024-09-10T10:44:22.079428+0000","last_fullsized":"2024-09-10T10:44:22.079428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:57.313000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080739+0000","last_change":"2024-09-10T10:44:09.142509+0000","last_active":"2024-09-10T10:44:22.080739+0000","last_peered":"2024-09-10T10:44:22.080739+0000","last_clean":"2024-09-10T10:44:22.080739+0000","last_became_active":"2024-09-10T10:44:09.142048+0000","last_became_peered":"2024-09-10T10:44:09.142048+0000","last_unstale":"2024-09-10T10:44:22.080739+0000","last_undegraded":"2024-09-10T10:44:22.080739+0000","last_fullsized":"2024-09-10T10:44:22.080739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:57.937241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085180+0000","last_change":"2024-09-10T10:44:11.034851+0000","last_active":"2024-09-10T10:44:22.085180+0000","last_peered":"2024-09-10T10:44:22.085180+0000","last_clean":"2024-09-10T10:44:22.085180+0000","last_became_active":"2024-09-10T10:44:11.034524+0000","last_became_peered":"2024-09-10T10:44:11.034524+0000","last_unstale":"2024-09-10T10:44:22.085180+0000","last_undegraded":"2024-09-10T10:44:22.085180+0000","last_fullsized":"2024-09-10T10:44:22.085180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:06.356958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'26","reported_seq":79,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:33.116162+0000","last_change":"2024-09-10T10:44:09.024313+0000","last_active":"2024-09-10T10:44:33.116162+0000","last_peered":"2024-09-10T10:44:33.116162+0000","last_clean":"2024-09-10T10:44:33.116162+0000","last_became_active":"2024-09-10T10:44:09.023933+0000","last_became_peered":"2024-09-10T10:44:09.023933+0000","last_unstale":"2024-09-10T10:44:33.116162+0000","last_undegraded":"2024-09-10T10:44:33.116162+0000","last_fullsized":"2024-09-10T10:44:33.116162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:37.522786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52100,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":65,"num_write":27,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083275+0000","last_change":"2024-09-10T10:44:11.039171+0000","last_active":"2024-09-10T10:44:22.083275+0000","last_peered":"2024-09-10T10:44:22.083275+0000","last_clean":"2024-09-10T10:44:22.083275+0000","last_became_active":"2024-09-10T10:44:11.038833+0000","last_became_peered":"2024-09-10T10:44:11.038833+0000","last_unstale":"2024-09-10T10:44:22.083275+0000","last_undegraded":"2024-09-10T10:44:22.083275+0000","last_fullsized":"2024-09-10T10:44:22.083275+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:37.115626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081645+0000","last_change":"2024-09-10T10:44:09.021368+0000","last_active":"2024-09-10T10:44:22.081645+0000","last_peered":"2024-09-10T10:44:22.081645+0000","last_clean":"2024-09-10T10:44:22.081645+0000","last_became_active":"2024-09-10T10:44:09.020805+0000","last_became_peered":"2024-09-10T10:44:09.020805+0000","last_unstale":"2024-09-10T10:44:22.081645+0000","last_undegraded":"2024-09-10T10:44:22.081645+0000","last_fullsized":"2024-09-10T10:44:22.081645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:20.408776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077226+0000","last_change":"2024-09-10T10:44:11.037539+0000","last_active":"2024-09-10T10:44:22.077226+0000","last_peered":"2024-09-10T10:44:22.077226+0000","last_clean":"2024-09-10T10:44:22.077226+0000","last_became_active":"2024-09-10T10:44:11.036607+0000","last_became_peered":"2024-09-10T10:44:11.036607+0000","last_unstale":"2024-09-10T10:44:22.077226+0000","last_undegraded":"2024-09-10T10:44:22.077226+0000","last_fullsized":"2024-09-10T10:44:22.077226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:01:46.029665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080495+0000","last_change":"2024-09-10T10:44:09.023766+0000","last_active":"2024-09-10T10:44:22.080495+0000","last_peered":"2024-09-10T10:44:22.080495+0000","last_clean":"2024-09-10T10:44:22.080495+0000","last_became_active":"2024-09-10T10:44:09.023422+0000","last_became_peered":"2024-09-10T10:44:09.023422+0000","last_unstale":"2024-09-10T10:44:22.080495+0000","last_undegraded":"2024-09-10T10:44:22.080495+0000","last_fullsized":"2024-09-10T10:44:22.080495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:56.009610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082817+0000","last_change":"2024-09-10T10:44:11.038846+0000","last_active":"2024-09-10T10:44:22.082817+0000","last_peered":"2024-09-10T10:44:22.082817+0000","last_clean":"2024-09-10T10:44:22.082817+0000","last_became_active":"2024-09-10T10:44:11.038477+0000","last_became_peered":"2024-09-10T10:44:11.038477+0000","last_unstale":"2024-09-10T10:44:22.082817+0000","last_undegraded":"2024-09-10T10:44:22.082817+0000","last_fullsized":"2024-09-10T10:44:22.082817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:26.255612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'3","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:32.105971+0000","last_change":"2024-09-10T10:44:09.023394+0000","last_active":"2024-09-10T10:44:32.105971+0000","last_peered":"2024-09-10T10:44:32.105971+0000","last_clean":"2024-09-10T10:44:32.105971+0000","last_became_active":"2024-09-10T10:44:09.023049+0000","last_became_peered":"2024-09-10T10:44:09.023049+0000","last_unstale":"2024-09-10T10:44:32.105971+0000","last_undegraded":"2024-09-10T10:44:32.105971+0000","last_fullsized":"2024-09-10T10:44:32.105971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:00.024671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081712+0000","last_change":"2024-09-10T10:44:11.041162+0000","last_active":"2024-09-10T10:44:22.081712+0000","last_peered":"2024-09-10T10:44:22.081712+0000","last_clean":"2024-09-10T10:44:22.081712+0000","last_became_active":"2024-09-10T10:44:11.040848+0000","last_became_peered":"2024-09-10T10:44:11.040848+0000","last_unstale":"2024-09-10T10:44:22.081712+0000","last_undegraded":"2024-09-10T10:44:22.081712+0000","last_fullsized":"2024-09-10T10:44:22.081712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:13.044447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082834+0000","last_change":"2024-09-10T10:44:09.142490+0000","last_active":"2024-09-10T10:44:22.082834+0000","last_peered":"2024-09-10T10:44:22.082834+0000","last_clean":"2024-09-10T10:44:22.082834+0000","last_became_active":"2024-09-10T10:44:09.141983+0000","last_became_peered":"2024-09-10T10:44:09.141983+0000","last_unstale":"2024-09-10T10:44:22.082834+0000","last_undegraded":"2024-09-10T10:44:22.082834+0000","last_fullsized":"2024-09-10T10:44:22.082834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:34.481241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080333+0000","last_change":"2024-09-10T10:44:11.042109+0000","last_active":"2024-09-10T10:44:22.080333+0000","last_peered":"2024-09-10T10:44:22.080333+0000","last_clean":"2024-09-10T10:44:22.080333+0000","last_became_active":"2024-09-10T10:44:11.041154+0000","last_became_peered":"2024-09-10T10:44:11.041154+0000","last_unstale":"2024-09-10T10:44:22.080333+0000","last_undegraded":"2024-09-10T10:44:22.080333+0000","last_fullsized":"2024-09-10T10:44:22.080333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:50.134773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083174+0000","last_change":"2024-09-10T10:44:09.019117+0000","last_active":"2024-09-10T10:44:22.083174+0000","last_peered":"2024-09-10T10:44:22.083174+0000","last_clean":"2024-09-10T10:44:22.083174+0000","last_became_active":"2024-09-10T10:44:09.018733+0000","last_became_peered":"2024-09-10T10:44:09.018733+0000","last_unstale":"2024-09-10T10:44:22.083174+0000","last_undegraded":"2024-09-10T10:44:22.083174+0000","last_fullsized":"2024-09-10T10:44:22.083174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:40.049342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078300+0000","last_change":"2024-09-10T10:44:11.040381+0000","last_active":"2024-09-10T10:44:22.078300+0000","last_peered":"2024-09-10T10:44:22.078300+0000","last_clean":"2024-09-10T10:44:22.078300+0000","last_became_active":"2024-09-10T10:44:11.040053+0000","last_became_peered":"2024-09-10T10:44:11.040053+0000","last_unstale":"2024-09-10T10:44:22.078300+0000","last_undegraded":"2024-09-10T10:44:22.078300+0000","last_fullsized":"2024-09-10T10:44:22.078300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:29.043171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077403+0000","last_change":"2024-09-10T10:44:09.018173+0000","last_active":"2024-09-10T10:44:22.077403+0000","last_peered":"2024-09-10T10:44:22.077403+0000","last_clean":"2024-09-10T10:44:22.077403+0000","last_became_active":"2024-09-10T10:44:09.017790+0000","last_became_peered":"2024-09-10T10:44:09.017790+0000","last_unstale":"2024-09-10T10:44:22.077403+0000","last_undegraded":"2024-09-10T10:44:22.077403+0000","last_fullsized":"2024-09-10T10:44:22.077403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:43.515112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082583+0000","last_change":"2024-09-10T10:44:11.035293+0000","last_active":"2024-09-10T10:44:22.082583+0000","last_peered":"2024-09-10T10:44:22.082583+0000","last_clean":"2024-09-10T10:44:22.082583+0000","last_became_active":"2024-09-10T10:44:11.034958+0000","last_became_peered":"2024-09-10T10:44:11.034958+0000","last_unstale":"2024-09-10T10:44:22.082583+0000","last_undegraded":"2024-09-10T10:44:22.082583+0000","last_fullsized":"2024-09-10T10:44:22.082583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:09.397229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080789+0000","last_change":"2024-09-10T10:44:09.021322+0000","last_active":"2024-09-10T10:44:22.080789+0000","last_peered":"2024-09-10T10:44:22.080789+0000","last_clean":"2024-09-10T10:44:22.080789+0000","last_became_active":"2024-09-10T10:44:09.021018+0000","last_became_peered":"2024-09-10T10:44:09.021018+0000","last_unstale":"2024-09-10T10:44:22.080789+0000","last_undegraded":"2024-09-10T10:44:22.080789+0000","last_fullsized":"2024-09-10T10:44:22.080789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:48.999125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082090+0000","last_change":"2024-09-10T10:44:11.040545+0000","last_active":"2024-09-10T10:44:22.082090+0000","last_peered":"2024-09-10T10:44:22.082090+0000","last_clean":"2024-09-10T10:44:22.082090+0000","last_became_active":"2024-09-10T10:44:11.040042+0000","last_became_peered":"2024-09-10T10:44:11.040042+0000","last_unstale":"2024-09-10T10:44:22.082090+0000","last_undegraded":"2024-09-10T10:44:22.082090+0000","last_fullsized":"2024-09-10T10:44:22.082090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:14.367771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081581+0000","last_change":"2024-09-10T10:44:09.142944+0000","last_active":"2024-09-10T10:44:22.081581+0000","last_peered":"2024-09-10T10:44:22.081581+0000","last_clean":"2024-09-10T10:44:22.081581+0000","last_became_active":"2024-09-10T10:44:09.142550+0000","last_became_peered":"2024-09-10T10:44:09.142550+0000","last_unstale":"2024-09-10T10:44:22.081581+0000","last_undegraded":"2024-09-10T10:44:22.081581+0000","last_fullsized":"2024-09-10T10:44:22.081581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:56.247251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082331+0000","last_change":"2024-09-10T10:44:11.038205+0000","last_active":"2024-09-10T10:44:22.082331+0000","last_peered":"2024-09-10T10:44:22.082331+0000","last_clean":"2024-09-10T10:44:22.082331+0000","last_became_active":"2024-09-10T10:44:11.037817+0000","last_became_peered":"2024-09-10T10:44:11.037817+0000","last_unstale":"2024-09-10T10:44:22.082331+0000","last_undegraded":"2024-09-10T10:44:22.082331+0000","last_fullsized":"2024-09-10T10:44:22.082331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:39.221897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":343,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081904+0000","last_change":"2024-09-10T10:43:01.771132+0000","last_active":"2024-09-10T10:44:22.081904+0000","last_peered":"2024-09-10T10:44:22.081904+0000","last_clean":"2024-09-10T10:44:22.081904+0000","last_became_active":"2024-09-10T10:43:01.770568+0000","last_became_peered":"2024-09-10T10:43:01.770568+0000","last_unstale":"2024-09-10T10:44:22.081904+0000","last_undegraded":"2024-09-10T10:44:22.081904+0000","last_fullsized":"2024-09-10T10:44:22.081904+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:39:37.547474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:39:37.547474+0000","last_clean_scrub_stamp":"2024-09-10T10:39:37.547474+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:09.699899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.091961+0000","last_change":"2024-09-10T10:44:09.021115+0000","last_active":"2024-09-10T10:44:22.091961+0000","last_peered":"2024-09-10T10:44:22.091961+0000","last_clean":"2024-09-10T10:44:22.091961+0000","last_became_active":"2024-09-10T10:44:09.020747+0000","last_became_peered":"2024-09-10T10:44:09.020747+0000","last_unstale":"2024-09-10T10:44:22.091961+0000","last_undegraded":"2024-09-10T10:44:22.091961+0000","last_fullsized":"2024-09-10T10:44:22.091961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:15.828535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153652+0000","last_change":"2024-09-10T10:44:11.046484+0000","last_active":"2024-09-10T10:44:22.153652+0000","last_peered":"2024-09-10T10:44:22.153652+0000","last_clean":"2024-09-10T10:44:22.153652+0000","last_became_active":"2024-09-10T10:44:11.046142+0000","last_became_peered":"2024-09-10T10:44:11.046142+0000","last_unstale":"2024-09-10T10:44:22.153652+0000","last_undegraded":"2024-09-10T10:44:22.153652+0000","last_fullsized":"2024-09-10T10:44:22.153652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:15.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.088392+0000","last_change":"2024-09-10T10:44:22.087926+0000","last_active":"2024-09-10T10:44:22.088392+0000","last_peered":"2024-09-10T10:44:22.088392+0000","last_clean":"2024-09-10T10:44:22.088392+0000","last_became_active":"2024-09-10T10:44:22.084248+0000","last_became_peered":"2024-09-10T10:44:22.084248+0000","last_unstale":"2024-09-10T10:44:22.088392+0000","last_undegraded":"2024-09-10T10:44:22.088392+0000","last_fullsized":"2024-09-10T10:44:22.088392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:15.774609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077128+0000","last_change":"2024-09-10T10:44:11.034105+0000","last_active":"2024-09-10T10:44:22.077128+0000","last_peered":"2024-09-10T10:44:22.077128+0000","last_clean":"2024-09-10T10:44:22.077128+0000","last_became_active":"2024-09-10T10:44:11.033792+0000","last_became_peered":"2024-09-10T10:44:11.033792+0000","last_unstale":"2024-09-10T10:44:22.077128+0000","last_undegraded":"2024-09-10T10:44:22.077128+0000","last_fullsized":"2024-09-10T10:44:22.077128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.131963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:33.106609+0000","last_change":"2024-09-10T10:44:09.142514+0000","last_active":"2024-09-10T10:44:33.106609+0000","last_peered":"2024-09-10T10:44:33.106609+0000","last_clean":"2024-09-10T10:44:33.106609+0000","last_became_active":"2024-09-10T10:44:09.142115+0000","last_became_peered":"2024-09-10T10:44:09.142115+0000","last_unstale":"2024-09-10T10:44:33.106609+0000","last_undegraded":"2024-09-10T10:44:33.106609+0000","last_fullsized":"2024-09-10T10:44:33.106609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:37.520105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080531+0000","last_change":"2024-09-10T10:44:11.040122+0000","last_active":"2024-09-10T10:44:22.080531+0000","last_peered":"2024-09-10T10:44:22.080531+0000","last_clean":"2024-09-10T10:44:22.080531+0000","last_became_active":"2024-09-10T10:44:11.039649+0000","last_became_peered":"2024-09-10T10:44:11.039649+0000","last_unstale":"2024-09-10T10:44:22.080531+0000","last_undegraded":"2024-09-10T10:44:22.080531+0000","last_fullsized":"2024-09-10T10:44:22.080531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:26.401981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083151+0000","last_change":"2024-09-10T10:44:09.024933+0000","last_active":"2024-09-10T10:44:22.083151+0000","last_peered":"2024-09-10T10:44:22.083151+0000","last_clean":"2024-09-10T10:44:22.083151+0000","last_became_active":"2024-09-10T10:44:09.024579+0000","last_became_peered":"2024-09-10T10:44:09.024579+0000","last_unstale":"2024-09-10T10:44:22.083151+0000","last_undegraded":"2024-09-10T10:44:22.083151+0000","last_fullsized":"2024-09-10T10:44:22.083151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:45.644932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084217+0000","last_change":"2024-09-10T10:44:22.084217+0000","last_active":"2024-09-10T10:44:22.084217+0000","last_peered":"2024-09-10T10:44:22.084217+0000","last_clean":"2024-09-10T10:44:22.084217+0000","last_became_active":"2024-09-10T10:44:22.083707+0000","last_became_peered":"2024-09-10T10:44:22.083707+0000","last_unstale":"2024-09-10T10:44:22.084217+0000","last_undegraded":"2024-09-10T10:44:22.084217+0000","last_fullsized":"2024-09-10T10:44:22.084217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:30.124690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080402+0000","last_change":"2024-09-10T10:44:09.142462+0000","last_active":"2024-09-10T10:44:22.080402+0000","last_peered":"2024-09-10T10:44:22.080402+0000","last_clean":"2024-09-10T10:44:22.080402+0000","last_became_active":"2024-09-10T10:44:09.142086+0000","last_became_peered":"2024-09-10T10:44:09.142086+0000","last_unstale":"2024-09-10T10:44:22.080402+0000","last_undegraded":"2024-09-10T10:44:22.080402+0000","last_fullsized":"2024-09-10T10:44:22.080402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:28.019899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082676+0000","last_change":"2024-09-10T10:44:11.036928+0000","last_active":"2024-09-10T10:44:22.082676+0000","last_peered":"2024-09-10T10:44:22.082676+0000","last_clean":"2024-09-10T10:44:22.082676+0000","last_became_active":"2024-09-10T10:44:11.036626+0000","last_became_peered":"2024-09-10T10:44:11.036626+0000","last_unstale":"2024-09-10T10:44:22.082676+0000","last_undegraded":"2024-09-10T10:44:22.082676+0000","last_fullsized":"2024-09-10T10:44:22.082676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:42.624867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083407+0000","last_change":"2024-09-10T10:44:09.142521+0000","last_active":"2024-09-10T10:44:22.083407+0000","last_peered":"2024-09-10T10:44:22.083407+0000","last_clean":"2024-09-10T10:44:22.083407+0000","last_became_active":"2024-09-10T10:44:09.142061+0000","last_became_peered":"2024-09-10T10:44:09.142061+0000","last_unstale":"2024-09-10T10:44:22.083407+0000","last_undegraded":"2024-09-10T10:44:22.083407+0000","last_fullsized":"2024-09-10T10:44:22.083407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:14.054388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077489+0000","last_change":"2024-09-10T10:44:11.031953+0000","last_active":"2024-09-10T10:44:22.077489+0000","last_peered":"2024-09-10T10:44:22.077489+0000","last_clean":"2024-09-10T10:44:22.077489+0000","last_became_active":"2024-09-10T10:44:11.031610+0000","last_became_peered":"2024-09-10T10:44:11.031610+0000","last_unstale":"2024-09-10T10:44:22.077489+0000","last_undegraded":"2024-09-10T10:44:22.077489+0000","last_fullsized":"2024-09-10T10:44:22.077489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:03.305651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:33.029846+0000","last_change":"2024-09-10T10:44:09.016752+0000","last_active":"2024-09-10T10:44:33.029846+0000","last_peered":"2024-09-10T10:44:33.029846+0000","last_clean":"2024-09-10T10:44:33.029846+0000","last_became_active":"2024-09-10T10:44:09.016464+0000","last_became_peered":"2024-09-10T10:44:09.016464+0000","last_unstale":"2024-09-10T10:44:33.029846+0000","last_undegraded":"2024-09-10T10:44:33.029846+0000","last_fullsized":"2024-09-10T10:44:33.029846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:25.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153747+0000","last_change":"2024-09-10T10:44:11.044793+0000","last_active":"2024-09-10T10:44:22.153747+0000","last_peered":"2024-09-10T10:44:22.153747+0000","last_clean":"2024-09-10T10:44:22.153747+0000","last_became_active":"2024-09-10T10:44:11.043611+0000","last_became_peered":"2024-09-10T10:44:11.043611+0000","last_unstale":"2024-09-10T10:44:22.153747+0000","last_undegraded":"2024-09-10T10:44:22.153747+0000","last_fullsized":"2024-09-10T10:44:22.153747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:20.235527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.091934+0000","last_change":"2024-09-10T10:44:09.017385+0000","last_active":"2024-09-10T10:44:22.091934+0000","last_peered":"2024-09-10T10:44:22.091934+0000","last_clean":"2024-09-10T10:44:22.091934+0000","last_became_active":"2024-09-10T10:44:09.017033+0000","last_became_peered":"2024-09-10T10:44:09.017033+0000","last_unstale":"2024-09-10T10:44:22.091934+0000","last_undegraded":"2024-09-10T10:44:22.091934+0000","last_fullsized":"2024-09-10T10:44:22.091934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:57.963765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079263+0000","last_change":"2024-09-10T10:44:11.039194+0000","last_active":"2024-09-10T10:44:22.079263+0000","last_peered":"2024-09-10T10:44:22.079263+0000","last_clean":"2024-09-10T10:44:22.079263+0000","last_became_active":"2024-09-10T10:44:11.038290+0000","last_became_peered":"2024-09-10T10:44:11.038290+0000","last_unstale":"2024-09-10T10:44:22.079263+0000","last_undegraded":"2024-09-10T10:44:22.079263+0000","last_fullsized":"2024-09-10T10:44:22.079263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:50.938124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085070+0000","last_change":"2024-09-10T10:44:09.142618+0000","last_active":"2024-09-10T10:44:22.085070+0000","last_peered":"2024-09-10T10:44:22.085070+0000","last_clean":"2024-09-10T10:44:22.085070+0000","last_became_active":"2024-09-10T10:44:09.142232+0000","last_became_peered":"2024-09-10T10:44:09.142232+0000","last_unstale":"2024-09-10T10:44:22.085070+0000","last_undegraded":"2024-09-10T10:44:22.085070+0000","last_fullsized":"2024-09-10T10:44:22.085070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:18.797080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082392+0000","last_change":"2024-09-10T10:44:11.039288+0000","last_active":"2024-09-10T10:44:22.082392+0000","last_peered":"2024-09-10T10:44:22.082392+0000","last_clean":"2024-09-10T10:44:22.082392+0000","last_became_active":"2024-09-10T10:44:11.038955+0000","last_became_peered":"2024-09-10T10:44:11.038955+0000","last_unstale":"2024-09-10T10:44:22.082392+0000","last_undegraded":"2024-09-10T10:44:22.082392+0000","last_fullsized":"2024-09-10T10:44:22.082392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:29.564274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153587+0000","last_change":"2024-09-10T10:44:09.020638+0000","last_active":"2024-09-10T10:44:22.153587+0000","last_peered":"2024-09-10T10:44:22.153587+0000","last_clean":"2024-09-10T10:44:22.153587+0000","last_became_active":"2024-09-10T10:44:09.020320+0000","last_became_peered":"2024-09-10T10:44:09.020320+0000","last_unstale":"2024-09-10T10:44:22.153587+0000","last_undegraded":"2024-09-10T10:44:22.153587+0000","last_fullsized":"2024-09-10T10:44:22.153587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:00.793392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078085+0000","last_change":"2024-09-10T10:44:11.034010+0000","last_active":"2024-09-10T10:44:22.078085+0000","last_peered":"2024-09-10T10:44:22.078085+0000","last_clean":"2024-09-10T10:44:22.078085+0000","last_became_active":"2024-09-10T10:44:11.033639+0000","last_became_peered":"2024-09-10T10:44:11.033639+0000","last_unstale":"2024-09-10T10:44:22.078085+0000","last_undegraded":"2024-09-10T10:44:22.078085+0000","last_fullsized":"2024-09-10T10:44:22.078085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:20.127131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081171+0000","last_change":"2024-09-10T10:44:09.019478+0000","last_active":"2024-09-10T10:44:22.081171+0000","last_peered":"2024-09-10T10:44:22.081171+0000","last_clean":"2024-09-10T10:44:22.081171+0000","last_became_active":"2024-09-10T10:44:09.019033+0000","last_became_peered":"2024-09-10T10:44:09.019033+0000","last_unstale":"2024-09-10T10:44:22.081171+0000","last_undegraded":"2024-09-10T10:44:22.081171+0000","last_fullsized":"2024-09-10T10:44:22.081171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:17.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153647+0000","last_change":"2024-09-10T10:44:11.036118+0000","last_active":"2024-09-10T10:44:22.153647+0000","last_peered":"2024-09-10T10:44:22.153647+0000","last_clean":"2024-09-10T10:44:22.153647+0000","last_became_active":"2024-09-10T10:44:11.035648+0000","last_became_peered":"2024-09-10T10:44:11.035648+0000","last_unstale":"2024-09-10T10:44:22.153647+0000","last_undegraded":"2024-09-10T10:44:22.153647+0000","last_fullsized":"2024-09-10T10:44:22.153647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:52.134207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.100063+0000","last_change":"2024-09-10T10:44:22.099643+0000","last_active":"2024-09-10T10:44:22.100063+0000","last_peered":"2024-09-10T10:44:22.100063+0000","last_clean":"2024-09-10T10:44:22.100063+0000","last_became_active":"2024-09-10T10:44:22.093961+0000","last_became_peered":"2024-09-10T10:44:22.093961+0000","last_unstale":"2024-09-10T10:44:22.100063+0000","last_undegraded":"2024-09-10T10:44:22.100063+0000","last_fullsized":"2024-09-10T10:44:22.100063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:11.958415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082997+0000","last_change":"2024-09-10T10:44:11.035067+0000","last_active":"2024-09-10T10:44:22.082997+0000","last_peered":"2024-09-10T10:44:22.082997+0000","last_clean":"2024-09-10T10:44:22.082997+0000","last_became_active":"2024-09-10T10:44:11.034756+0000","last_became_peered":"2024-09-10T10:44:11.034756+0000","last_unstale":"2024-09-10T10:44:22.082997+0000","last_undegraded":"2024-09-10T10:44:22.082997+0000","last_fullsized":"2024-09-10T10:44:22.082997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:20.425948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080613+0000","last_change":"2024-09-10T10:44:11.039774+0000","last_active":"2024-09-10T10:44:22.080613+0000","last_peered":"2024-09-10T10:44:22.080613+0000","last_clean":"2024-09-10T10:44:22.080613+0000","last_became_active":"2024-09-10T10:44:11.039433+0000","last_became_peered":"2024-09-10T10:44:11.039433+0000","last_unstale":"2024-09-10T10:44:22.080613+0000","last_undegraded":"2024-09-10T10:44:22.080613+0000","last_fullsized":"2024-09-10T10:44:22.080613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:17.743436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083623+0000","last_change":"2024-09-10T10:44:09.022130+0000","last_active":"2024-09-10T10:44:22.083623+0000","last_peered":"2024-09-10T10:44:22.083623+0000","last_clean":"2024-09-10T10:44:22.083623+0000","last_became_active":"2024-09-10T10:44:09.021769+0000","last_became_peered":"2024-09-10T10:44:09.021769+0000","last_unstale":"2024-09-10T10:44:22.083623+0000","last_undegraded":"2024-09-10T10:44:22.083623+0000","last_fullsized":"2024-09-10T10:44:22.083623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:03.215955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085154+0000","last_change":"2024-09-10T10:44:11.044147+0000","last_active":"2024-09-10T10:44:22.085154+0000","last_peered":"2024-09-10T10:44:22.085154+0000","last_clean":"2024-09-10T10:44:22.085154+0000","last_became_active":"2024-09-10T10:44:11.042545+0000","last_became_peered":"2024-09-10T10:44:11.042545+0000","last_unstale":"2024-09-10T10:44:22.085154+0000","last_undegraded":"2024-09-10T10:44:22.085154+0000","last_fullsized":"2024-09-10T10:44:22.085154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:59.360624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083361+0000","last_change":"2024-09-10T10:44:09.023396+0000","last_active":"2024-09-10T10:44:22.083361+0000","last_peered":"2024-09-10T10:44:22.083361+0000","last_clean":"2024-09-10T10:44:22.083361+0000","last_became_active":"2024-09-10T10:44:09.022876+0000","last_became_peered":"2024-09-10T10:44:09.022876+0000","last_unstale":"2024-09-10T10:44:22.083361+0000","last_undegraded":"2024-09-10T10:44:22.083361+0000","last_fullsized":"2024-09-10T10:44:22.083361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:01.901011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079435+0000","last_change":"2024-09-10T10:44:11.033338+0000","last_active":"2024-09-10T10:44:22.079435+0000","last_peered":"2024-09-10T10:44:22.079435+0000","last_clean":"2024-09-10T10:44:22.079435+0000","last_became_active":"2024-09-10T10:44:11.032941+0000","last_became_peered":"2024-09-10T10:44:11.032941+0000","last_unstale":"2024-09-10T10:44:22.079435+0000","last_undegraded":"2024-09-10T10:44:22.079435+0000","last_fullsized":"2024-09-10T10:44:22.079435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:13.787799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.154016+0000","last_change":"2024-09-10T10:44:09.143814+0000","last_active":"2024-09-10T10:44:22.154016+0000","last_peered":"2024-09-10T10:44:22.154016+0000","last_clean":"2024-09-10T10:44:22.154016+0000","last_became_active":"2024-09-10T10:44:09.141942+0000","last_became_peered":"2024-09-10T10:44:09.141942+0000","last_unstale":"2024-09-10T10:44:22.154016+0000","last_undegraded":"2024-09-10T10:44:22.154016+0000","last_fullsized":"2024-09-10T10:44:22.154016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:31.839457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153764+0000","last_change":"2024-09-10T10:44:11.041386+0000","last_active":"2024-09-10T10:44:22.153764+0000","last_peered":"2024-09-10T10:44:22.153764+0000","last_clean":"2024-09-10T10:44:22.153764+0000","last_became_active":"2024-09-10T10:44:11.041060+0000","last_became_peered":"2024-09-10T10:44:11.041060+0000","last_unstale":"2024-09-10T10:44:22.153764+0000","last_undegraded":"2024-09-10T10:44:22.153764+0000","last_fullsized":"2024-09-10T10:44:22.153764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:38.528977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080586+0000","last_change":"2024-09-10T10:44:09.023375+0000","last_active":"2024-09-10T10:44:22.080586+0000","last_peered":"2024-09-10T10:44:22.080586+0000","last_clean":"2024-09-10T10:44:22.080586+0000","last_became_active":"2024-09-10T10:44:09.023082+0000","last_became_peered":"2024-09-10T10:44:09.023082+0000","last_unstale":"2024-09-10T10:44:22.080586+0000","last_undegraded":"2024-09-10T10:44:22.080586+0000","last_fullsized":"2024-09-10T10:44:22.080586+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:51.685194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":55710,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":102,"num_read_kb":110,"num_write":157,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":145060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612625,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":880640,"data_stored":430529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841557,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712436,"statfs":{"total":95995035648,"available":95961534464,"internally_reserved":0,"allocated":1568768,"data_stored":1074110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005083,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":430765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":430685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495653,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1490944,"data_stored":1022224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626474,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40368,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704784,"statfs":{"total":95995035648,"available":95953698816,"internally_reserved":0,"allocated":884736,"data_stored":430154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855407,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1021908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986228,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":888832,"data_stored":430591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":430465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":472362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":471012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":905216,"data_stored":431425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":41065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:44:38.021 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T10:44:38.021 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T10:44:38.021 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T10:44:38.282 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:38.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:38 smithi007 ceph-mon[30873]: pgmap v289: 97 pgs: 97 active+clean; 631 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-10T10:44:38.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:38 smithi007 ceph-mon[30873]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:38.728 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:38 smithi133 ceph-mon[35282]: pgmap v289: 97 pgs: 97 active+clean; 631 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-10T10:44:38.728 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:38 smithi133 ceph-mon[35282]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:38.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.755+0000 7fb0827d2640 1 -- 172.21.15.7:0/4040144505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07c102e00 msgr2=0x7fb07c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.755+0000 7fb0827d2640 1 --2- 172.21.15.7:0/4040144505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07c102e00 0x7fb07c103280 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb070009900 tx=0x7fb07002a410 comp rx=0 tx=0).stop 2024-09-10T10:44:38.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.755+0000 7fb0827d2640 1 -- 172.21.15.7:0/4040144505 shutdown_connections 2024-09-10T10:44:38.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.755+0000 7fb0827d2640 1 --2- 172.21.15.7:0/4040144505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb07c1037c0 0x7fb07c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.755+0000 7fb0827d2640 1 --2- 172.21.15.7:0/4040144505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07c102e00 0x7fb07c103280 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.755+0000 7fb0827d2640 1 --2- 172.21.15.7:0/4040144505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101c00 0x7fb07c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.755+0000 7fb0827d2640 1 -- 172.21.15.7:0/4040144505 >> 172.21.15.7:0/4040144505 conn(0x7fb07c0fd390 msgr2=0x7fb07c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.755+0000 7fb0827d2640 1 -- 172.21.15.7:0/4040144505 shutdown_connections 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.755+0000 7fb0827d2640 1 -- 172.21.15.7:0/4040144505 wait complete. 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb0827d2640 1 Processor -- start 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb0827d2640 1 -- start start 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb0827d2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101c00 0x7fb07c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb0827d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07c102e00 0x7fb07c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb0827d2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb07c1037c0 0x7fb07c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb0827d2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c10d8e0 con 0x7fb07c102e00 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb0827d2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c10d760 con 0x7fb07c101c00 2024-09-10T10:44:38.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb0827d2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c10da60 con 0x7fb07c1037c0 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb07b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07c102e00 0x7fb07c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb07b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07c102e00 0x7fb07c0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49708/0 (socket says 172.21.15.7:49708) 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb07b7fe640 1 -- 172.21.15.7:0/367862338 learned_addr learned my addr 172.21.15.7:0/367862338 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb07bfff640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101c00 0x7fb07c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb080d6f640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb07c1037c0 0x7fb07c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb07b7fe640 1 -- 172.21.15.7:0/367862338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb07c1037c0 msgr2=0x7fb07c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb07b7fe640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb07c1037c0 0x7fb07c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb07b7fe640 1 -- 172.21.15.7:0/367862338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101c00 msgr2=0x7fb07c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb07b7fe640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101c00 0x7fb07c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb07b7fe640 1 -- 172.21.15.7:0/367862338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c0753e0 con 0x7fb07c102e00 2024-09-10T10:44:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.756+0000 7fb080d6f640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb07c1037c0 0x7fb07c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.757+0000 7fb07b7fe640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07c102e00 0x7fb07c0745e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb070009230 tx=0x7fb070004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.757+0000 7fb0797fa640 1 -- 172.21.15.7:0/367862338 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07003f030 con 0x7fb07c102e00 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.757+0000 7fb0827d2640 1 -- 172.21.15.7:0/367862338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07c1a20c0 con 0x7fb07c102e00 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.757+0000 7fb0797fa640 1 -- 172.21.15.7:0/367862338 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb070003ce0 con 0x7fb07c102e00 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.757+0000 7fb0797fa640 1 -- 172.21.15.7:0/367862338 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070035070 con 0x7fb07c102e00 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.757+0000 7fb0827d2640 1 -- 172.21.15.7:0/367862338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07c1a2640 con 0x7fb07c102e00 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.758+0000 7fb0827d2640 1 -- 172.21.15.7:0/367862338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07c109070 con 0x7fb07c102e00 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.759+0000 7fb0797fa640 1 -- 172.21.15.7:0/367862338 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb070042030 con 0x7fb07c102e00 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.760+0000 7fb0797fa640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb0640a2c10 0x7fb0640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.760+0000 7fb07bfff640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb0640a2c10 0x7fb0640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.760+0000 7fb0797fa640 1 -- 172.21.15.7:0/367862338 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fb0700efbb0 con 0x7fb07c102e00 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.760+0000 7fb07bfff640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb0640a2c10 0x7fb0640a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb07c102c60 tx=0x7fb06c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:38.764+0000 7fb0797fa640 1 -- 172.21.15.7:0/367862338 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0700bc9c0 con 0x7fb07c102e00 2024-09-10T10:44:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:38 smithi103 ceph-mon[35160]: pgmap v289: 97 pgs: 97 active+clean; 631 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-09-10T10:44:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:38 smithi103 ceph-mon[35160]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:44:39.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.007+0000 7fb0827d2640 1 -- 172.21.15.7:0/367862338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb07c062df0 con 0x7fb07c102e00 2024-09-10T10:44:39.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.010+0000 7fb0797fa640 1 -- 172.21.15.7:0/367862338 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb0700bc2c0 con 0x7fb07c102e00 2024-09-10T10:44:39.011 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-10T10:44:39.011 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAXI+Bm6FWKABAADXvT/xUZXabMx/4os/Qmhw== 2024-09-10T10:44:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 -- 172.21.15.7:0/367862338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb0640a2c10 msgr2=0x7fb0640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb0640a2c10 0x7fb0640a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb07c102c60 tx=0x7fb06c00b390 comp rx=0 tx=0).stop 2024-09-10T10:44:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 -- 172.21.15.7:0/367862338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07c102e00 msgr2=0x7fb07c0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07c102e00 0x7fb07c0745e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb070009230 tx=0x7fb070004360 comp rx=0 tx=0).stop 2024-09-10T10:44:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 -- 172.21.15.7:0/367862338 shutdown_connections 2024-09-10T10:44:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb0640a2c10 0x7fb0640a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb07c1037c0 0x7fb07c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07c102e00 0x7fb07c0745e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 --2- 172.21.15.7:0/367862338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb07c101c00 0x7fb07c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 -- 172.21.15.7:0/367862338 >> 172.21.15.7:0/367862338 conn(0x7fb07c0fd390 msgr2=0x7fb07c106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 -- 172.21.15.7:0/367862338 shutdown_connections 2024-09-10T10:44:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:39.012+0000 7fb0827d2640 1 -- 172.21.15.7:0/367862338 wait complete. 2024-09-10T10:44:39.625 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:44:39.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T10:44:39.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T10:44:39.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:39 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/367862338' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T10:44:39.665 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:39 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/367862338' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T10:44:39.666 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T10:44:39.666 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T10:44:39.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T10:44:39.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:39 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/367862338' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T10:44:39.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:39 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/367862338' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T10:44:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:39 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/367862338' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T10:44:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:39 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/367862338' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T10:44:39.965 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.372+0000 7f0a69b05640 1 -- 172.21.15.7:0/3628311489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a64106f10 msgr2=0x7f0a6410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.372+0000 7f0a69b05640 1 --2- 172.21.15.7:0/3628311489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a64106f10 0x7f0a6410ea90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009ca0 tx=0x7f0a5c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.372+0000 7f0a69b05640 1 -- 172.21.15.7:0/3628311489 shutdown_connections 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.372+0000 7f0a69b05640 1 --2- 172.21.15.7:0/3628311489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a64106f10 0x7f0a6410ea90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.372+0000 7f0a69b05640 1 --2- 172.21.15.7:0/3628311489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a64102790 0x7f0a641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.372+0000 7f0a69b05640 1 --2- 172.21.15.7:0/3628311489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a64101de0 0x7f0a641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.372+0000 7f0a69b05640 1 -- 172.21.15.7:0/3628311489 >> 172.21.15.7:0/3628311489 conn(0x7f0a640fd8c0 msgr2=0x7f0a640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.372+0000 7f0a69b05640 1 -- 172.21.15.7:0/3628311489 shutdown_connections 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.372+0000 7f0a69b05640 1 -- 172.21.15.7:0/3628311489 wait complete. 2024-09-10T10:44:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.373+0000 7f0a69b05640 1 Processor -- start 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.373+0000 7f0a69b05640 1 -- start start 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.373+0000 7f0a69b05640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a64101de0 0x7f0a64198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.373+0000 7f0a69b05640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a64102790 0x7f0a641989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.373+0000 7f0a69b05640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a64106f10 0x7f0a6419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a69b05640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a64111f90 con 0x7f0a64106f10 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a69b05640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a64111e10 con 0x7f0a64101de0 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a69b05640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0a64112110 con 0x7f0a64102790 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a63fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a64106f10 0x7f0a6419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a63fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a64106f10 0x7f0a6419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40850/0 (socket says 172.21.15.7:40850) 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a63fff640 1 -- 172.21.15.7:0/769072338 learned_addr learned my addr 172.21.15.7:0/769072338 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a637fe640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a64101de0 0x7f0a64198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a62ffd640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a64102790 0x7f0a641989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a63fff640 1 -- 172.21.15.7:0/769072338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a64102790 msgr2=0x7f0a641989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a63fff640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a64102790 0x7f0a641989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a63fff640 1 -- 172.21.15.7:0/769072338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a64101de0 msgr2=0x7f0a64198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a63fff640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a64101de0 0x7f0a64198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a63fff640 1 -- 172.21.15.7:0/769072338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6419d410 con 0x7f0a64106f10 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a62ffd640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a64102790 0x7f0a641989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a63fff640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a64106f10 0x7f0a6419cd30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c031a20 tx=0x7f0a5c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a60ff9640 1 -- 172.21.15.7:0/769072338 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c003c30 con 0x7f0a64106f10 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a60ff9640 1 -- 172.21.15.7:0/769072338 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a5c003dd0 con 0x7f0a64106f10 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a69b05640 1 -- 172.21.15.7:0/769072338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6419d6a0 con 0x7f0a64106f10 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a60ff9640 1 -- 172.21.15.7:0/769072338 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c02fcf0 con 0x7f0a64106f10 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.374+0000 7f0a69b05640 1 -- 172.21.15.7:0/769072338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a641a4fe0 con 0x7f0a64106f10 2024-09-10T10:44:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.375+0000 7f0a69b05640 1 -- 172.21.15.7:0/769072338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a641021c0 con 0x7f0a64106f10 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.377+0000 7f0a60ff9640 1 -- 172.21.15.7:0/769072338 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0a5c004050 con 0x7f0a64106f10 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.377+0000 7f0a60ff9640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0a4c0a2c10 0x7f0a4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.377+0000 7f0a637fe640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0a4c0a2c10 0x7f0a4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.377+0000 7f0a60ff9640 1 -- 172.21.15.7:0/769072338 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f0a5c0f0500 con 0x7f0a64106f10 2024-09-10T10:44:40.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.378+0000 7f0a637fe640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0a4c0a2c10 0x7f0a4c0a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009890 tx=0x7f0a54008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.381+0000 7f0a60ff9640 1 -- 172.21.15.7:0/769072338 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a5c0bd4b0 con 0x7f0a64106f10 2024-09-10T10:44:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.608+0000 7f0a69b05640 1 -- 172.21.15.7:0/769072338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0a6410a150 con 0x7f0a64106f10 2024-09-10T10:44:40.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.613+0000 7f0a60ff9640 1 -- 172.21.15.7:0/769072338 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263191 (secure 0 0 0) 0x7f0a5c02fe90 con 0x7f0a64106f10 2024-09-10T10:44:40.617 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:40.619 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:40 smithi007 ceph-mon[30873]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-09-10T10:44:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 -- 172.21.15.7:0/769072338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0a4c0a2c10 msgr2=0x7f0a4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0a4c0a2c10 0x7f0a4c0a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009890 tx=0x7f0a54008040 comp rx=0 tx=0).stop 2024-09-10T10:44:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 -- 172.21.15.7:0/769072338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a64106f10 msgr2=0x7f0a6419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a64106f10 0x7f0a6419cd30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c031a20 tx=0x7f0a5c031a50 comp rx=0 tx=0).stop 2024-09-10T10:44:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 -- 172.21.15.7:0/769072338 shutdown_connections 2024-09-10T10:44:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0a4c0a2c10 0x7f0a4c0a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a64106f10 0x7f0a6419cd30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a64102790 0x7f0a641989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 --2- 172.21.15.7:0/769072338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a64101de0 0x7f0a64198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 -- 172.21.15.7:0/769072338 >> 172.21.15.7:0/769072338 conn(0x7f0a640fd8c0 msgr2=0x7f0a640fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 -- 172.21.15.7:0/769072338 shutdown_connections 2024-09-10T10:44:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:40.616+0000 7f0a69b05640 1 -- 172.21.15.7:0/769072338 wait complete. 2024-09-10T10:44:40.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:40 smithi133 ceph-mon[35282]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-09-10T10:44:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:40 smithi103 ceph-mon[35160]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-09-10T10:44:41.251 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3525096194},{"type":"v1","addr":"172.21.15.7:6801","nonce":3525096194}]},"active_addr":"172.21.15.7:6801/3525096194","active_change":"2024-09-10T10:37:20.588368+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.7:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":959611916}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3214781379}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1846259971}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1740947437}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3099326751}]}]} 2024-09-10T10:44:41.257 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T10:44:41.257 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T10:44:41.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:44:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:41.629 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:41 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/769072338' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T10:44:41.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:41 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/769072338' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T10:44:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:41 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/769072338' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T10:44:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 -- 172.21.15.7:0/3655900611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be4102ef0 msgr2=0x7f6be410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 --2- 172.21.15.7:0/3655900611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be4102ef0 0x7f6be410da40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009ca0 tx=0x7f6bdc02a950 comp rx=0 tx=0).stop 2024-09-10T10:44:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 -- 172.21.15.7:0/3655900611 shutdown_connections 2024-09-10T10:44:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 --2- 172.21.15.7:0/3655900611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be4102ef0 0x7f6be410da40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 --2- 172.21.15.7:0/3655900611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be4102570 0x7f6be41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 --2- 172.21.15.7:0/3655900611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6be4100e20 0x7f6be4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 -- 172.21.15.7:0/3655900611 >> 172.21.15.7:0/3655900611 conn(0x7f6be40fc7d0 msgr2=0x7f6be40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 -- 172.21.15.7:0/3655900611 shutdown_connections 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 -- 172.21.15.7:0/3655900611 wait complete. 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 Processor -- start 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.929+0000 7f6beb6a5640 1 -- start start 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6beb6a5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6be4100e20 0x7f6be419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6beb6a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be4102570 0x7f6be419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6beb6a5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be4102ef0 0x7f6be4196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6beb6a5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4110f80 con 0x7f6be4102570 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6beb6a5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4110e00 con 0x7f6be4100e20 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6beb6a5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6be4111100 con 0x7f6be4102ef0 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be8c40640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be4102570 0x7f6be419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be8c40640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be4102570 0x7f6be419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40872/0 (socket says 172.21.15.7:40872) 2024-09-10T10:44:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be8c40640 1 -- 172.21.15.7:0/1618330801 learned_addr learned my addr 172.21.15.7:0/1618330801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be9441640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6be4100e20 0x7f6be419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be9c42640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be4102ef0 0x7f6be4196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be8c40640 1 -- 172.21.15.7:0/1618330801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be4102ef0 msgr2=0x7f6be4196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be8c40640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be4102ef0 0x7f6be4196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be8c40640 1 -- 172.21.15.7:0/1618330801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6be4100e20 msgr2=0x7f6be419e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be8c40640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6be4100e20 0x7f6be419e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be8c40640 1 -- 172.21.15.7:0/1618330801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be4196cf0 con 0x7f6be4102570 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be9c42640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be4102ef0 0x7f6be4196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6be8c40640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be4102570 0x7f6be419e670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400c920 tx=0x7f6bd400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6bda7fc640 1 -- 172.21.15.7:0/1618330801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4007c70 con 0x7f6be4102570 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6beb6a5640 1 -- 172.21.15.7:0/1618330801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be4196fe0 con 0x7f6be4102570 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6bda7fc640 1 -- 172.21.15.7:0/1618330801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd4007e10 con 0x7f6be4102570 2024-09-10T10:44:41.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6bda7fc640 1 -- 172.21.15.7:0/1618330801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4018890 con 0x7f6be4102570 2024-09-10T10:44:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.930+0000 7f6beb6a5640 1 -- 172.21.15.7:0/1618330801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be41b3e00 con 0x7f6be4102570 2024-09-10T10:44:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.932+0000 7f6beb6a5640 1 -- 172.21.15.7:0/1618330801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be41021b0 con 0x7f6be4102570 2024-09-10T10:44:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.933+0000 7f6bda7fc640 1 -- 172.21.15.7:0/1618330801 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6bd4018a70 con 0x7f6be4102570 2024-09-10T10:44:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.933+0000 7f6bda7fc640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f6bd00a2c10 0x7f6bd00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.933+0000 7f6be9441640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f6bd00a2c10 0x7f6bd00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.933+0000 7f6bda7fc640 1 -- 172.21.15.7:0/1618330801 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f6bd40cf340 con 0x7f6be4102570 2024-09-10T10:44:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.934+0000 7f6be9441640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f6bd00a2c10 0x7f6bd00a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6be00058c0 tx=0x7f6be000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:41.938+0000 7f6bda7fc640 1 -- 172.21.15.7:0/1618330801 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bd4014030 con 0x7f6be4102570 2024-09-10T10:44:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.116+0000 7f6beb6a5640 1 -- 172.21.15.7:0/1618330801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6be41b4030 con 0x7f6be4102570 2024-09-10T10:44:42.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.117+0000 7f6bda7fc640 1 -- 172.21.15.7:0/1618330801 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22646 (secure 0 0 0) 0x7f6bd409c150 con 0x7f6be4102570 2024-09-10T10:44:42.119 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:42.120 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:36:33.117443+0000","modified":"2024-09-10T10:44:22.070795+0000","last_up_change":"2024-09-10T10:43:23.913978+0000","last_in_change":"2024-09-10T10:42:59.733844+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:39:37.104114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:07.441548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:09.970273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9239e99-3a62-4883-b2f1-8964ebb54492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6803","nonce":704452390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6805","nonce":704452390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6809","nonce":704452390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6807","nonce":704452390}]},"public_addr":"172.21.15.7:6803/704452390","cluster_addr":"172.21.15.7:6805/704452390","heartbeat_back_addr":"172.21.15.7:6809/704452390","heartbeat_front_addr":"172.21.15.7:6807/704452390","state":["exists","up"]},{"osd":1,"uuid":"d750ff16-48c8-421d-b46b-49f0d2b7f4e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6801","nonce":2409158189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6803","nonce":2409158189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6807","nonce":2409158189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6805","nonce":2409158189}]},"public_addr":"172.21.15.103:6801/2409158189","cluster_addr":"172.21.15.103:6803/2409158189","heartbeat_back_addr":"172.21.15.103:6807/2409158189","heartbeat_front_addr":"172.21.15.103:6805/2409158189","state":["exists","up"]},{"osd":2,"uuid":"1ecb070a-a2f1-499a-99e7-f1967ebb52ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6801","nonce":1657349858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6803","nonce":1657349858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6807","nonce":1657349858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6805","nonce":1657349858}]},"public_addr":"172.21.15.133:6801/1657349858","cluster_addr":"172.21.15.133:6803/1657349858","heartbeat_back_addr":"172.21.15.133:6807/1657349858","heartbeat_front_addr":"172.21.15.133:6805/1657349858","state":["exists","up"]},{"osd":3,"uuid":"8e53d79e-038f-4a7f-9ac9-16b60f47bb4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6811","nonce":871781144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6813","nonce":871781144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6817","nonce":871781144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6815","nonce":871781144}]},"public_addr":"172.21.15.7:6811/871781144","cluster_addr":"172.21.15.7:6813/871781144","heartbeat_back_addr":"172.21.15.7:6817/871781144","heartbeat_front_addr":"172.21.15.7:6815/871781144","state":["exists","up"]},{"osd":4,"uuid":"e4f04c13-594c-43dd-9a98-d2138061b1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6809","nonce":1937243043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6811","nonce":1937243043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6815","nonce":1937243043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6813","nonce":1937243043}]},"public_addr":"172.21.15.103:6809/1937243043","cluster_addr":"172.21.15.103:6811/1937243043","heartbeat_back_addr":"172.21.15.103:6815/1937243043","heartbeat_front_addr":"172.21.15.103:6813/1937243043","state":["exists","up"]},{"osd":5,"uuid":"73143170-5816-407d-ae03-dcf896057681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6809","nonce":577882774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6811","nonce":577882774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6815","nonce":577882774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6813","nonce":577882774}]},"public_addr":"172.21.15.133:6809/577882774","cluster_addr":"172.21.15.133:6811/577882774","heartbeat_back_addr":"172.21.15.133:6815/577882774","heartbeat_front_addr":"172.21.15.133:6813/577882774","state":["exists","up"]},{"osd":6,"uuid":"3a5904d5-36de-47b0-97fa-5a10b24d0c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6819","nonce":2646192734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6821","nonce":2646192734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6825","nonce":2646192734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6823","nonce":2646192734}]},"public_addr":"172.21.15.7:6819/2646192734","cluster_addr":"172.21.15.7:6821/2646192734","heartbeat_back_addr":"172.21.15.7:6825/2646192734","heartbeat_front_addr":"172.21.15.7:6823/2646192734","state":["exists","up"]},{"osd":7,"uuid":"91e79d42-decd-4405-b9f2-9480f7edfca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6817","nonce":3858867236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6819","nonce":3858867236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6823","nonce":3858867236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6821","nonce":3858867236}]},"public_addr":"172.21.15.103:6817/3858867236","cluster_addr":"172.21.15.103:6819/3858867236","heartbeat_back_addr":"172.21.15.103:6823/3858867236","heartbeat_front_addr":"172.21.15.103:6821/3858867236","state":["exists","up"]},{"osd":8,"uuid":"64373754-e81a-430c-967d-4b4f289f3473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6817","nonce":131428321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6819","nonce":131428321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6823","nonce":131428321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6821","nonce":131428321}]},"public_addr":"172.21.15.133:6817/131428321","cluster_addr":"172.21.15.133:6819/131428321","heartbeat_back_addr":"172.21.15.133:6823/131428321","heartbeat_front_addr":"172.21.15.133:6821/131428321","state":["exists","up"]},{"osd":9,"uuid":"91ebe2c9-bda3-4596-ac76-5678c71f850b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6827","nonce":945633240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6829","nonce":945633240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6833","nonce":945633240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6831","nonce":945633240}]},"public_addr":"172.21.15.7:6827/945633240","cluster_addr":"172.21.15.7:6829/945633240","heartbeat_back_addr":"172.21.15.7:6833/945633240","heartbeat_front_addr":"172.21.15.7:6831/945633240","state":["exists","up"]},{"osd":10,"uuid":"2f65568f-89d1-4a67-a222-8e263d34c710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6825","nonce":907835659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6827","nonce":907835659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6831","nonce":907835659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6829","nonce":907835659}]},"public_addr":"172.21.15.103:6825/907835659","cluster_addr":"172.21.15.103:6827/907835659","heartbeat_back_addr":"172.21.15.103:6831/907835659","heartbeat_front_addr":"172.21.15.103:6829/907835659","state":["exists","up"]},{"osd":11,"uuid":"4b49839b-32e9-4105-abee-a253f1aa3f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6825","nonce":2538184038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6827","nonce":2538184038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6831","nonce":2538184038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6829","nonce":2538184038}]},"public_addr":"172.21.15.133:6825/2538184038","cluster_addr":"172.21.15.133:6827/2538184038","heartbeat_back_addr":"172.21.15.133:6831/2538184038","heartbeat_front_addr":"172.21.15.133:6829/2538184038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:44.541279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:10.103733+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:35.290268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:01.070763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:25.666941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:50.925733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:16.486537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:41.464155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:06.610089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:31.994080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:57.000696+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:21.398669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3394895925":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6800/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/262340568":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/3768226338":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/258851660":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3411487770":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/11850255":"2024-09-11T10:36:55.407956+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.119+0000 7f6beb6a5640 1 -- 172.21.15.7:0/1618330801 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f6bd00a2c10 msgr2=0x7f6bd00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.119+0000 7f6beb6a5640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f6bd00a2c10 0x7f6bd00a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6be00058c0 tx=0x7f6be000a4c0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.119+0000 7f6beb6a5640 1 -- 172.21.15.7:0/1618330801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be4102570 msgr2=0x7f6be419e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.119+0000 7f6beb6a5640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be4102570 0x7f6be419e670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400c920 tx=0x7f6bd400cde0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.119+0000 7f6beb6a5640 1 -- 172.21.15.7:0/1618330801 shutdown_connections 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.119+0000 7f6beb6a5640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f6bd00a2c10 0x7f6bd00a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.119+0000 7f6beb6a5640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6be4102ef0 0x7f6be4196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.119+0000 7f6beb6a5640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6be4102570 0x7f6be419e670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.119+0000 7f6beb6a5640 1 --2- 172.21.15.7:0/1618330801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6be4100e20 0x7f6be419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.119+0000 7f6beb6a5640 1 -- 172.21.15.7:0/1618330801 >> 172.21.15.7:0/1618330801 conn(0x7f6be40fc7d0 msgr2=0x7f6be4069fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.120+0000 7f6beb6a5640 1 -- 172.21.15.7:0/1618330801 shutdown_connections 2024-09-10T10:44:42.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:42.120+0000 7f6beb6a5640 1 -- 172.21.15.7:0/1618330801 wait complete. 2024-09-10T10:44:42.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:42 smithi133 ceph-mon[35282]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-10T10:44:42.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:42 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1618330801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:42.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:42 smithi007 ceph-mon[30873]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-10T10:44:42.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:42 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1618330801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:42.818 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T10:44:42.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:44:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:42 smithi103 ceph-mon[35160]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-10T10:44:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:42 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1618330801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:43.080 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:43.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.524+0000 7f5f66328640 1 -- 172.21.15.7:0/2807748114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f60103660 msgr2=0x7f5f6010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.524+0000 7f5f66328640 1 --2- 172.21.15.7:0/2807748114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f60103660 0x7f5f6010bb50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009900 tx=0x7f5f5402a480 comp rx=0 tx=0).stop 2024-09-10T10:44:43.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.524+0000 7f5f66328640 1 -- 172.21.15.7:0/2807748114 shutdown_connections 2024-09-10T10:44:43.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.524+0000 7f5f66328640 1 --2- 172.21.15.7:0/2807748114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f6010c170 0x7f5f6010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.524+0000 7f5f66328640 1 --2- 172.21.15.7:0/2807748114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f60103660 0x7f5f6010bb50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.524+0000 7f5f66328640 1 --2- 172.21.15.7:0/2807748114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f60102d40 0x7f5f60103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.524+0000 7f5f66328640 1 -- 172.21.15.7:0/2807748114 >> 172.21.15.7:0/2807748114 conn(0x7f5f600fc7a0 msgr2=0x7f5f600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:43.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.525+0000 7f5f66328640 1 -- 172.21.15.7:0/2807748114 shutdown_connections 2024-09-10T10:44:43.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.525+0000 7f5f66328640 1 -- 172.21.15.7:0/2807748114 wait complete. 2024-09-10T10:44:43.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.525+0000 7f5f66328640 1 Processor -- start 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.525+0000 7f5f66328640 1 -- start start 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f66328640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f60102d40 0x7f5f601009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f66328640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f60103660 0x7f5f60100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f66328640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f6010c170 0x7f5f60101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f66328640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60111f70 con 0x7f5f60102d40 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f66328640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60111df0 con 0x7f5f6010c170 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f66328640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f601120f0 con 0x7f5f60103660 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f60102d40 0x7f5f601009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f60102d40 0x7f5f601009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40900/0 (socket says 172.21.15.7:40900) 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5ffff640 1 -- 172.21.15.7:0/2168546898 learned_addr learned my addr 172.21.15.7:0/2168546898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5f7fe640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f60103660 0x7f5f60100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f648c5640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f6010c170 0x7f5f60101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5ffff640 1 -- 172.21.15.7:0/2168546898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f60103660 msgr2=0x7f5f60100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5ffff640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f60103660 0x7f5f60100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5ffff640 1 -- 172.21.15.7:0/2168546898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f6010c170 msgr2=0x7f5f60101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5ffff640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f6010c170 0x7f5f60101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5ffff640 1 -- 172.21.15.7:0/2168546898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f601a6930 con 0x7f5f60102d40 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5f7fe640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f60103660 0x7f5f60100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5ffff640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f60102d40 0x7f5f601009a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000c890 tx=0x7f5f5000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.526+0000 7f5f5d7fa640 1 -- 172.21.15.7:0/2168546898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50004220 con 0x7f5f60102d40 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.527+0000 7f5f66328640 1 -- 172.21.15.7:0/2168546898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f601a6c20 con 0x7f5f60102d40 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.527+0000 7f5f5d7fa640 1 -- 172.21.15.7:0/2168546898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f500043c0 con 0x7f5f60102d40 2024-09-10T10:44:43.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.527+0000 7f5f5d7fa640 1 -- 172.21.15.7:0/2168546898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f500049e0 con 0x7f5f60102d40 2024-09-10T10:44:43.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.527+0000 7f5f66328640 1 -- 172.21.15.7:0/2168546898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f601a7160 con 0x7f5f60102d40 2024-09-10T10:44:43.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.528+0000 7f5f66328640 1 -- 172.21.15.7:0/2168546898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f60103120 con 0x7f5f60102d40 2024-09-10T10:44:43.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.529+0000 7f5f5d7fa640 1 -- 172.21.15.7:0/2168546898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5f500079b0 con 0x7f5f60102d40 2024-09-10T10:44:43.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.529+0000 7f5f5d7fa640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5f480a2c10 0x7f5f480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:43.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.530+0000 7f5f5f7fe640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5f480a2c10 0x7f5f480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:43.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.530+0000 7f5f5d7fa640 1 -- 172.21.15.7:0/2168546898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f5f500cf2d0 con 0x7f5f60102d40 2024-09-10T10:44:43.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.530+0000 7f5f5f7fe640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5f480a2c10 0x7f5f480a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5f60102430 tx=0x7f5f54005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:43.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.534+0000 7f5f5d7fa640 1 -- 172.21.15.7:0/2168546898 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f50014030 con 0x7f5f60102d40 2024-09-10T10:44:43.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.710+0000 7f5f66328640 1 -- 172.21.15.7:0/2168546898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f60106c50 con 0x7f5f60102d40 2024-09-10T10:44:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.712+0000 7f5f5d7fa640 1 -- 172.21.15.7:0/2168546898 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22646 (secure 0 0 0) 0x7f5f5009c0e0 con 0x7f5f60102d40 2024-09-10T10:44:43.713 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:44:43.714 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:36:33.117443+0000","modified":"2024-09-10T10:44:22.070795+0000","last_up_change":"2024-09-10T10:43:23.913978+0000","last_in_change":"2024-09-10T10:42:59.733844+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:39:37.104114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:07.441548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:09.970273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9239e99-3a62-4883-b2f1-8964ebb54492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6803","nonce":704452390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6805","nonce":704452390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6809","nonce":704452390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6807","nonce":704452390}]},"public_addr":"172.21.15.7:6803/704452390","cluster_addr":"172.21.15.7:6805/704452390","heartbeat_back_addr":"172.21.15.7:6809/704452390","heartbeat_front_addr":"172.21.15.7:6807/704452390","state":["exists","up"]},{"osd":1,"uuid":"d750ff16-48c8-421d-b46b-49f0d2b7f4e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6801","nonce":2409158189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6803","nonce":2409158189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6807","nonce":2409158189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6805","nonce":2409158189}]},"public_addr":"172.21.15.103:6801/2409158189","cluster_addr":"172.21.15.103:6803/2409158189","heartbeat_back_addr":"172.21.15.103:6807/2409158189","heartbeat_front_addr":"172.21.15.103:6805/2409158189","state":["exists","up"]},{"osd":2,"uuid":"1ecb070a-a2f1-499a-99e7-f1967ebb52ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6801","nonce":1657349858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6803","nonce":1657349858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6807","nonce":1657349858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6805","nonce":1657349858}]},"public_addr":"172.21.15.133:6801/1657349858","cluster_addr":"172.21.15.133:6803/1657349858","heartbeat_back_addr":"172.21.15.133:6807/1657349858","heartbeat_front_addr":"172.21.15.133:6805/1657349858","state":["exists","up"]},{"osd":3,"uuid":"8e53d79e-038f-4a7f-9ac9-16b60f47bb4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6811","nonce":871781144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6813","nonce":871781144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6817","nonce":871781144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6815","nonce":871781144}]},"public_addr":"172.21.15.7:6811/871781144","cluster_addr":"172.21.15.7:6813/871781144","heartbeat_back_addr":"172.21.15.7:6817/871781144","heartbeat_front_addr":"172.21.15.7:6815/871781144","state":["exists","up"]},{"osd":4,"uuid":"e4f04c13-594c-43dd-9a98-d2138061b1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6809","nonce":1937243043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6811","nonce":1937243043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6815","nonce":1937243043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6813","nonce":1937243043}]},"public_addr":"172.21.15.103:6809/1937243043","cluster_addr":"172.21.15.103:6811/1937243043","heartbeat_back_addr":"172.21.15.103:6815/1937243043","heartbeat_front_addr":"172.21.15.103:6813/1937243043","state":["exists","up"]},{"osd":5,"uuid":"73143170-5816-407d-ae03-dcf896057681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6809","nonce":577882774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6811","nonce":577882774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6815","nonce":577882774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6813","nonce":577882774}]},"public_addr":"172.21.15.133:6809/577882774","cluster_addr":"172.21.15.133:6811/577882774","heartbeat_back_addr":"172.21.15.133:6815/577882774","heartbeat_front_addr":"172.21.15.133:6813/577882774","state":["exists","up"]},{"osd":6,"uuid":"3a5904d5-36de-47b0-97fa-5a10b24d0c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6819","nonce":2646192734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6821","nonce":2646192734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6825","nonce":2646192734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6823","nonce":2646192734}]},"public_addr":"172.21.15.7:6819/2646192734","cluster_addr":"172.21.15.7:6821/2646192734","heartbeat_back_addr":"172.21.15.7:6825/2646192734","heartbeat_front_addr":"172.21.15.7:6823/2646192734","state":["exists","up"]},{"osd":7,"uuid":"91e79d42-decd-4405-b9f2-9480f7edfca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6817","nonce":3858867236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6819","nonce":3858867236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6823","nonce":3858867236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6821","nonce":3858867236}]},"public_addr":"172.21.15.103:6817/3858867236","cluster_addr":"172.21.15.103:6819/3858867236","heartbeat_back_addr":"172.21.15.103:6823/3858867236","heartbeat_front_addr":"172.21.15.103:6821/3858867236","state":["exists","up"]},{"osd":8,"uuid":"64373754-e81a-430c-967d-4b4f289f3473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6817","nonce":131428321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6819","nonce":131428321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6823","nonce":131428321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6821","nonce":131428321}]},"public_addr":"172.21.15.133:6817/131428321","cluster_addr":"172.21.15.133:6819/131428321","heartbeat_back_addr":"172.21.15.133:6823/131428321","heartbeat_front_addr":"172.21.15.133:6821/131428321","state":["exists","up"]},{"osd":9,"uuid":"91ebe2c9-bda3-4596-ac76-5678c71f850b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6827","nonce":945633240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6829","nonce":945633240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6833","nonce":945633240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6831","nonce":945633240}]},"public_addr":"172.21.15.7:6827/945633240","cluster_addr":"172.21.15.7:6829/945633240","heartbeat_back_addr":"172.21.15.7:6833/945633240","heartbeat_front_addr":"172.21.15.7:6831/945633240","state":["exists","up"]},{"osd":10,"uuid":"2f65568f-89d1-4a67-a222-8e263d34c710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6825","nonce":907835659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6827","nonce":907835659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6831","nonce":907835659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6829","nonce":907835659}]},"public_addr":"172.21.15.103:6825/907835659","cluster_addr":"172.21.15.103:6827/907835659","heartbeat_back_addr":"172.21.15.103:6831/907835659","heartbeat_front_addr":"172.21.15.103:6829/907835659","state":["exists","up"]},{"osd":11,"uuid":"4b49839b-32e9-4105-abee-a253f1aa3f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6825","nonce":2538184038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6827","nonce":2538184038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6831","nonce":2538184038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6829","nonce":2538184038}]},"public_addr":"172.21.15.133:6825/2538184038","cluster_addr":"172.21.15.133:6827/2538184038","heartbeat_back_addr":"172.21.15.133:6831/2538184038","heartbeat_front_addr":"172.21.15.133:6829/2538184038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:44.541279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:10.103733+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:35.290268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:01.070763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:25.666941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:50.925733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:16.486537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:41.464155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:06.610089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:31.994080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:57.000696+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:21.398669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3394895925":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6800/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/262340568":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/3768226338":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/258851660":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3411487770":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/11850255":"2024-09-11T10:36:55.407956+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.714+0000 7f5f66328640 1 -- 172.21.15.7:0/2168546898 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5f480a2c10 msgr2=0x7f5f480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.714+0000 7f5f66328640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5f480a2c10 0x7f5f480a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5f60102430 tx=0x7f5f54005a20 comp rx=0 tx=0).stop 2024-09-10T10:44:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.714+0000 7f5f66328640 1 -- 172.21.15.7:0/2168546898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f60102d40 msgr2=0x7f5f601009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.714+0000 7f5f66328640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f60102d40 0x7f5f601009a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000c890 tx=0x7f5f5000cd50 comp rx=0 tx=0).stop 2024-09-10T10:44:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.715+0000 7f5f66328640 1 -- 172.21.15.7:0/2168546898 shutdown_connections 2024-09-10T10:44:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.715+0000 7f5f66328640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5f480a2c10 0x7f5f480a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.715+0000 7f5f66328640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f6010c170 0x7f5f60101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.715+0000 7f5f66328640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f60103660 0x7f5f60100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.715+0000 7f5f66328640 1 --2- 172.21.15.7:0/2168546898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f60102d40 0x7f5f601009a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.715+0000 7f5f66328640 1 -- 172.21.15.7:0/2168546898 >> 172.21.15.7:0/2168546898 conn(0x7f5f600fc7a0 msgr2=0x7f5f600fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.715+0000 7f5f66328640 1 -- 172.21.15.7:0/2168546898 shutdown_connections 2024-09-10T10:44:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:43.715+0000 7f5f66328640 1 -- 172.21.15.7:0/2168546898 wait complete. 2024-09-10T10:44:44.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T10:44:44.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T10:44:44.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T10:44:44.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T10:44:44.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T10:44:44.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T10:44:44.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T10:44:44.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T10:44:44.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T10:44:44.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T10:44:44.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T10:44:44.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T10:44:44.497 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:44 smithi007 ceph-mon[30873]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-10T10:44:44.497 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:44 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2168546898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:44.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:44 smithi133 ceph-mon[35282]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-10T10:44:44.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:44 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2168546898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:44.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:44 smithi103 ceph-mon[35160]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-09-10T10:44:44.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:44 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2168546898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:44:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.210 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.938+0000 7f076600e640 1 -- 172.21.15.7:0/2209370734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0760102790 msgr2=0x7f07601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.938+0000 7f076600e640 1 --2- 172.21.15.7:0/2209370734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0760102790 0x7f07601067e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0748009900 tx=0x7f074802a410 comp rx=0 tx=0).stop 2024-09-10T10:44:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.938+0000 7f076600e640 1 -- 172.21.15.7:0/2209370734 shutdown_connections 2024-09-10T10:44:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.938+0000 7f076600e640 1 --2- 172.21.15.7:0/2209370734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0760106f10 0x7f076010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.938+0000 7f076600e640 1 --2- 172.21.15.7:0/2209370734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0760102790 0x7f07601067e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.938+0000 7f076600e640 1 --2- 172.21.15.7:0/2209370734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0760101de0 0x7f07601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.938+0000 7f076600e640 1 -- 172.21.15.7:0/2209370734 >> 172.21.15.7:0/2209370734 conn(0x7f07600fd8c0 msgr2=0x7f07600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:45.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.938+0000 7f076600e640 1 -- 172.21.15.7:0/2209370734 shutdown_connections 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.939+0000 7f076600e640 1 -- 172.21.15.7:0/2209370734 wait complete. 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.939+0000 7f076600e640 1 Processor -- start 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.939+0000 7f076600e640 1 -- start start 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.939+0000 7f076600e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0760101de0 0x7f0760198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.939+0000 7f076600e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0760102790 0x7f0760198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.939+0000 7f076600e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0760106f10 0x7f076019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.939+0000 7f076600e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0760111f30 con 0x7f0760101de0 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.939+0000 7f076600e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0760111db0 con 0x7f0760102790 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.939+0000 7f076600e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f07601120b0 con 0x7f0760106f10 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.939+0000 7f075f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0760101de0 0x7f0760198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0760101de0 0x7f0760198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40928/0 (socket says 172.21.15.7:40928) 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075f7fe640 1 -- 172.21.15.7:0/1624128635 learned_addr learned my addr 172.21.15.7:0/1624128635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:45.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f0757fff640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0760102790 0x7f0760198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075ffff640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0760106f10 0x7f076019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075f7fe640 1 -- 172.21.15.7:0/1624128635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0760106f10 msgr2=0x7f076019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075f7fe640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0760106f10 0x7f076019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075f7fe640 1 -- 172.21.15.7:0/1624128635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0760102790 msgr2=0x7f0760198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075f7fe640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0760102790 0x7f0760198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075f7fe640 1 -- 172.21.15.7:0/1624128635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076019d400 con 0x7f0760101de0 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075ffff640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0760106f10 0x7f076019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075f7fe640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0760101de0 0x7f0760198450 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f075000c920 tx=0x7f075000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075d7fa640 1 -- 172.21.15.7:0/1624128635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750007c70 con 0x7f0760101de0 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075d7fa640 1 -- 172.21.15.7:0/1624128635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0750007e10 con 0x7f0760101de0 2024-09-10T10:44:45.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076019d6f0 con 0x7f0760101de0 2024-09-10T10:44:45.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f075d7fa640 1 -- 172.21.15.7:0/1624128635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750018890 con 0x7f0760101de0 2024-09-10T10:44:45.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.940+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07601a4fd0 con 0x7f0760101de0 2024-09-10T10:44:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.941+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0760062900 con 0x7f0760101de0 2024-09-10T10:44:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.943+0000 7f075d7fa640 1 -- 172.21.15.7:0/1624128635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0750018a30 con 0x7f0760101de0 2024-09-10T10:44:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.943+0000 7f075d7fa640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f074c0a2c10 0x7f074c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.943+0000 7f0757fff640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f074c0a2c10 0x7f074c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.944+0000 7f075d7fa640 1 -- 172.21.15.7:0/1624128635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f07500cf730 con 0x7f0760101de0 2024-09-10T10:44:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.944+0000 7f075d7fa640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] conn(0x7f074c0ac7c0 0x7f074c0aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.944+0000 7f075d7fa640 1 -- 172.21.15.7:0/1624128635 --> [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f074c0af330 con 0x7f074c0ac7c0 2024-09-10T10:44:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.944+0000 7f075d7fa640 1 -- 172.21.15.7:0/1624128635 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f07500cfb80 con 0x7f0760101de0 2024-09-10T10:44:45.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.944+0000 7f0757fff640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f074c0a2c10 0x7f074c0a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0748009230 tx=0x7f07480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:45.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.944+0000 7f075ffff640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] conn(0x7f074c0ac7c0 0x7f074c0aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:45.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.945+0000 7f075ffff640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] conn(0x7f074c0ac7c0 0x7f074c0aec60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:45.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.945+0000 7f075d7fa640 1 -- 172.21.15.7:0/1624128635 <== osd.8 v2:172.21.15.133:6816/131428321 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f074c0af330 con 0x7f074c0ac7c0 2024-09-10T10:44:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.961+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 --> [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0760062d80 con 0x7f074c0ac7c0 2024-09-10T10:44:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.961+0000 7f075d7fa640 1 -- 172.21.15.7:0/1624128635 <== osd.8 v2:172.21.15.133:6816/131428321 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0760062d80 con 0x7f074c0ac7c0 2024-09-10T10:44:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.961+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 >> [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] conn(0x7f074c0ac7c0 msgr2=0x7f074c0aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.961+0000 7f076600e640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] conn(0x7f074c0ac7c0 0x7f074c0aec60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.962+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f074c0a2c10 msgr2=0x7f074c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.962+0000 7f076600e640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f074c0a2c10 0x7f074c0a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0748009230 tx=0x7f07480058d0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.962+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0760101de0 msgr2=0x7f0760198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.962+0000 7f076600e640 1 --2- 172.21.15.7:0/1624128635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0760101de0 0x7f0760198450 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f075000c920 tx=0x7f075000cde0 comp rx=0 tx=0).stop 2024-09-10T10:44:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.962+0000 7f075ffff640 1 -- 172.21.15.7:0/1624128635 reap_dead start 2024-09-10T10:44:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.962+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 shutdown_connections 2024-09-10T10:44:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.962+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 >> 172.21.15.7:0/1624128635 conn(0x7f07600fd8c0 msgr2=0x7f07600fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.962+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 shutdown_connections 2024-09-10T10:44:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:45.962+0000 7f076600e640 1 -- 172.21.15.7:0/1624128635 wait complete. 2024-09-10T10:44:45.970 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.974 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.974 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.975 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.976 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.979 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.979 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.982 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.990 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:45.998 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.368+0000 7f4bb3173640 1 -- 172.21.15.7:0/3080976561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bac1008c0 msgr2=0x7f4bac100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.368+0000 7f4bb3173640 1 --2- 172.21.15.7:0/3080976561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bac1008c0 0x7f4bac100ca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c009900 tx=0x7f4b9c02a410 comp rx=0 tx=0).stop 2024-09-10T10:44:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.368+0000 7f4bb3173640 1 -- 172.21.15.7:0/3080976561 shutdown_connections 2024-09-10T10:44:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.368+0000 7f4bb3173640 1 --2- 172.21.15.7:0/3080976561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bac105ec0 0x7f4bac106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.368+0000 7f4bb3173640 1 --2- 172.21.15.7:0/3080976561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bac1011e0 0x7f4bac105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.368+0000 7f4bb3173640 1 --2- 172.21.15.7:0/3080976561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bac1008c0 0x7f4bac100ca0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.368+0000 7f4bb3173640 1 -- 172.21.15.7:0/3080976561 >> 172.21.15.7:0/3080976561 conn(0x7f4bac0fc7d0 msgr2=0x7f4bac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.369+0000 7f4bb3173640 1 -- 172.21.15.7:0/3080976561 shutdown_connections 2024-09-10T10:44:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.369+0000 7f4bb3173640 1 -- 172.21.15.7:0/3080976561 wait complete. 2024-09-10T10:44:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.369+0000 7f4bb3173640 1 Processor -- start 2024-09-10T10:44:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.369+0000 7f4bb3173640 1 -- start start 2024-09-10T10:44:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4bb3173640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bac1008c0 0x7f4bac1a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4bb3173640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bac1011e0 0x7f4bac1a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4bb3173640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bac105ec0 0x7f4bac19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4bb3173640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac118660 con 0x7f4bac1011e0 2024-09-10T10:44:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4bb3173640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac1184e0 con 0x7f4bac1008c0 2024-09-10T10:44:46.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4bb3173640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac1187e0 con 0x7f4bac105ec0 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4babfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bac1011e0 0x7f4bac1a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4babfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bac1011e0 0x7f4bac1a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40958/0 (socket says 172.21.15.7:40958) 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4babfff640 1 -- 172.21.15.7:0/3684374595 learned_addr learned my addr 172.21.15.7:0/3684374595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4bb0f0f640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bac1008c0 0x7f4bac1a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4babfff640 1 -- 172.21.15.7:0/3684374595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bac105ec0 msgr2=0x7f4bac19e9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4babfff640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bac105ec0 0x7f4bac19e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4babfff640 1 -- 172.21.15.7:0/3684374595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bac1008c0 msgr2=0x7f4bac1a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4babfff640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bac1008c0 0x7f4bac1a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.370+0000 7f4babfff640 1 -- 172.21.15.7:0/3684374595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac19f0e0 con 0x7f4bac1011e0 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.371+0000 7f4babfff640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bac1011e0 0x7f4bac1a4d30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000d950 tx=0x7f4ba000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:46.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.371+0000 7f4ba9ffb640 1 -- 172.21.15.7:0/3684374595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0017030 con 0x7f4bac1011e0 2024-09-10T10:44:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.371+0000 7f4ba9ffb640 1 -- 172.21.15.7:0/3684374595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ba0004020 con 0x7f4bac1011e0 2024-09-10T10:44:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.371+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bac19f3d0 con 0x7f4bac1011e0 2024-09-10T10:44:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.371+0000 7f4ba9ffb640 1 -- 172.21.15.7:0/3684374595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00047a0 con 0x7f4bac1011e0 2024-09-10T10:44:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.371+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bac1ab6c0 con 0x7f4bac1011e0 2024-09-10T10:44:46.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.373+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bac1100d0 con 0x7f4bac1011e0 2024-09-10T10:44:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.375+0000 7f4ba9ffb640 1 -- 172.21.15.7:0/3684374595 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4ba00041d0 con 0x7f4bac1011e0 2024-09-10T10:44:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.376+0000 7f4ba9ffb640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4b940a2c10 0x7f4b940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.376+0000 7f4bb0f0f640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4b940a2c10 0x7f4b940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.376+0000 7f4ba9ffb640 1 -- 172.21.15.7:0/3684374595 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f4ba00cf2f0 con 0x7f4bac1011e0 2024-09-10T10:44:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.376+0000 7f4ba9ffb640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] conn(0x7f4b940ac7c0 0x7f4b940aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:46.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.376+0000 7f4ba9ffb640 1 -- 172.21.15.7:0/3684374595 --> [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b940af330 con 0x7f4b940ac7c0 2024-09-10T10:44:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.376+0000 7f4ba9ffb640 1 -- 172.21.15.7:0/3684374595 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f4ba00cf6a0 con 0x7f4bac1011e0 2024-09-10T10:44:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.376+0000 7f4bb0f0f640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4b940a2c10 0x7f4b940a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c006010 tx=0x7f4b9c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.376+0000 7f4bb1710640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] conn(0x7f4b940ac7c0 0x7f4b940aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.377+0000 7f4bb1710640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] conn(0x7f4b940ac7c0 0x7f4b940aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.378+0000 7f4ba9ffb640 1 -- 172.21.15.7:0/3684374595 <== osd.7 v2:172.21.15.103:6816/3858867236 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b940af330 con 0x7f4b940ac7c0 2024-09-10T10:44:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.393+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 --> [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4bac06a850 con 0x7f4b940ac7c0 2024-09-10T10:44:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.393+0000 7f4ba9ffb640 1 -- 172.21.15.7:0/3684374595 <== osd.7 v2:172.21.15.103:6816/3858867236 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4bac06a850 con 0x7f4b940ac7c0 2024-09-10T10:44:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 >> [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] conn(0x7f4b940ac7c0 msgr2=0x7f4b940aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb3173640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] conn(0x7f4b940ac7c0 0x7f4b940aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4b940a2c10 msgr2=0x7f4b940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb3173640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4b940a2c10 0x7f4b940a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c006010 tx=0x7f4b9c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bac1011e0 msgr2=0x7f4bac1a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb3173640 1 --2- 172.21.15.7:0/3684374595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bac1011e0 0x7f4bac1a4d30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000d950 tx=0x7f4ba000de10 comp rx=0 tx=0).stop 2024-09-10T10:44:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb1710640 1 -- 172.21.15.7:0/3684374595 reap_dead start 2024-09-10T10:44:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 shutdown_connections 2024-09-10T10:44:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 >> 172.21.15.7:0/3684374595 conn(0x7f4bac0fc7d0 msgr2=0x7f4bac10f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 shutdown_connections 2024-09-10T10:44:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:46.394+0000 7f4bb3173640 1 -- 172.21.15.7:0/3684374595 wait complete. 2024-09-10T10:44:46.614 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:46 smithi007 ceph-mon[30873]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-10T10:44:46.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:46 smithi133 ceph-mon[35282]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-10T10:44:46.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:46 smithi103 ceph-mon[35160]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-10T10:44:47.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.494+0000 7f7b3ea46640 1 -- 172.21.15.7:0/2959504099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38102d30 msgr2=0x7f7b38103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:47.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.494+0000 7f7b3ea46640 1 --2- 172.21.15.7:0/2959504099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38102d30 0x7f7b38103110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009930 tx=0x7f7b2802a440 comp rx=0 tx=0).stop 2024-09-10T10:44:47.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.494+0000 7f7b3ea46640 1 -- 172.21.15.7:0/2959504099 shutdown_connections 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.494+0000 7f7b3ea46640 1 --2- 172.21.15.7:0/2959504099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b381082a0 0x7f7b3810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.494+0000 7f7b3ea46640 1 --2- 172.21.15.7:0/2959504099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b381036e0 0x7f7b38107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.494+0000 7f7b3ea46640 1 --2- 172.21.15.7:0/2959504099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38102d30 0x7f7b38103110 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.494+0000 7f7b3ea46640 1 -- 172.21.15.7:0/2959504099 >> 172.21.15.7:0/2959504099 conn(0x7f7b380fc7d0 msgr2=0x7f7b380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.494+0000 7f7b3ea46640 1 -- 172.21.15.7:0/2959504099 shutdown_connections 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.494+0000 7f7b3ea46640 1 -- 172.21.15.7:0/2959504099 wait complete. 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.494+0000 7f7b3ea46640 1 Processor -- start 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b3ea46640 1 -- start start 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b3ea46640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38102d30 0x7f7b38198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b3ea46640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b381036e0 0x7f7b38198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b3ea46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b381082a0 0x7f7b3819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b3ea46640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b381120e0 con 0x7f7b38102d30 2024-09-10T10:44:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b3ea46640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38111f60 con 0x7f7b381082a0 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b3ea46640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38112260 con 0x7f7b381036e0 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38102d30 0x7f7b38198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38102d30 0x7f7b38198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40988/0 (socket says 172.21.15.7:40988) 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b37fff640 1 -- 172.21.15.7:0/3168568399 learned_addr learned my addr 172.21.15.7:0/3168568399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b2ffff640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b381036e0 0x7f7b38198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b37fff640 1 -- 172.21.15.7:0/3168568399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b381036e0 msgr2=0x7f7b38198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b37fff640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b381036e0 0x7f7b38198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b37fff640 1 -- 172.21.15.7:0/3168568399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b381082a0 msgr2=0x7f7b3819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b3cfe3640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b381082a0 0x7f7b3819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b37fff640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b381082a0 0x7f7b3819cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.495+0000 7f7b37fff640 1 -- 172.21.15.7:0/3168568399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3819d610 con 0x7f7b38102d30 2024-09-10T10:44:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.496+0000 7f7b37fff640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38102d30 0x7f7b38198660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009a50 tx=0x7f7b28004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:47.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.496+0000 7f7b35ffb640 1 -- 172.21.15.7:0/3168568399 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2803f030 con 0x7f7b38102d30 2024-09-10T10:44:47.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.496+0000 7f7b35ffb640 1 -- 172.21.15.7:0/3168568399 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b28003ce0 con 0x7f7b38102d30 2024-09-10T10:44:47.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.496+0000 7f7b35ffb640 1 -- 172.21.15.7:0/3168568399 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28035070 con 0x7f7b38102d30 2024-09-10T10:44:47.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.496+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3819d8a0 con 0x7f7b38102d30 2024-09-10T10:44:47.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.496+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b381a51e0 con 0x7f7b38102d30 2024-09-10T10:44:47.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.497+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b38062980 con 0x7f7b38102d30 2024-09-10T10:44:47.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.498+0000 7f7b35ffb640 1 -- 172.21.15.7:0/3168568399 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7b28042030 con 0x7f7b38102d30 2024-09-10T10:44:47.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.499+0000 7f7b35ffb640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f7b240a2c10 0x7f7b240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:47.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.499+0000 7f7b2ffff640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f7b240a2c10 0x7f7b240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.499+0000 7f7b35ffb640 1 -- 172.21.15.7:0/3168568399 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f7b280efca0 con 0x7f7b38102d30 2024-09-10T10:44:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.499+0000 7f7b35ffb640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] conn(0x7f7b240ac7c0 0x7f7b240aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.499+0000 7f7b35ffb640 1 -- 172.21.15.7:0/3168568399 --> [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b28003b30 con 0x7f7b240ac7c0 2024-09-10T10:44:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.499+0000 7f7b35ffb640 1 -- 172.21.15.7:0/3168568399 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f7b280f0050 con 0x7f7b38102d30 2024-09-10T10:44:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.499+0000 7f7b3cfe3640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] conn(0x7f7b240ac7c0 0x7f7b240aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.499+0000 7f7b2ffff640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f7b240a2c10 0x7f7b240a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7b380fee90 tx=0x7f7b20008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.499+0000 7f7b3cfe3640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] conn(0x7f7b240ac7c0 0x7f7b240aec60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.500+0000 7f7b35ffb640 1 -- 172.21.15.7:0/3168568399 <== osd.3 v2:172.21.15.7:6810/871781144 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b28003b30 con 0x7f7b240ac7c0 2024-09-10T10:44:47.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 --> [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b38062e00 con 0x7f7b240ac7c0 2024-09-10T10:44:47.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b35ffb640 1 -- 172.21.15.7:0/3168568399 <== osd.3 v2:172.21.15.7:6810/871781144 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7b38062e00 con 0x7f7b240ac7c0 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] conn(0x7f7b240ac7c0 msgr2=0x7f7b240aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] conn(0x7f7b240ac7c0 0x7f7b240aec60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f7b240a2c10 msgr2=0x7f7b240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f7b240a2c10 0x7f7b240a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7b380fee90 tx=0x7f7b20008040 comp rx=0 tx=0).stop 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38102d30 msgr2=0x7f7b38198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 --2- 172.21.15.7:0/3168568399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b38102d30 0x7f7b38198660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009a50 tx=0x7f7b28004270 comp rx=0 tx=0).stop 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3cfe3640 1 -- 172.21.15.7:0/3168568399 reap_dead start 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 shutdown_connections 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 >> 172.21.15.7:0/3168568399 conn(0x7f7b380fc7d0 msgr2=0x7f7b38105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 shutdown_connections 2024-09-10T10:44:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:47.514+0000 7f7b3ea46640 1 -- 172.21.15.7:0/3168568399 wait complete. 2024-09-10T10:44:48.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.158+0000 7f02696bd640 1 -- 172.21.15.7:0/1382246304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0264101420 msgr2=0x7f026410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.158+0000 7f02696bd640 1 --2- 172.21.15.7:0/1382246304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0264101420 0x7f026410d950 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f025c009cd0 tx=0x7f025c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.158+0000 7f02696bd640 1 -- 172.21.15.7:0/1382246304 shutdown_connections 2024-09-10T10:44:48.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.158+0000 7f02696bd640 1 --2- 172.21.15.7:0/1382246304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0264101420 0x7f026410d950 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.158+0000 7f02696bd640 1 --2- 172.21.15.7:0/1382246304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0264100a80 0x7f0264100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.158+0000 7f02696bd640 1 --2- 172.21.15.7:0/1382246304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0264106a80 0x7f0264106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.158+0000 7f02696bd640 1 -- 172.21.15.7:0/1382246304 >> 172.21.15.7:0/1382246304 conn(0x7f02640fc7a0 msgr2=0x7f02640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.158+0000 7f02696bd640 1 -- 172.21.15.7:0/1382246304 shutdown_connections 2024-09-10T10:44:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.158+0000 7f02696bd640 1 -- 172.21.15.7:0/1382246304 wait complete. 2024-09-10T10:44:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f02696bd640 1 Processor -- start 2024-09-10T10:44:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f02696bd640 1 -- start start 2024-09-10T10:44:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f02696bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0264100a80 0x7f0264198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f02696bd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0264101420 0x7f0264198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f02696bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0264106a80 0x7f026419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f02696bd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0264110e80 con 0x7f0264100a80 2024-09-10T10:44:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f02696bd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0264110d00 con 0x7f0264106a80 2024-09-10T10:44:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f02696bd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0264111000 con 0x7f0264101420 2024-09-10T10:44:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f0262ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0264100a80 0x7f0264198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f0262ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0264100a80 0x7f0264198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40996/0 (socket says 172.21.15.7:40996) 2024-09-10T10:44:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f0262ffd640 1 -- 172.21.15.7:0/1235136618 learned_addr learned my addr 172.21.15.7:0/1235136618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f02637fe640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0264106a80 0x7f026419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.159+0000 7f025bfff640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0264101420 0x7f0264198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f0262ffd640 1 -- 172.21.15.7:0/1235136618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0264101420 msgr2=0x7f0264198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f0262ffd640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0264101420 0x7f0264198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f0262ffd640 1 -- 172.21.15.7:0/1235136618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0264106a80 msgr2=0x7f026419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f0262ffd640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0264106a80 0x7f026419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f0262ffd640 1 -- 172.21.15.7:0/1235136618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026419d3c0 con 0x7f0264100a80 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f0262ffd640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0264100a80 0x7f0264198410 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f025400ea60 tx=0x7f025400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f0260ff9640 1 -- 172.21.15.7:0/1235136618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025400cde0 con 0x7f0264100a80 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026419d6b0 con 0x7f0264100a80 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f0260ff9640 1 -- 172.21.15.7:0/1235136618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02540035f0 con 0x7f0264100a80 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f0260ff9640 1 -- 172.21.15.7:0/1235136618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254010670 con 0x7f0264100a80 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.160+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02641a4f90 con 0x7f0264100a80 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.161+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0264062980 con 0x7f0264100a80 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.162+0000 7f0260ff9640 1 -- 172.21.15.7:0/1235136618 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f02540037a0 con 0x7f0264100a80 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.163+0000 7f0260ff9640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f02500a2c10 0x7f02500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.163+0000 7f025bfff640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f02500a2c10 0x7f02500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.163+0000 7f0260ff9640 1 -- 172.21.15.7:0/1235136618 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f02540cf4b0 con 0x7f0264100a80 2024-09-10T10:44:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.163+0000 7f0260ff9640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] conn(0x7f02500ac7c0 0x7f02500aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.163+0000 7f0260ff9640 1 -- 172.21.15.7:0/1235136618 --> [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f025401ee50 con 0x7f02500ac7c0 2024-09-10T10:44:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.163+0000 7f0260ff9640 1 -- 172.21.15.7:0/1235136618 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f02540d6050 con 0x7f0264100a80 2024-09-10T10:44:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.163+0000 7f025bfff640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f02500a2c10 0x7f02500a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0264105190 tx=0x7f024c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.163+0000 7f02637fe640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] conn(0x7f02500ac7c0 0x7f02500aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.164+0000 7f02637fe640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] conn(0x7f02500ac7c0 0x7f02500aec60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.165+0000 7f0260ff9640 1 -- 172.21.15.7:0/1235136618 <== osd.5 v2:172.21.15.133:6808/577882774 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f025401ee50 con 0x7f02500ac7c0 2024-09-10T10:44:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.178+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 --> [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0264062db0 con 0x7f02500ac7c0 2024-09-10T10:44:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f0260ff9640 1 -- 172.21.15.7:0/1235136618 <== osd.5 v2:172.21.15.133:6808/577882774 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0264062db0 con 0x7f02500ac7c0 2024-09-10T10:44:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 >> [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] conn(0x7f02500ac7c0 msgr2=0x7f02500aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02696bd640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] conn(0x7f02500ac7c0 0x7f02500aec60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f02500a2c10 msgr2=0x7f02500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02696bd640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f02500a2c10 0x7f02500a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0264105190 tx=0x7f024c005f90 comp rx=0 tx=0).stop 2024-09-10T10:44:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0264100a80 msgr2=0x7f0264198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02696bd640 1 --2- 172.21.15.7:0/1235136618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0264100a80 0x7f0264198410 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f025400ea60 tx=0x7f025400ef20 comp rx=0 tx=0).stop 2024-09-10T10:44:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02637fe640 1 -- 172.21.15.7:0/1235136618 reap_dead start 2024-09-10T10:44:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 shutdown_connections 2024-09-10T10:44:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 >> 172.21.15.7:0/1235136618 conn(0x7f02640fc7a0 msgr2=0x7f026406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 shutdown_connections 2024-09-10T10:44:48.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.179+0000 7f02696bd640 1 -- 172.21.15.7:0/1235136618 wait complete. 2024-09-10T10:44:48.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.496+0000 7f47044aa640 1 -- 172.21.15.7:0/3739143895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc101420 msgr2=0x7f46fc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.496+0000 7f47044aa640 1 --2- 172.21.15.7:0/3739143895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc101420 0x7f46fc10d950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009cd0 tx=0x7f46f402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.496+0000 7f47044aa640 1 -- 172.21.15.7:0/3739143895 shutdown_connections 2024-09-10T10:44:48.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.496+0000 7f47044aa640 1 --2- 172.21.15.7:0/3739143895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc101420 0x7f46fc10d950 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.496+0000 7f47044aa640 1 --2- 172.21.15.7:0/3739143895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc100a80 0x7f46fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.496+0000 7f47044aa640 1 --2- 172.21.15.7:0/3739143895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46fc106a80 0x7f46fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.496+0000 7f47044aa640 1 -- 172.21.15.7:0/3739143895 >> 172.21.15.7:0/3739143895 conn(0x7f46fc0fc7a0 msgr2=0x7f46fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.496+0000 7f47044aa640 1 -- 172.21.15.7:0/3739143895 shutdown_connections 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.496+0000 7f47044aa640 1 -- 172.21.15.7:0/3739143895 wait complete. 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f47044aa640 1 Processor -- start 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f47044aa640 1 -- start start 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f47044aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc100a80 0x7f46fc1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f47044aa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc101420 0x7f46fc1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f47044aa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46fc106a80 0x7f46fc19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f47044aa640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc110e80 con 0x7f46fc101420 2024-09-10T10:44:48.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f47044aa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc110d00 con 0x7f46fc106a80 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f47044aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc111000 con 0x7f46fc100a80 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f4701a45640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc101420 0x7f46fc1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f4701a45640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc101420 0x7f46fc1977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41014/0 (socket says 172.21.15.7:41014) 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f4701a45640 1 -- 172.21.15.7:0/3109132467 learned_addr learned my addr 172.21.15.7:0/3109132467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f4702a47640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46fc106a80 0x7f46fc19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f4701a45640 1 -- 172.21.15.7:0/3109132467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc100a80 msgr2=0x7f46fc1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f4701a45640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc100a80 0x7f46fc1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f4701a45640 1 -- 172.21.15.7:0/3109132467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46fc106a80 msgr2=0x7f46fc19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.497+0000 7f4701a45640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46fc106a80 0x7f46fc19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.498+0000 7f4701a45640 1 -- 172.21.15.7:0/3109132467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc19c250 con 0x7f46fc101420 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.498+0000 7f4702a47640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46fc106a80 0x7f46fc19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:48.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.498+0000 7f4701a45640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc101420 0x7f46fc1977e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c920 tx=0x7f46ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.498+0000 7f46f37fe640 1 -- 172.21.15.7:0/3109132467 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec007c70 con 0x7f46fc101420 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.498+0000 7f46f37fe640 1 -- 172.21.15.7:0/3109132467 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ec007e10 con 0x7f46fc101420 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.498+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc19c540 con 0x7f46fc101420 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.498+0000 7f46f37fe640 1 -- 172.21.15.7:0/3109132467 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec018890 con 0x7f46fc101420 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.498+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc06a790 con 0x7f46fc101420 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.499+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46fc0fd880 con 0x7f46fc101420 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.500+0000 7f46f37fe640 1 -- 172.21.15.7:0/3109132467 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f46ec018a30 con 0x7f46fc101420 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.501+0000 7f46f37fe640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f46e80a2c90 0x7f46e80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.501+0000 7f46f37fe640 1 -- 172.21.15.7:0/3109132467 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f46ec0cf750 con 0x7f46fc101420 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.501+0000 7f4702246640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f46e80a2c90 0x7f46e80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.501+0000 7f46f37fe640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] conn(0x7f46e80ac840 0x7f46e80aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.501+0000 7f46f37fe640 1 -- 172.21.15.7:0/3109132467 --> [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46e80af3b0 con 0x7f46e80ac840 2024-09-10T10:44:48.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.501+0000 7f46f37fe640 1 -- 172.21.15.7:0/3109132467 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f46ec0cfba0 con 0x7f46fc101420 2024-09-10T10:44:48.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.501+0000 7f4702a47640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] conn(0x7f46e80ac840 0x7f46e80aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.501+0000 7f4702246640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f46e80a2c90 0x7f46e80a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46f800a860 tx=0x7f46f8005b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.502+0000 7f4702a47640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] conn(0x7f46e80ac840 0x7f46e80aece0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.503+0000 7f46f37fe640 1 -- 172.21.15.7:0/3109132467 <== osd.2 v2:172.21.15.133:6800/1657349858 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46e80af3b0 con 0x7f46e80ac840 2024-09-10T10:44:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.516+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 --> [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46fc1021c0 con 0x7f46e80ac840 2024-09-10T10:44:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f46f37fe640 1 -- 172.21.15.7:0/3109132467 <== osd.2 v2:172.21.15.133:6800/1657349858 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f46fc1021c0 con 0x7f46e80ac840 2024-09-10T10:44:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 >> [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] conn(0x7f46e80ac840 msgr2=0x7f46e80aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f47044aa640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] conn(0x7f46e80ac840 0x7f46e80aece0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f46e80a2c90 msgr2=0x7f46e80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f47044aa640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f46e80a2c90 0x7f46e80a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46f800a860 tx=0x7f46f8005b60 comp rx=0 tx=0).stop 2024-09-10T10:44:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc101420 msgr2=0x7f46fc1977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f47044aa640 1 --2- 172.21.15.7:0/3109132467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46fc101420 0x7f46fc1977e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c920 tx=0x7f46ec00cde0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f4702a47640 1 -- 172.21.15.7:0/3109132467 reap_dead start 2024-09-10T10:44:48.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 shutdown_connections 2024-09-10T10:44:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 >> 172.21.15.7:0/3109132467 conn(0x7f46fc0fc7a0 msgr2=0x7f46fc0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 shutdown_connections 2024-09-10T10:44:48.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.517+0000 7f47044aa640 1 -- 172.21.15.7:0/3109132467 wait complete. 2024-09-10T10:44:48.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:48 smithi007 ceph-mon[30873]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-10T10:44:48.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:48 smithi133 ceph-mon[35282]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-10T10:44:48.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:48 smithi103 ceph-mon[35160]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-10T10:44:48.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.942+0000 7f642f4cb640 1 -- 172.21.15.7:0/3546662110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6428100a80 msgr2=0x7f6428100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.942+0000 7f642f4cb640 1 --2- 172.21.15.7:0/3546662110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6428100a80 0x7f6428100ee0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6418009930 tx=0x7f641802a420 comp rx=0 tx=0).stop 2024-09-10T10:44:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.942+0000 7f642f4cb640 1 -- 172.21.15.7:0/3546662110 shutdown_connections 2024-09-10T10:44:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.942+0000 7f642f4cb640 1 --2- 172.21.15.7:0/3546662110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6428101420 0x7f642810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.942+0000 7f642f4cb640 1 --2- 172.21.15.7:0/3546662110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6428100a80 0x7f6428100ee0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.942+0000 7f642f4cb640 1 --2- 172.21.15.7:0/3546662110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6428106a80 0x7f6428106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.942+0000 7f642f4cb640 1 -- 172.21.15.7:0/3546662110 >> 172.21.15.7:0/3546662110 conn(0x7f64280fc7a0 msgr2=0x7f64280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.942+0000 7f642f4cb640 1 -- 172.21.15.7:0/3546662110 shutdown_connections 2024-09-10T10:44:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.942+0000 7f642f4cb640 1 -- 172.21.15.7:0/3546662110 wait complete. 2024-09-10T10:44:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642f4cb640 1 Processor -- start 2024-09-10T10:44:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642f4cb640 1 -- start start 2024-09-10T10:44:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642f4cb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6428100a80 0x7f64280fcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642f4cb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6428101420 0x7f64280fd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642f4cb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6428106a80 0x7f64280fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642f4cb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6428110e80 con 0x7f6428100a80 2024-09-10T10:44:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642f4cb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6428110d00 con 0x7f6428101420 2024-09-10T10:44:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642f4cb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6428111000 con 0x7f6428106a80 2024-09-10T10:44:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642d267640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6428100a80 0x7f64280fcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642d267640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6428100a80 0x7f64280fcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41032/0 (socket says 172.21.15.7:41032) 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642d267640 1 -- 172.21.15.7:0/2167641112 learned_addr learned my addr 172.21.15.7:0/2167641112 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642ca66640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6428101420 0x7f64280fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642d267640 1 -- 172.21.15.7:0/2167641112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6428106a80 msgr2=0x7f64280fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642d267640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6428106a80 0x7f64280fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642d267640 1 -- 172.21.15.7:0/2167641112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6428101420 msgr2=0x7f64280fd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642d267640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6428101420 0x7f64280fd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.943+0000 7f642d267640 1 -- 172.21.15.7:0/2167641112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64281a6940 con 0x7f6428100a80 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.944+0000 7f642d267640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6428100a80 0x7f64280fcdd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f642400d950 tx=0x7f642400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.944+0000 7f641e7fc640 1 -- 172.21.15.7:0/2167641112 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424017070 con 0x7f6428100a80 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.944+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64281a6c30 con 0x7f6428100a80 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.944+0000 7f641e7fc640 1 -- 172.21.15.7:0/2167641112 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6424009c30 con 0x7f6428100a80 2024-09-10T10:44:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.944+0000 7f641e7fc640 1 -- 172.21.15.7:0/2167641112 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642400f6b0 con 0x7f6428100a80 2024-09-10T10:44:48.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.944+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64281a7170 con 0x7f6428100a80 2024-09-10T10:44:48.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.945+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6428062900 con 0x7f6428100a80 2024-09-10T10:44:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.946+0000 7f641e7fc640 1 -- 172.21.15.7:0/2167641112 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f64240109d0 con 0x7f6428100a80 2024-09-10T10:44:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.947+0000 7f641e7fc640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f64140a2c10 0x7f64140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.947+0000 7f642ca66640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f64140a2c10 0x7f64140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.947+0000 7f641e7fc640 1 -- 172.21.15.7:0/2167641112 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f64240cf220 con 0x7f6428100a80 2024-09-10T10:44:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.947+0000 7f641e7fc640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] conn(0x7f64140ac7c0 0x7f64140aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.947+0000 7f641e7fc640 1 -- 172.21.15.7:0/2167641112 --> [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64140af330 con 0x7f64140ac7c0 2024-09-10T10:44:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.947+0000 7f641e7fc640 1 -- 172.21.15.7:0/2167641112 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f64240cf670 con 0x7f6428100a80 2024-09-10T10:44:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.947+0000 7f642ca66640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f64140a2c10 0x7f64140a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f64180096c0 tx=0x7f6418000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.947+0000 7f642da68640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] conn(0x7f64140ac7c0 0x7f64140aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.948+0000 7f642da68640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] conn(0x7f64140ac7c0 0x7f64140aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.949+0000 7f641e7fc640 1 -- 172.21.15.7:0/2167641112 <== osd.11 v2:172.21.15.133:6824/2538184038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f64140af330 con 0x7f64140ac7c0 2024-09-10T10:44:48.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.962+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 --> [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6428062d80 con 0x7f64140ac7c0 2024-09-10T10:44:48.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.963+0000 7f641e7fc640 1 -- 172.21.15.7:0/2167641112 <== osd.11 v2:172.21.15.133:6824/2538184038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6428062d80 con 0x7f64140ac7c0 2024-09-10T10:44:48.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.963+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 >> [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] conn(0x7f64140ac7c0 msgr2=0x7f64140aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.963+0000 7f642f4cb640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] conn(0x7f64140ac7c0 0x7f64140aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.963+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f64140a2c10 msgr2=0x7f64140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.963+0000 7f642f4cb640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f64140a2c10 0x7f64140a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f64180096c0 tx=0x7f6418000bc0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.963+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6428100a80 msgr2=0x7f64280fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.963+0000 7f642f4cb640 1 --2- 172.21.15.7:0/2167641112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6428100a80 0x7f64280fcdd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f642400d950 tx=0x7f642400de10 comp rx=0 tx=0).stop 2024-09-10T10:44:48.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.963+0000 7f642da68640 1 -- 172.21.15.7:0/2167641112 reap_dead start 2024-09-10T10:44:48.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.964+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 shutdown_connections 2024-09-10T10:44:48.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.964+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 >> 172.21.15.7:0/2167641112 conn(0x7f64280fc7a0 msgr2=0x7f642806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:48.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.964+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 shutdown_connections 2024-09-10T10:44:48.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:48.964+0000 7f642f4cb640 1 -- 172.21.15.7:0/2167641112 wait complete. 2024-09-10T10:44:49.603 INFO:teuthology.orchestra.run.smithi007.stdout:197568495656 2024-09-10T10:44:49.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T10:44:50.310 INFO:teuthology.orchestra.run.smithi007.stdout:103079215164 2024-09-10T10:44:50.310 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T10:44:50.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:50 smithi007 ceph-mon[30873]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T10:44:50.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.503+0000 7f09d4a41640 1 -- 172.21.15.7:0/1445202095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d0100e20 msgr2=0x7f09d0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.504+0000 7f09d4a41640 1 --2- 172.21.15.7:0/1445202095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d0100e20 0x7f09d0101200 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009900 tx=0x7f09c002a480 comp rx=0 tx=0).stop 2024-09-10T10:44:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.504+0000 7f09d4a41640 1 -- 172.21.15.7:0/1445202095 shutdown_connections 2024-09-10T10:44:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.504+0000 7f09d4a41640 1 --2- 172.21.15.7:0/1445202095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09d0102ef0 0x7f09d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.504+0000 7f09d4a41640 1 --2- 172.21.15.7:0/1445202095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0102570 0x7f09d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.504+0000 7f09d4a41640 1 --2- 172.21.15.7:0/1445202095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d0100e20 0x7f09d0101200 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.504+0000 7f09d4a41640 1 -- 172.21.15.7:0/1445202095 >> 172.21.15.7:0/1445202095 conn(0x7f09d00fc7d0 msgr2=0x7f09d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.504+0000 7f09d4a41640 1 -- 172.21.15.7:0/1445202095 shutdown_connections 2024-09-10T10:44:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.504+0000 7f09d4a41640 1 -- 172.21.15.7:0/1445202095 wait complete. 2024-09-10T10:44:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09d4a41640 1 Processor -- start 2024-09-10T10:44:50.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09d4a41640 1 -- start start 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09d4a41640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09d0100e20 0x7f09d01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09d4a41640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0102570 0x7f09d0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09d4a41640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d0102ef0 0x7f09d019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09d4a41640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0110fc0 con 0x7f09d0102ef0 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09d4a41640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0110e40 con 0x7f09d0100e20 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09d4a41640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0111140 con 0x7f09d0102570 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09ced9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d0102ef0 0x7f09d019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09ced9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d0102ef0 0x7f09d019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43594/0 (socket says 172.21.15.7:43594) 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09ced9d640 1 -- 172.21.15.7:0/2569867951 learned_addr learned my addr 172.21.15.7:0/2569867951 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09ce59c640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09d0100e20 0x7f09d01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.505+0000 7f09cdd9b640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0102570 0x7f09d0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09ced9d640 1 -- 172.21.15.7:0/2569867951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0102570 msgr2=0x7f09d0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09ced9d640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d0102570 0x7f09d0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09ced9d640 1 -- 172.21.15.7:0/2569867951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09d0100e20 msgr2=0x7f09d01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09ced9d640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09d0100e20 0x7f09d01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09ced9d640 1 -- 172.21.15.7:0/2569867951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d019d560 con 0x7f09d0102ef0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09ced9d640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d0102ef0 0x7f09d019ce80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f09c800b970 tx=0x7f09c800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09bf7fe640 1 -- 172.21.15.7:0/2569867951 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8019070 con 0x7f09d0102ef0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d019d850 con 0x7f09d0102ef0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09bf7fe640 1 -- 172.21.15.7:0/2569867951 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c800cb40 con 0x7f09d0102ef0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09bf7fe640 1 -- 172.21.15.7:0/2569867951 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8007650 con 0x7f09d0102ef0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.506+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d01a5130 con 0x7f09d0102ef0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.507+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09d0106bf0 con 0x7f09d0102ef0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.508+0000 7f09bf7fe640 1 -- 172.21.15.7:0/2569867951 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f09c801c070 con 0x7f09d0102ef0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.509+0000 7f09bf7fe640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09b80a2c10 0x7f09b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.509+0000 7f09ce59c640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09b80a2c10 0x7f09b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.509+0000 7f09bf7fe640 1 -- 172.21.15.7:0/2569867951 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f09c80ceee0 con 0x7f09d0102ef0 2024-09-10T10:44:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.509+0000 7f09bf7fe640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] conn(0x7f09b80ac7c0 0x7f09b80aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.509+0000 7f09bf7fe640 1 -- 172.21.15.7:0/2569867951 --> [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09c80047d0 con 0x7f09b80ac7c0 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.509+0000 7f09bf7fe640 1 -- 172.21.15.7:0/2569867951 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f09c80cf290 con 0x7f09d0102ef0 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.509+0000 7f09ce59c640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09b80a2c10 0x7f09b80a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009230 tx=0x7f09c00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.510+0000 7f09cdd9b640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] conn(0x7f09b80ac7c0 0x7f09b80aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.510+0000 7f09cdd9b640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] conn(0x7f09b80ac7c0 0x7f09b80aec60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.511+0000 7f09bf7fe640 1 -- 172.21.15.7:0/2569867951 <== osd.4 v2:172.21.15.103:6808/1937243043 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09c80047d0 con 0x7f09b80ac7c0 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.525+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 --> [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09d01021b0 con 0x7f09b80ac7c0 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.525+0000 7f09bf7fe640 1 -- 172.21.15.7:0/2569867951 <== osd.4 v2:172.21.15.103:6808/1937243043 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f09d01021b0 con 0x7f09b80ac7c0 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 >> [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] conn(0x7f09b80ac7c0 msgr2=0x7f09b80aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09d4a41640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] conn(0x7f09b80ac7c0 0x7f09b80aec60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09b80a2c10 msgr2=0x7f09b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09d4a41640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09b80a2c10 0x7f09b80a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009230 tx=0x7f09c00058d0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d0102ef0 msgr2=0x7f09d019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09d4a41640 1 --2- 172.21.15.7:0/2569867951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d0102ef0 0x7f09d019ce80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f09c800b970 tx=0x7f09c800be30 comp rx=0 tx=0).stop 2024-09-10T10:44:50.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09ced9d640 1 -- 172.21.15.7:0/2569867951 reap_dead start 2024-09-10T10:44:50.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 shutdown_connections 2024-09-10T10:44:50.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 >> 172.21.15.7:0/2569867951 conn(0x7f09d00fc7d0 msgr2=0x7f09d006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:50.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 shutdown_connections 2024-09-10T10:44:50.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:50.526+0000 7f09d4a41640 1 -- 172.21.15.7:0/2569867951 wait complete. 2024-09-10T10:44:50.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:50 smithi133 ceph-mon[35282]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T10:44:50.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:50 smithi103 ceph-mon[35160]: pgmap v295: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-10T10:44:51.474 INFO:teuthology.orchestra.run.smithi007.stdout:150323855410 2024-09-10T10:44:51.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T10:44:51.941 INFO:teuthology.orchestra.run.smithi007.stdout:313532612628 2024-09-10T10:44:51.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T10:44:52.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.097+0000 7f3150cd8640 1 -- 172.21.15.7:0/3483714501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f314c1036e0 msgr2=0x7f314c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.097+0000 7f3150cd8640 1 --2- 172.21.15.7:0/3483714501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f314c1036e0 0x7f314c107b90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3134009cd0 tx=0x7f313402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.097+0000 7f3150cd8640 1 -- 172.21.15.7:0/3483714501 shutdown_connections 2024-09-10T10:44:52.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.097+0000 7f3150cd8640 1 --2- 172.21.15.7:0/3483714501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c1082a0 0x7f314c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.097+0000 7f3150cd8640 1 --2- 172.21.15.7:0/3483714501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f314c1036e0 0x7f314c107b90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.097+0000 7f3150cd8640 1 --2- 172.21.15.7:0/3483714501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f314c102d30 0x7f314c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.097+0000 7f3150cd8640 1 -- 172.21.15.7:0/3483714501 >> 172.21.15.7:0/3483714501 conn(0x7f314c0fc7d0 msgr2=0x7f314c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:52.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.097+0000 7f3150cd8640 1 -- 172.21.15.7:0/3483714501 shutdown_connections 2024-09-10T10:44:52.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.097+0000 7f3150cd8640 1 -- 172.21.15.7:0/3483714501 wait complete. 2024-09-10T10:44:52.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3150cd8640 1 Processor -- start 2024-09-10T10:44:52.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3150cd8640 1 -- start start 2024-09-10T10:44:52.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3150cd8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c102d30 0x7f314c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3150cd8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f314c1036e0 0x7f314c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3150cd8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f314c1082a0 0x7f314c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3150cd8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f314c1120a0 con 0x7f314c1036e0 2024-09-10T10:44:52.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3150cd8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f314c111f20 con 0x7f314c1082a0 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3150cd8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f314c112220 con 0x7f314c102d30 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3149d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f314c1036e0 0x7f314c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3149d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f314c1036e0 0x7f314c198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43616/0 (socket says 172.21.15.7:43616) 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f3149d9b640 1 -- 172.21.15.7:0/3119013329 learned_addr learned my addr 172.21.15.7:0/3119013329 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f314ad9d640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f314c1082a0 0x7f314c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.098+0000 7f314a59c640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c102d30 0x7f314c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f3149d9b640 1 -- 172.21.15.7:0/3119013329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c102d30 msgr2=0x7f314c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f3149d9b640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c102d30 0x7f314c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f3149d9b640 1 -- 172.21.15.7:0/3119013329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f314c1082a0 msgr2=0x7f314c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f3149d9b640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f314c1082a0 0x7f314c19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f3149d9b640 1 -- 172.21.15.7:0/3119013329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c19d590 con 0x7f314c1036e0 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f3149d9b640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f314c1036e0 0x7f314c198b20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3134000f80 tx=0x7f3134031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f31437fe640 1 -- 172.21.15.7:0/3119013329 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134004040 con 0x7f314c1036e0 2024-09-10T10:44:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314c19d820 con 0x7f314c1036e0 2024-09-10T10:44:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f31437fe640 1 -- 172.21.15.7:0/3119013329 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31340046c0 con 0x7f314c1036e0 2024-09-10T10:44:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f31437fe640 1 -- 172.21.15.7:0/3119013329 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134030e40 con 0x7f314c1036e0 2024-09-10T10:44:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.099+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314c1a5160 con 0x7f314c1036e0 2024-09-10T10:44:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.100+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f314c062980 con 0x7f314c1036e0 2024-09-10T10:44:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.102+0000 7f31437fe640 1 -- 172.21.15.7:0/3119013329 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f31340305a0 con 0x7f314c1036e0 2024-09-10T10:44:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.103+0000 7f31437fe640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f31380a2c10 0x7f31380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.103+0000 7f314a59c640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f31380a2c10 0x7f31380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.103+0000 7f31437fe640 1 -- 172.21.15.7:0/3119013329 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f313402e030 con 0x7f314c1036e0 2024-09-10T10:44:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.103+0000 7f31437fe640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] conn(0x7f31380ac7c0 0x7f31380aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.103+0000 7f31437fe640 1 -- 172.21.15.7:0/3119013329 --> [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f313400b150 con 0x7f31380ac7c0 2024-09-10T10:44:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.103+0000 7f31437fe640 1 -- 172.21.15.7:0/3119013329 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f31340f0a40 con 0x7f314c1036e0 2024-09-10T10:44:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.104+0000 7f314ad9d640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] conn(0x7f31380ac7c0 0x7f31380aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.104+0000 7f314a59c640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f31380a2c10 0x7f31380a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f313c009bd0 tx=0x7f313c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.104+0000 7f314ad9d640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] conn(0x7f31380ac7c0 0x7f31380aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.105+0000 7f31437fe640 1 -- 172.21.15.7:0/3119013329 <== osd.9 v2:172.21.15.7:6826/945633240 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f313400b150 con 0x7f31380ac7c0 2024-09-10T10:44:52.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.119+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 --> [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f314c062d80 con 0x7f31380ac7c0 2024-09-10T10:44:52.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.119+0000 7f31437fe640 1 -- 172.21.15.7:0/3119013329 <== osd.9 v2:172.21.15.7:6826/945633240 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f314c062d80 con 0x7f31380ac7c0 2024-09-10T10:44:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.119+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] conn(0x7f31380ac7c0 msgr2=0x7f31380aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.119+0000 7f3150cd8640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] conn(0x7f31380ac7c0 0x7f31380aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.120+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f31380a2c10 msgr2=0x7f31380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.120+0000 7f3150cd8640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f31380a2c10 0x7f31380a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f313c009bd0 tx=0x7f313c005f90 comp rx=0 tx=0).stop 2024-09-10T10:44:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.120+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f314c1036e0 msgr2=0x7f314c198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.120+0000 7f3150cd8640 1 --2- 172.21.15.7:0/3119013329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f314c1036e0 0x7f314c198b20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3134000f80 tx=0x7f3134031e60 comp rx=0 tx=0).stop 2024-09-10T10:44:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.120+0000 7f314ad9d640 1 -- 172.21.15.7:0/3119013329 reap_dead start 2024-09-10T10:44:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.120+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 shutdown_connections 2024-09-10T10:44:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.120+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 >> 172.21.15.7:0/3119013329 conn(0x7f314c0fc7d0 msgr2=0x7f314c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.120+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 shutdown_connections 2024-09-10T10:44:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:52.120+0000 7f3150cd8640 1 -- 172.21.15.7:0/3119013329 wait complete. 2024-09-10T10:44:52.523 INFO:teuthology.orchestra.run.smithi007.stdout:77309411394 2024-09-10T10:44:52.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T10:44:52.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:52 smithi133 ceph-mon[35282]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T10:44:52.753 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:52 smithi007 ceph-mon[30873]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T10:44:52.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:52 smithi103 ceph-mon[35160]: pgmap v296: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T10:44:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.357+0000 7f23214eb640 1 -- 172.21.15.7:0/2526308371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f231c101de0 msgr2=0x7f231c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.357+0000 7f23214eb640 1 --2- 172.21.15.7:0/2526308371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f231c101de0 0x7f231c1021c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f230c009900 tx=0x7f230c02a410 comp rx=0 tx=0).stop 2024-09-10T10:44:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.357+0000 7f23214eb640 1 -- 172.21.15.7:0/2526308371 shutdown_connections 2024-09-10T10:44:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.357+0000 7f23214eb640 1 --2- 172.21.15.7:0/2526308371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f231c106f10 0x7f231c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.357+0000 7f23214eb640 1 --2- 172.21.15.7:0/2526308371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c102790 0x7f231c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.357+0000 7f23214eb640 1 --2- 172.21.15.7:0/2526308371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f231c101de0 0x7f231c1021c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.357+0000 7f23214eb640 1 -- 172.21.15.7:0/2526308371 >> 172.21.15.7:0/2526308371 conn(0x7f231c0fd8c0 msgr2=0x7f231c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.357+0000 7f23214eb640 1 -- 172.21.15.7:0/2526308371 shutdown_connections 2024-09-10T10:44:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.357+0000 7f23214eb640 1 -- 172.21.15.7:0/2526308371 wait complete. 2024-09-10T10:44:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f23214eb640 1 Processor -- start 2024-09-10T10:44:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f23214eb640 1 -- start start 2024-09-10T10:44:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f23214eb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f231c101de0 0x7f231c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f23214eb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f231c102790 0x7f231c1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f23214eb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c106f10 0x7f231c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f23214eb640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f231c111f90 con 0x7f231c101de0 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f23214eb640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f231c111e10 con 0x7f231c102790 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f23214eb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f231c112110 con 0x7f231c106f10 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f231c101de0 0x7f231c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f231c101de0 0x7f231c1a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43634/0 (socket says 172.21.15.7:43634) 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231affd640 1 -- 172.21.15.7:0/451328389 learned_addr learned my addr 172.21.15.7:0/451328389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231a7fc640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f231c102790 0x7f231c1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231b7fe640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c106f10 0x7f231c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231affd640 1 -- 172.21.15.7:0/451328389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c106f10 msgr2=0x7f231c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231affd640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f231c106f10 0x7f231c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231affd640 1 -- 172.21.15.7:0/451328389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f231c102790 msgr2=0x7f231c1a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231affd640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f231c102790 0x7f231c1a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231affd640 1 -- 172.21.15.7:0/451328389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c19acd0 con 0x7f231c101de0 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.358+0000 7f231a7fc640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f231c102790 0x7f231c1a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.359+0000 7f231affd640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f231c101de0 0x7f231c1a03e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f230c005ab0 tx=0x7f230c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.359+0000 7f2313fff640 1 -- 172.21.15.7:0/451328389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c03f030 con 0x7f231c101de0 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.359+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231c19af60 con 0x7f231c101de0 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.359+0000 7f2313fff640 1 -- 172.21.15.7:0/451328389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f230c003ce0 con 0x7f231c101de0 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.359+0000 7f2313fff640 1 -- 172.21.15.7:0/451328389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c035070 con 0x7f231c101de0 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.359+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231c1a72b0 con 0x7f231c101de0 2024-09-10T10:44:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.360+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f231c10e0d0 con 0x7f231c101de0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.361+0000 7f2313fff640 1 -- 172.21.15.7:0/451328389 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f230c042030 con 0x7f231c101de0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.362+0000 7f2313fff640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f23080a2c10 0x7f23080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.362+0000 7f231a7fc640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f23080a2c10 0x7f23080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.362+0000 7f2313fff640 1 -- 172.21.15.7:0/451328389 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f230c0efe00 con 0x7f231c101de0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.362+0000 7f2313fff640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] conn(0x7f23080ac7c0 0x7f23080aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.362+0000 7f2313fff640 1 -- 172.21.15.7:0/451328389 --> [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f230c001d30 con 0x7f23080ac7c0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.362+0000 7f2313fff640 1 -- 172.21.15.7:0/451328389 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f230c0f0250 con 0x7f231c101de0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.362+0000 7f231b7fe640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] conn(0x7f23080ac7c0 0x7f23080aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.362+0000 7f231a7fc640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f23080a2c10 0x7f23080a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f231c0ff770 tx=0x7f2304005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.362+0000 7f231b7fe640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] conn(0x7f23080ac7c0 0x7f23080aec60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.363+0000 7f2313fff640 1 -- 172.21.15.7:0/451328389 <== osd.6 v2:172.21.15.7:6818/2646192734 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f230c001d30 con 0x7f23080ac7c0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 --> [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f231c1032c0 con 0x7f23080ac7c0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f2313fff640 1 -- 172.21.15.7:0/451328389 <== osd.6 v2:172.21.15.7:6818/2646192734 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f231c1032c0 con 0x7f23080ac7c0 2024-09-10T10:44:53.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] conn(0x7f23080ac7c0 msgr2=0x7f23080aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f23214eb640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] conn(0x7f23080ac7c0 0x7f23080aec60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f23080a2c10 msgr2=0x7f23080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f23214eb640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f23080a2c10 0x7f23080a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f231c0ff770 tx=0x7f2304005ec0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f231c101de0 msgr2=0x7f231c1a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f23214eb640 1 --2- 172.21.15.7:0/451328389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f231c101de0 0x7f231c1a03e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f230c005ab0 tx=0x7f230c0042f0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f231b7fe640 1 -- 172.21.15.7:0/451328389 reap_dead start 2024-09-10T10:44:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 shutdown_connections 2024-09-10T10:44:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.377+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 >> 172.21.15.7:0/451328389 conn(0x7f231c0fd8c0 msgr2=0x7f231c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.378+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 shutdown_connections 2024-09-10T10:44:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:53.378+0000 7f23214eb640 1 -- 172.21.15.7:0/451328389 wait complete. 2024-09-10T10:44:53.975 INFO:teuthology.orchestra.run.smithi007.stdout:261993005087 2024-09-10T10:44:53.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T10:44:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.122+0000 7fe218c9c640 1 -- 172.21.15.7:0/2391665576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214102df0 msgr2=0x7fe2141031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.122+0000 7fe218c9c640 1 --2- 172.21.15.7:0/2391665576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214102df0 0x7fe2141031d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe204009900 tx=0x7fe20402a410 comp rx=0 tx=0).stop 2024-09-10T10:44:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.122+0000 7fe218c9c640 1 -- 172.21.15.7:0/2391665576 shutdown_connections 2024-09-10T10:44:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.122+0000 7fe218c9c640 1 --2- 172.21.15.7:0/2391665576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe21410c2f0 0x7fe21410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.122+0000 7fe218c9c640 1 --2- 172.21.15.7:0/2391665576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2141037a0 0x7fe21410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.122+0000 7fe218c9c640 1 --2- 172.21.15.7:0/2391665576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214102df0 0x7fe2141031d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.122+0000 7fe218c9c640 1 -- 172.21.15.7:0/2391665576 >> 172.21.15.7:0/2391665576 conn(0x7fe2140fc7d0 msgr2=0x7fe2140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:54.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.123+0000 7fe218c9c640 1 -- 172.21.15.7:0/2391665576 shutdown_connections 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.123+0000 7fe218c9c640 1 -- 172.21.15.7:0/2391665576 wait complete. 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.123+0000 7fe218c9c640 1 Processor -- start 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.123+0000 7fe218c9c640 1 -- start start 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe218c9c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe214102df0 0x7fe2141a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe218c9c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2141037a0 0x7fe2141a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe218c9c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21410c2f0 0x7fe21419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe218c9c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe214112150 con 0x7fe21410c2f0 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe218c9c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe214111fd0 con 0x7fe2141037a0 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe218c9c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141122d0 con 0x7fe214102df0 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe212d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21410c2f0 0x7fe21419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe212d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21410c2f0 0x7fe21419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43652/0 (socket says 172.21.15.7:43652) 2024-09-10T10:44:54.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe212d9d640 1 -- 172.21.15.7:0/302801075 learned_addr learned my addr 172.21.15.7:0/302801075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe21259c640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe214102df0 0x7fe2141a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe20bfff640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2141037a0 0x7fe2141a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe212d9d640 1 -- 172.21.15.7:0/302801075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe214102df0 msgr2=0x7fe2141a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe212d9d640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe214102df0 0x7fe2141a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe212d9d640 1 -- 172.21.15.7:0/302801075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2141037a0 msgr2=0x7fe2141a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe212d9d640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2141037a0 0x7fe2141a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe212d9d640 1 -- 172.21.15.7:0/302801075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21419af50 con 0x7fe21410c2f0 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.124+0000 7fe20bfff640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2141037a0 0x7fe2141a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.125+0000 7fe212d9d640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21410c2f0 0x7fe21419a6e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00eaa0 tx=0x7fe20c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.125+0000 7fe20b7fe640 1 -- 172.21.15.7:0/302801075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00cdb0 con 0x7fe21410c2f0 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.125+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21419b240 con 0x7fe21410c2f0 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.125+0000 7fe20b7fe640 1 -- 172.21.15.7:0/302801075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe20c005020 con 0x7fe21410c2f0 2024-09-10T10:44:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.125+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2141a7590 con 0x7fe21410c2f0 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.125+0000 7fe20b7fe640 1 -- 172.21.15.7:0/302801075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0116b0 con 0x7fe21410c2f0 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.126+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe214102440 con 0x7fe21410c2f0 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.127+0000 7fe20b7fe640 1 -- 172.21.15.7:0/302801075 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe20c005560 con 0x7fe21410c2f0 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.128+0000 7fe20b7fe640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe2000a2c10 0x7fe2000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.128+0000 7fe21259c640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe2000a2c10 0x7fe2000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.128+0000 7fe20b7fe640 1 -- 172.21.15.7:0/302801075 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fe20c0cf7f0 con 0x7fe21410c2f0 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.128+0000 7fe20b7fe640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] conn(0x7fe2000ac7c0 0x7fe2000aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.129+0000 7fe20b7fe640 1 -- 172.21.15.7:0/302801075 --> [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe20c014ec0 con 0x7fe2000ac7c0 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.129+0000 7fe20b7fe640 1 -- 172.21.15.7:0/302801075 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fe20c0cfc40 con 0x7fe21410c2f0 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.129+0000 7fe20bfff640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] conn(0x7fe2000ac7c0 0x7fe2000aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.129+0000 7fe21259c640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe2000a2c10 0x7fe2000a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe204009230 tx=0x7fe2040058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:54.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.129+0000 7fe20bfff640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] conn(0x7fe2000ac7c0 0x7fe2000aec60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:54.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.130+0000 7fe20b7fe640 1 -- 172.21.15.7:0/302801075 <== osd.1 v2:172.21.15.103:6800/2409158189 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe20c014ec0 con 0x7fe2000ac7c0 2024-09-10T10:44:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.144+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 --> [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe21406a790 con 0x7fe2000ac7c0 2024-09-10T10:44:54.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.145+0000 7fe20b7fe640 1 -- 172.21.15.7:0/302801075 <== osd.1 v2:172.21.15.103:6800/2409158189 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe21406a790 con 0x7fe2000ac7c0 2024-09-10T10:44:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.145+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 >> [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] conn(0x7fe2000ac7c0 msgr2=0x7fe2000aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.145+0000 7fe218c9c640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] conn(0x7fe2000ac7c0 0x7fe2000aec60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.145+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe2000a2c10 msgr2=0x7fe2000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.145+0000 7fe218c9c640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe2000a2c10 0x7fe2000a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe204009230 tx=0x7fe2040058d0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.145+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21410c2f0 msgr2=0x7fe21419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.145+0000 7fe218c9c640 1 --2- 172.21.15.7:0/302801075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21410c2f0 0x7fe21419a6e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00eaa0 tx=0x7fe20c00ef60 comp rx=0 tx=0).stop 2024-09-10T10:44:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.145+0000 7fe212d9d640 1 -- 172.21.15.7:0/302801075 reap_dead start 2024-09-10T10:44:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.146+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 shutdown_connections 2024-09-10T10:44:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.146+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 >> 172.21.15.7:0/302801075 conn(0x7fe2140fc7d0 msgr2=0x7fe21410d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:54.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.146+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 shutdown_connections 2024-09-10T10:44:54.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:54.146+0000 7fe218c9c640 1 -- 172.21.15.7:0/302801075 wait complete. 2024-09-10T10:44:54.592 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:54 smithi007 ceph-mon[30873]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:44:54.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:54 smithi133 ceph-mon[35282]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:44:54.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:54 smithi103 ceph-mon[35160]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:44:55.004 INFO:teuthology.orchestra.run.smithi007.stdout:231928234019 2024-09-10T10:44:55.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T10:44:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.518+0000 7f5751f5d640 1 -- 172.21.15.7:0/4072529857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574c102dd0 msgr2=0x7f574c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.518+0000 7f5751f5d640 1 --2- 172.21.15.7:0/4072529857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574c102dd0 0x7f574c1031b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f573c009cd0 tx=0x7f573c02a8f0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.518+0000 7f5751f5d640 1 -- 172.21.15.7:0/4072529857 shutdown_connections 2024-09-10T10:44:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.518+0000 7f5751f5d640 1 --2- 172.21.15.7:0/4072529857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f574c108300 0x7f574c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.518+0000 7f5751f5d640 1 --2- 172.21.15.7:0/4072529857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f574c103780 0x7f574c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.518+0000 7f5751f5d640 1 --2- 172.21.15.7:0/4072529857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574c102dd0 0x7f574c1031b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.518+0000 7f5751f5d640 1 -- 172.21.15.7:0/4072529857 >> 172.21.15.7:0/4072529857 conn(0x7f574c0fc7d0 msgr2=0x7f574c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.518+0000 7f5751f5d640 1 -- 172.21.15.7:0/4072529857 shutdown_connections 2024-09-10T10:44:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.518+0000 7f5751f5d640 1 -- 172.21.15.7:0/4072529857 wait complete. 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f5751f5d640 1 Processor -- start 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f5751f5d640 1 -- start start 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f5751f5d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f574c102dd0 0x7f574c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f5751f5d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f574c103780 0x7f574c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f5751f5d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574c108300 0x7f574c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f5751f5d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f574c1120f0 con 0x7f574c108300 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f5751f5d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f574c111f70 con 0x7f574c102dd0 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f5751f5d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f574c112270 con 0x7f574c103780 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f574bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574c108300 0x7f574c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f574bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574c108300 0x7f574c19cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43664/0 (socket says 172.21.15.7:43664) 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f574bfff640 1 -- 172.21.15.7:0/1575662965 learned_addr learned my addr 172.21.15.7:0/1575662965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f574b7fe640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f574c102dd0 0x7f574c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.519+0000 7f574affd640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f574c103780 0x7f574c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f574bfff640 1 -- 172.21.15.7:0/1575662965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f574c103780 msgr2=0x7f574c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f574bfff640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f574c103780 0x7f574c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f574bfff640 1 -- 172.21.15.7:0/1575662965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f574c102dd0 msgr2=0x7f574c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f574bfff640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f574c102dd0 0x7f574c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f574bfff640 1 -- 172.21.15.7:0/1575662965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c19d5c0 con 0x7f574c108300 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f574affd640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f574c103780 0x7f574c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f574bfff640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574c108300 0x7f574c19cee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f574400c970 tx=0x7f574400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f5748ff9640 1 -- 172.21.15.7:0/1575662965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744007bb0 con 0x7f574c108300 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f5748ff9640 1 -- 172.21.15.7:0/1575662965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5744007d50 con 0x7f574c108300 2024-09-10T10:44:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c19d8b0 con 0x7f574c108300 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f5748ff9640 1 -- 172.21.15.7:0/1575662965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574400f450 con 0x7f574c108300 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.520+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c1a5190 con 0x7f574c108300 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.521+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f574c062980 con 0x7f574c108300 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.522+0000 7f5748ff9640 1 -- 172.21.15.7:0/1575662965 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5744004f60 con 0x7f574c108300 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.523+0000 7f5748ff9640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f57340a2c10 0x7f57340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.523+0000 7f574b7fe640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f57340a2c10 0x7f57340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.523+0000 7f5748ff9640 1 -- 172.21.15.7:0/1575662965 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f57440cef30 con 0x7f574c108300 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.523+0000 7f5748ff9640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] conn(0x7f57340ac7c0 0x7f57340aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.523+0000 7f5748ff9640 1 -- 172.21.15.7:0/1575662965 --> [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5744004490 con 0x7f57340ac7c0 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.523+0000 7f5748ff9640 1 -- 172.21.15.7:0/1575662965 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f57440d15f0 con 0x7f574c108300 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.523+0000 7f574affd640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] conn(0x7f57340ac7c0 0x7f57340aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.523+0000 7f574b7fe640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f57340a2c10 0x7f57340a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f573c009810 tx=0x7f573c009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.523+0000 7f574affd640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] conn(0x7f57340ac7c0 0x7f57340aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.524+0000 7f5748ff9640 1 -- 172.21.15.7:0/1575662965 <== osd.0 v2:172.21.15.7:6802/704452390 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5744004490 con 0x7f57340ac7c0 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.538+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 --> [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f574c062e00 con 0x7f57340ac7c0 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.538+0000 7f5748ff9640 1 -- 172.21.15.7:0/1575662965 <== osd.0 v2:172.21.15.7:6802/704452390 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f574c062e00 con 0x7f57340ac7c0 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.538+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] conn(0x7f57340ac7c0 msgr2=0x7f57340aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.538+0000 7f5751f5d640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] conn(0x7f57340ac7c0 0x7f57340aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.538+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f57340a2c10 msgr2=0x7f57340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.538+0000 7f5751f5d640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f57340a2c10 0x7f57340a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f573c009810 tx=0x7f573c009780 comp rx=0 tx=0).stop 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.538+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574c108300 msgr2=0x7f574c19cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.538+0000 7f5751f5d640 1 --2- 172.21.15.7:0/1575662965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574c108300 0x7f574c19cee0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f574400c970 tx=0x7f574400ce30 comp rx=0 tx=0).stop 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.539+0000 7f574bfff640 1 -- 172.21.15.7:0/1575662965 reap_dead start 2024-09-10T10:44:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.539+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 shutdown_connections 2024-09-10T10:44:55.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.539+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 >> 172.21.15.7:0/1575662965 conn(0x7f574c0fc7d0 msgr2=0x7f574c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:55.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.539+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 shutdown_connections 2024-09-10T10:44:55.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:55.539+0000 7f5751f5d640 1 -- 172.21.15.7:0/1575662965 wait complete. 2024-09-10T10:44:55.956 INFO:teuthology.orchestra.run.smithi007.stdout:55834574920 2024-09-10T10:44:55.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T10:44:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.361+0000 7f37bdff9640 1 -- 172.21.15.7:0/267590086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8106f10 msgr2=0x7f37b810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.361+0000 7f37bdff9640 1 --2- 172.21.15.7:0/267590086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8106f10 0x7f37b810ea90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009ca0 tx=0x7f37b002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.361+0000 7f37bdff9640 1 -- 172.21.15.7:0/267590086 shutdown_connections 2024-09-10T10:44:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.361+0000 7f37bdff9640 1 --2- 172.21.15.7:0/267590086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8106f10 0x7f37b810ea90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.361+0000 7f37bdff9640 1 --2- 172.21.15.7:0/267590086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b8102790 0x7f37b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.361+0000 7f37bdff9640 1 --2- 172.21.15.7:0/267590086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b8101de0 0x7f37b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.361+0000 7f37bdff9640 1 -- 172.21.15.7:0/267590086 >> 172.21.15.7:0/267590086 conn(0x7f37b80fd8c0 msgr2=0x7f37b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.361+0000 7f37bdff9640 1 -- 172.21.15.7:0/267590086 shutdown_connections 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.361+0000 7f37bdff9640 1 -- 172.21.15.7:0/267590086 wait complete. 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.362+0000 7f37bdff9640 1 Processor -- start 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.362+0000 7f37bdff9640 1 -- start start 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.362+0000 7f37bdff9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b8101de0 0x7f37b819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.362+0000 7f37bdff9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b8102790 0x7f37b819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.362+0000 7f37bdff9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8106f10 0x7f37b819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.362+0000 7f37bdff9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8111f70 con 0x7f37b8106f10 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.362+0000 7f37bdff9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8111df0 con 0x7f37b8101de0 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.362+0000 7f37bdff9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f37b81120f0 con 0x7f37b8102790 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8106f10 0x7f37b819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8106f10 0x7f37b819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43682/0 (socket says 172.21.15.7:43682) 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b7fff640 1 -- 172.21.15.7:0/898428685 learned_addr learned my addr 172.21.15.7:0/898428685 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b77fe640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b8101de0 0x7f37b819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b6ffd640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b8102790 0x7f37b819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b7fff640 1 -- 172.21.15.7:0/898428685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b8102790 msgr2=0x7f37b819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b7fff640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b8102790 0x7f37b819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b7fff640 1 -- 172.21.15.7:0/898428685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b8101de0 msgr2=0x7f37b819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b7fff640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b8101de0 0x7f37b819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b7fff640 1 -- 172.21.15.7:0/898428685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b819f600 con 0x7f37b8106f10 2024-09-10T10:44:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b7fff640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8106f10 0x7f37b819ef20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f37b0031a20 tx=0x7f37b0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b4ff9640 1 -- 172.21.15.7:0/898428685 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0003c30 con 0x7f37b8106f10 2024-09-10T10:44:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b819f890 con 0x7f37b8106f10 2024-09-10T10:44:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b4ff9640 1 -- 172.21.15.7:0/898428685 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b0003dd0 con 0x7f37b8106f10 2024-09-10T10:44:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37b4ff9640 1 -- 172.21.15.7:0/898428685 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b002fc40 con 0x7f37b8106f10 2024-09-10T10:44:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.363+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81a71d0 con 0x7f37b8106f10 2024-09-10T10:44:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.364+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37b80feed0 con 0x7f37b8106f10 2024-09-10T10:44:56.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.366+0000 7f37b4ff9640 1 -- 172.21.15.7:0/898428685 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f37b0004050 con 0x7f37b8106f10 2024-09-10T10:44:56.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.366+0000 7f37b4ff9640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37a00a2c10 0x7f37a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.366+0000 7f37b77fe640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37a00a2c10 0x7f37a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.366+0000 7f37b4ff9640 1 -- 172.21.15.7:0/898428685 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f37b00f04e0 con 0x7f37b8106f10 2024-09-10T10:44:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.367+0000 7f37b4ff9640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] conn(0x7f37a00ac690 0x7f37a00aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.367+0000 7f37b4ff9640 1 -- 172.21.15.7:0/898428685 --> [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37a00af200 con 0x7f37a00ac690 2024-09-10T10:44:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.367+0000 7f37b4ff9640 1 -- 172.21.15.7:0/898428685 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f37b00f0930 con 0x7f37b8106f10 2024-09-10T10:44:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.367+0000 7f37b77fe640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37a00a2c10 0x7f37a00a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f37a80058c0 tx=0x7f37a800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.367+0000 7f37b6ffd640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] conn(0x7f37a00ac690 0x7f37a00aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.367+0000 7f37b6ffd640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] conn(0x7f37a00ac690 0x7f37a00aeb30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.368+0000 7f37b4ff9640 1 -- 172.21.15.7:0/898428685 <== osd.10 v2:172.21.15.103:6824/907835659 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37a00af200 con 0x7f37a00ac690 2024-09-10T10:44:56.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.382+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 --> [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37b81032c0 con 0x7f37a00ac690 2024-09-10T10:44:56.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.382+0000 7f37b4ff9640 1 -- 172.21.15.7:0/898428685 <== osd.10 v2:172.21.15.103:6824/907835659 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f37b81032c0 con 0x7f37a00ac690 2024-09-10T10:44:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.382+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 >> [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] conn(0x7f37a00ac690 msgr2=0x7f37a00aeb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.383+0000 7f37bdff9640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] conn(0x7f37a00ac690 0x7f37a00aeb30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.383+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37a00a2c10 msgr2=0x7f37a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.383+0000 7f37bdff9640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f37a00a2c10 0x7f37a00a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f37a80058c0 tx=0x7f37a800a2d0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.383+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8106f10 msgr2=0x7f37b819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.383+0000 7f37bdff9640 1 --2- 172.21.15.7:0/898428685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b8106f10 0x7f37b819ef20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f37b0031a20 tx=0x7f37b0031a50 comp rx=0 tx=0).stop 2024-09-10T10:44:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.383+0000 7f37b7fff640 1 -- 172.21.15.7:0/898428685 reap_dead start 2024-09-10T10:44:56.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.383+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 shutdown_connections 2024-09-10T10:44:56.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.383+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 >> 172.21.15.7:0/898428685 conn(0x7f37b80fd8c0 msgr2=0x7f37b80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:56.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.383+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 shutdown_connections 2024-09-10T10:44:56.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:56.383+0000 7f37bdff9640 1 -- 172.21.15.7:0/898428685 wait complete. 2024-09-10T10:44:56.591 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:56 smithi007 ceph-mon[30873]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:44:56.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:56 smithi103 ceph-mon[35160]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:44:56.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:56 smithi133 ceph-mon[35282]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:44:57.175 INFO:teuthology.orchestra.run.smithi007.stdout:34359738445 2024-09-10T10:44:57.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T10:44:57.719 INFO:teuthology.orchestra.run.smithi007.stdout:283467841562 2024-09-10T10:44:57.719 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T10:44:58.158 INFO:teuthology.orchestra.run.smithi007.stdout:180388626478 2024-09-10T10:44:58.158 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T10:44:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:58.669 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:58.670 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:58.670 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:44:58.707 INFO:teuthology.orchestra.run.smithi007.stdout:133143986232 2024-09-10T10:44:58.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T10:44:58.732 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:44:58 smithi007 ceph-mon[30873]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-10T10:44:58.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:44:58 smithi103 ceph-mon[35160]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-10T10:44:58.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:44:58 smithi133 ceph-mon[35282]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-10T10:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.635+0000 7fefee3ed640 1 -- 172.21.15.7:0/876701563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe8106f10 msgr2=0x7fefe810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.635+0000 7fefee3ed640 1 --2- 172.21.15.7:0/876701563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe8106f10 0x7fefe810ea90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009ca0 tx=0x7fefe002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.635+0000 7fefee3ed640 1 -- 172.21.15.7:0/876701563 shutdown_connections 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.635+0000 7fefee3ed640 1 --2- 172.21.15.7:0/876701563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe8106f10 0x7fefe810ea90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.635+0000 7fefee3ed640 1 --2- 172.21.15.7:0/876701563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8102790 0x7fefe81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.635+0000 7fefee3ed640 1 --2- 172.21.15.7:0/876701563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefe8101de0 0x7fefe81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.635+0000 7fefee3ed640 1 -- 172.21.15.7:0/876701563 >> 172.21.15.7:0/876701563 conn(0x7fefe80fd8c0 msgr2=0x7fefe80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.635+0000 7fefee3ed640 1 -- 172.21.15.7:0/876701563 shutdown_connections 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.635+0000 7fefee3ed640 1 -- 172.21.15.7:0/876701563 wait complete. 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefee3ed640 1 Processor -- start 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefee3ed640 1 -- start start 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefee3ed640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8101de0 0x7fefe8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefee3ed640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe8102790 0x7fefe81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefee3ed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefe8106f10 0x7fefe819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefee3ed640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8111f70 con 0x7fefe8102790 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefee3ed640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8111df0 con 0x7fefe8106f10 2024-09-10T10:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefee3ed640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fefe81120f0 con 0x7fefe8101de0 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefe77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe8102790 0x7fefe81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefe77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe8102790 0x7fefe81988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43706/0 (socket says 172.21.15.7:43706) 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.636+0000 7fefe77fe640 1 -- 172.21.15.7:0/336371176 learned_addr learned my addr 172.21.15.7:0/336371176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefec98a640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefe8106f10 0x7fefe819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefe77fe640 1 -- 172.21.15.7:0/336371176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8101de0 msgr2=0x7fefe8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefe7fff640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8101de0 0x7fefe8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefe77fe640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8101de0 0x7fefe8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefe77fe640 1 -- 172.21.15.7:0/336371176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefe8106f10 msgr2=0x7fefe819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefe77fe640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefe8106f10 0x7fefe819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefe77fe640 1 -- 172.21.15.7:0/336371176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe819d340 con 0x7fefe8102790 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefec98a640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefe8106f10 0x7fefe819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefe77fe640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe8102790 0x7fefe81988d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00d950 tx=0x7fefdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefe57fa640 1 -- 172.21.15.7:0/336371176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc017070 con 0x7fefe8102790 2024-09-10T10:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefe57fa640 1 -- 172.21.15.7:0/336371176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefdc009c30 con 0x7fefe8102790 2024-09-10T10:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefee3ed640 1 -- 172.21.15.7:0/336371176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe819d630 con 0x7fefe8102790 2024-09-10T10:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefe57fa640 1 -- 172.21.15.7:0/336371176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc00f6b0 con 0x7fefe8102790 2024-09-10T10:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.637+0000 7fefee3ed640 1 -- 172.21.15.7:0/336371176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe81a4f10 con 0x7fefe8102790 2024-09-10T10:44:59.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.638+0000 7fefee3ed640 1 -- 172.21.15.7:0/336371176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe81021c0 con 0x7fefe8102790 2024-09-10T10:44:59.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.639+0000 7fefe57fa640 1 -- 172.21.15.7:0/336371176 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fefdc0109d0 con 0x7fefe8102790 2024-09-10T10:44:59.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.640+0000 7fefe57fa640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fefd00a2c90 0x7fefd00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.640+0000 7fefe7fff640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fefd00a2c90 0x7fefd00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.641+0000 7fefe57fa640 1 -- 172.21.15.7:0/336371176 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fefdc0cee20 con 0x7fefe8102790 2024-09-10T10:44:59.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.641+0000 7fefe7fff640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fefd00a2c90 0x7fefd00a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fefd8005fd0 tx=0x7fefd8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.645+0000 7fefe57fa640 1 -- 172.21.15.7:0/336371176 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefdc009de0 con 0x7fefe8102790 2024-09-10T10:44:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.819+0000 7f79a0799640 1 -- 172.21.15.7:0/3104845433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7998106b80 msgr2=0x7f7998106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.819+0000 7f79a0799640 1 --2- 172.21.15.7:0/3104845433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7998106b80 0x7f7998106f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f798c009ca0 tx=0x7f798c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.819+0000 7f79a0799640 1 -- 172.21.15.7:0/3104845433 shutdown_connections 2024-09-10T10:44:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.819+0000 7f79a0799640 1 --2- 172.21.15.7:0/3104845433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7998100aa0 0x7f799810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.819+0000 7f79a0799640 1 --2- 172.21.15.7:0/3104845433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7998100100 0x7f7998100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.819+0000 7f79a0799640 1 --2- 172.21.15.7:0/3104845433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7998106b80 0x7f7998106f60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.820+0000 7f79a0799640 1 -- 172.21.15.7:0/3104845433 >> 172.21.15.7:0/3104845433 conn(0x7f7998077230 msgr2=0x7f79980fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.820+0000 7f79a0799640 1 -- 172.21.15.7:0/3104845433 shutdown_connections 2024-09-10T10:44:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.820+0000 7f79a0799640 1 -- 172.21.15.7:0/3104845433 wait complete. 2024-09-10T10:44:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.820+0000 7f79a0799640 1 Processor -- start 2024-09-10T10:44:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.820+0000 7f79a0799640 1 -- start start 2024-09-10T10:44:59.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.820+0000 7f79a0799640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7998100100 0x7f7998198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.820+0000 7f79a0799640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7998100aa0 0x7f7998198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.820+0000 7f79a0799640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7998106b80 0x7f799819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f79a0799640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7998112050 con 0x7f7998100aa0 2024-09-10T10:44:59.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f79a0799640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7998111ed0 con 0x7f7998100100 2024-09-10T10:44:59.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f79a0799640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79981121d0 con 0x7f7998106b80 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799dd34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7998100aa0 0x7f7998198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799dd34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7998100aa0 0x7f7998198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43726/0 (socket says 172.21.15.7:43726) 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799dd34640 1 -- 172.21.15.7:0/3603600616 learned_addr learned my addr 172.21.15.7:0/3603600616 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799ed36640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7998106b80 0x7f799819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799e535640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7998100100 0x7f7998198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799dd34640 1 -- 172.21.15.7:0/3603600616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7998106b80 msgr2=0x7f799819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799dd34640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7998106b80 0x7f799819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799dd34640 1 -- 172.21.15.7:0/3603600616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7998100100 msgr2=0x7f7998198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799dd34640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7998100100 0x7f7998198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799dd34640 1 -- 172.21.15.7:0/3603600616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799819d3f0 con 0x7f7998100aa0 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f799dd34640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7998100aa0 0x7f7998198980 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f799400c920 tx=0x7f799400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f798b7fe640 1 -- 172.21.15.7:0/3603600616 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994007c70 con 0x7f7998100aa0 2024-09-10T10:44:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.821+0000 7f79a0799640 1 -- 172.21.15.7:0/3603600616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799819d6e0 con 0x7f7998100aa0 2024-09-10T10:44:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.822+0000 7f798b7fe640 1 -- 172.21.15.7:0/3603600616 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7994007e10 con 0x7f7998100aa0 2024-09-10T10:44:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.822+0000 7f798b7fe640 1 -- 172.21.15.7:0/3603600616 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994018890 con 0x7f7998100aa0 2024-09-10T10:44:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.822+0000 7f79a0799640 1 -- 172.21.15.7:0/3603600616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79981a4fc0 con 0x7f7998100aa0 2024-09-10T10:44:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.822+0000 7fefee3ed640 1 -- 172.21.15.7:0/336371176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fefe810a150 con 0x7fefe8102790 2024-09-10T10:44:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.823+0000 7fefe57fa640 1 -- 172.21.15.7:0/336371176 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fefdc09bbd0 con 0x7fefe8102790 2024-09-10T10:44:59.827 INFO:teuthology.orchestra.run.smithi007.stdout:34359738446 2024-09-10T10:44:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.823+0000 7f79a0799640 1 -- 172.21.15.7:0/3603600616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7998100560 con 0x7f7998100aa0 2024-09-10T10:44:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.824+0000 7f798b7fe640 1 -- 172.21.15.7:0/3603600616 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7994018a70 con 0x7f7998100aa0 2024-09-10T10:44:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 -- 172.21.15.7:0/336371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fefd00a2c90 msgr2=0x7fefd00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fefd00a2c90 0x7fefd00a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fefd8005fd0 tx=0x7fefd8005850 comp rx=0 tx=0).stop 2024-09-10T10:44:59.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 -- 172.21.15.7:0/336371176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe8102790 msgr2=0x7fefe81988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe8102790 0x7fefe81988d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00d950 tx=0x7fefdc00de10 comp rx=0 tx=0).stop 2024-09-10T10:44:59.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7f798b7fe640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f79840a2c90 0x7f79840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 -- 172.21.15.7:0/336371176 shutdown_connections 2024-09-10T10:44:59.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fefd00a2c90 0x7fefd00a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7f798b7fe640 1 -- 172.21.15.7:0/3603600616 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f79940cf3f0 con 0x7f7998100aa0 2024-09-10T10:44:59.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefe8106f10 0x7fefe819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7f799e535640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f79840a2c90 0x7f79840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe8102790 0x7fefe81988d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 --2- 172.21.15.7:0/336371176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefe8101de0 0x7fefe8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 -- 172.21.15.7:0/336371176 >> 172.21.15.7:0/336371176 conn(0x7fefe80fd8c0 msgr2=0x7fefe80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 -- 172.21.15.7:0/336371176 shutdown_connections 2024-09-10T10:44:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.825+0000 7fefee3ed640 1 -- 172.21.15.7:0/336371176 wait complete. 2024-09-10T10:44:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.826+0000 7f799e535640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f79840a2c90 0x7f79840a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f798c02f540 tx=0x7f798c02f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:44:59.830+0000 7f798b7fe640 1 -- 172.21.15.7:0/3603600616 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7994014030 con 0x7f7998100aa0 2024-09-10T10:45:00.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.012+0000 7f79a0799640 1 -- 172.21.15.7:0/3603600616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f79981a5220 con 0x7f7998100aa0 2024-09-10T10:45:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.012+0000 7f798b7fe640 1 -- 172.21.15.7:0/3603600616 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f799409c1a0 con 0x7f7998100aa0 2024-09-10T10:45:00.014 INFO:teuthology.orchestra.run.smithi007.stdout:150323855413 2024-09-10T10:45:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 -- 172.21.15.7:0/3603600616 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f79840a2c90 msgr2=0x7f79840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f79840a2c90 0x7f79840a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f798c02f540 tx=0x7f798c02f310 comp rx=0 tx=0).stop 2024-09-10T10:45:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 -- 172.21.15.7:0/3603600616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7998100aa0 msgr2=0x7f7998198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7998100aa0 0x7f7998198980 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f799400c920 tx=0x7f799400cde0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 -- 172.21.15.7:0/3603600616 shutdown_connections 2024-09-10T10:45:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f79840a2c90 0x7f79840a5150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7998106b80 0x7f799819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7998100aa0 0x7f7998198980 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 --2- 172.21.15.7:0/3603600616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7998100100 0x7f7998198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 -- 172.21.15.7:0/3603600616 >> 172.21.15.7:0/3603600616 conn(0x7f7998077230 msgr2=0x7f7998104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 -- 172.21.15.7:0/3603600616 shutdown_connections 2024-09-10T10:45:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:00.014+0000 7f79a0799640 1 -- 172.21.15.7:0/3603600616 wait complete. 2024-09-10T10:45:00.540 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738446 for osd.0 2024-09-10T10:45:00.541 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:00.734 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:00 smithi007 ceph-mon[30873]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:00.734 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:00 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/336371176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T10:45:00.734 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:00 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3603600616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T10:45:00.781 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:00 smithi103 ceph-mon[35160]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:00 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/336371176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T10:45:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:00 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3603600616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T10:45:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:00 smithi133 ceph-mon[35282]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:00 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/336371176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T10:45:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:00 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3603600616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T10:45:01.110 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855410 got 150323855413 for osd.5 2024-09-10T10:45:01.110 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.408+0000 7f9e179ca640 1 -- 172.21.15.7:0/4053599066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e10105e80 msgr2=0x7f9e1010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.408+0000 7f9e179ca640 1 --2- 172.21.15.7:0/4053599066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e10105e80 0x7f9e1010da40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009ca0 tx=0x7f9e0802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.408+0000 7f9e179ca640 1 -- 172.21.15.7:0/4053599066 shutdown_connections 2024-09-10T10:45:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.408+0000 7f9e179ca640 1 --2- 172.21.15.7:0/4053599066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e10105e80 0x7f9e1010da40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.408+0000 7f9e179ca640 1 --2- 172.21.15.7:0/4053599066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e10101e30 0x7f9e10105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.408+0000 7f9e179ca640 1 --2- 172.21.15.7:0/4053599066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10101480 0x7f9e10101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.408+0000 7f9e179ca640 1 -- 172.21.15.7:0/4053599066 >> 172.21.15.7:0/4053599066 conn(0x7f9e10077210 msgr2=0x7f9e100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.408+0000 7f9e179ca640 1 -- 172.21.15.7:0/4053599066 shutdown_connections 2024-09-10T10:45:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.408+0000 7f9e179ca640 1 -- 172.21.15.7:0/4053599066 wait complete. 2024-09-10T10:45:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e179ca640 1 Processor -- start 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e179ca640 1 -- start start 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e179ca640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10101480 0x7f9e100fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e179ca640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e10101e30 0x7f9e100fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e179ca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e10105e80 0x7f9e100fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e179ca640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10110f40 con 0x7f9e10101e30 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e179ca640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10110dc0 con 0x7f9e10101480 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e179ca640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101110c0 con 0x7f9e10105e80 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e14f65640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e10101e30 0x7f9e100fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e15766640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10101480 0x7f9e100fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e14f65640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e10101e30 0x7f9e100fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32978/0 (socket says 172.21.15.7:32978) 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.409+0000 7f9e14f65640 1 -- 172.21.15.7:0/4030662004 learned_addr learned my addr 172.21.15.7:0/4030662004 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e15f67640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e10105e80 0x7f9e100fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e14f65640 1 -- 172.21.15.7:0/4030662004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e10105e80 msgr2=0x7f9e100fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e14f65640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e10105e80 0x7f9e100fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e14f65640 1 -- 172.21.15.7:0/4030662004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10101480 msgr2=0x7f9e100fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e14f65640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10101480 0x7f9e100fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e14f65640 1 -- 172.21.15.7:0/4030662004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e101a69c0 con 0x7f9e10101e30 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e15f67640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e10105e80 0x7f9e100fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e15766640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10101480 0x7f9e100fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e14f65640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e10101e30 0x7f9e100fcbd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00b4a0 tx=0x7f9dfc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e0e7fc640 1 -- 172.21.15.7:0/4030662004 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc014070 con 0x7f9e10101e30 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e179ca640 1 -- 172.21.15.7:0/4030662004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e101a6cb0 con 0x7f9e10101e30 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e0e7fc640 1 -- 172.21.15.7:0/4030662004 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dfc010af0 con 0x7f9e10101e30 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e0e7fc640 1 -- 172.21.15.7:0/4030662004 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc017d10 con 0x7f9e10101e30 2024-09-10T10:45:01.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.410+0000 7f9e179ca640 1 -- 172.21.15.7:0/4030662004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e101a71e0 con 0x7f9e10101e30 2024-09-10T10:45:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.411+0000 7f9e179ca640 1 -- 172.21.15.7:0/4030662004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e10101860 con 0x7f9e10101e30 2024-09-10T10:45:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.412+0000 7f9e0e7fc640 1 -- 172.21.15.7:0/4030662004 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9dfc017070 con 0x7f9e10101e30 2024-09-10T10:45:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.413+0000 7f9e0e7fc640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e000a2c10 0x7f9e000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.413+0000 7f9e15766640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e000a2c10 0x7f9e000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.413+0000 7f9e0e7fc640 1 -- 172.21.15.7:0/4030662004 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f9dfc0ce480 con 0x7f9e10101e30 2024-09-10T10:45:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.413+0000 7f9e15766640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e000a2c10 0x7f9e000a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400bfd0 tx=0x7f9e0400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.418+0000 7f9e0e7fc640 1 -- 172.21.15.7:0/4030662004 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9dfc09b340 con 0x7f9e10101e30 2024-09-10T10:45:01.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.600+0000 7f9e179ca640 1 -- 172.21.15.7:0/4030662004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9e1010a110 con 0x7f9e10101e30 2024-09-10T10:45:01.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.600+0000 7f9e0e7fc640 1 -- 172.21.15.7:0/4030662004 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9dfc09ac40 con 0x7f9e10101e30 2024-09-10T10:45:01.602 INFO:teuthology.orchestra.run.smithi007.stdout:55834574921 2024-09-10T10:45:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.602+0000 7f9e179ca640 1 -- 172.21.15.7:0/4030662004 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e000a2c10 msgr2=0x7f9e000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.602+0000 7f9e179ca640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e000a2c10 0x7f9e000a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9e0400bfd0 tx=0x7f9e0400b390 comp rx=0 tx=0).stop 2024-09-10T10:45:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.603+0000 7f9e179ca640 1 -- 172.21.15.7:0/4030662004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e10101e30 msgr2=0x7f9e100fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.603+0000 7f9e179ca640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e10101e30 0x7f9e100fcbd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00b4a0 tx=0x7f9dfc00b960 comp rx=0 tx=0).stop 2024-09-10T10:45:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.603+0000 7f9e179ca640 1 -- 172.21.15.7:0/4030662004 shutdown_connections 2024-09-10T10:45:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.603+0000 7f9e179ca640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e000a2c10 0x7f9e000a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.603+0000 7f9e179ca640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e10105e80 0x7f9e100fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.603+0000 7f9e179ca640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e10101e30 0x7f9e100fcbd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.603+0000 7f9e179ca640 1 --2- 172.21.15.7:0/4030662004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e10101480 0x7f9e100fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.603+0000 7f9e179ca640 1 -- 172.21.15.7:0/4030662004 >> 172.21.15.7:0/4030662004 conn(0x7f9e10077210 msgr2=0x7f9e101003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.603+0000 7f9e179ca640 1 -- 172.21.15.7:0/4030662004 shutdown_connections 2024-09-10T10:45:01.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.603+0000 7f9e179ca640 1 -- 172.21.15.7:0/4030662004 wait complete. 2024-09-10T10:45:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.784+0000 7f9e85495640 1 -- 172.21.15.7:0/1559543488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e8010c1b0 msgr2=0x7f9e8010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.784+0000 7f9e85495640 1 --2- 172.21.15.7:0/1559543488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e8010c1b0 0x7f9e8010e620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009ca0 tx=0x7f9e7802a950 comp rx=0 tx=0).stop 2024-09-10T10:45:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.784+0000 7f9e85495640 1 -- 172.21.15.7:0/1559543488 shutdown_connections 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.784+0000 7f9e85495640 1 --2- 172.21.15.7:0/1559543488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e8010c1b0 0x7f9e8010e620 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.784+0000 7f9e85495640 1 --2- 172.21.15.7:0/1559543488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e80103730 0x7f9e8010bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.784+0000 7f9e85495640 1 --2- 172.21.15.7:0/1559543488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e80102e10 0x7f9e801031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.784+0000 7f9e85495640 1 -- 172.21.15.7:0/1559543488 >> 172.21.15.7:0/1559543488 conn(0x7f9e800fc7d0 msgr2=0x7f9e800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.784+0000 7f9e85495640 1 -- 172.21.15.7:0/1559543488 shutdown_connections 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.784+0000 7f9e85495640 1 -- 172.21.15.7:0/1559543488 wait complete. 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e85495640 1 Processor -- start 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e85495640 1 -- start start 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e85495640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e80102e10 0x7f9e80100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e85495640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e80103730 0x7f9e80100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e85495640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e8010c1b0 0x7f9e801014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e85495640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9e80112040 con 0x7f9e80102e10 2024-09-10T10:45:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e85495640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e80111ec0 con 0x7f9e80103730 2024-09-10T10:45:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e85495640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801121c0 con 0x7f9e8010c1b0 2024-09-10T10:45:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e7effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e80102e10 0x7f9e80100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e7effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e80102e10 0x7f9e80100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32998/0 (socket says 172.21.15.7:32998) 2024-09-10T10:45:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e7effd640 1 -- 172.21.15.7:0/1418120905 learned_addr learned my addr 172.21.15.7:0/1418120905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e7f7fe640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e8010c1b0 0x7f9e801014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e767fc640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e80103730 0x7f9e80100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e7effd640 1 -- 172.21.15.7:0/1418120905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e8010c1b0 msgr2=0x7f9e801014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e7effd640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e8010c1b0 0x7f9e801014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e7effd640 1 -- 172.21.15.7:0/1418120905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e80103730 msgr2=0x7f9e80100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e7effd640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e80103730 0x7f9e80100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.785+0000 7f9e7effd640 1 -- 172.21.15.7:0/1418120905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e801a6a60 con 0x7f9e80102e10 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.786+0000 7f9e7effd640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e80102e10 0x7f9e80100a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000b4a0 tx=0x7f9e7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.786+0000 7f9e7cff9640 1 -- 172.21.15.7:0/1418120905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70014070 con 0x7f9e80102e10 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.786+0000 7f9e85495640 1 -- 172.21.15.7:0/1418120905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e801a6d50 con 0x7f9e80102e10 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.786+0000 7f9e7cff9640 1 -- 172.21.15.7:0/1418120905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e70010af0 con 0x7f9e80102e10 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.786+0000 7f9e7cff9640 1 -- 172.21.15.7:0/1418120905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70017d10 con 0x7f9e80102e10 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.786+0000 7f9e85495640 1 -- 172.21.15.7:0/1418120905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e801a71e0 con 0x7f9e80102e10 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.787+0000 7f9e85495640 1 -- 172.21.15.7:0/1418120905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e80103270 con 0x7f9e80102e10 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.788+0000 7f9e7cff9640 1 -- 172.21.15.7:0/1418120905 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9e70017070 con 0x7f9e80102e10 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.789+0000 7f9e7cff9640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e6c0a2c10 0x7f9e6c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.789+0000 7f9e767fc640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e6c0a2c10 0x7f9e6c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.790+0000 7f9e7cff9640 1 -- 172.21.15.7:0/1418120905 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f9e700ce8d0 con 0x7f9e80102e10 2024-09-10T10:45:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.790+0000 7f9e767fc640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e6c0a2c10 0x7f9e6c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e80102500 tx=0x7f9e68008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.793+0000 7f9e7cff9640 1 -- 172.21.15.7:0/1418120905 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e7009b7f0 con 0x7f9e80102e10 2024-09-10T10:45:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.968+0000 7f9e85495640 1 -- 172.21.15.7:0/1418120905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f9e80101fa0 con 0x7f9e80102e10 2024-09-10T10:45:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.968+0000 7f9e7cff9640 1 -- 172.21.15.7:0/1418120905 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9e7009b0f0 con 0x7f9e80102e10 2024-09-10T10:45:01.970 INFO:teuthology.orchestra.run.smithi007.stdout:231928234022 2024-09-10T10:45:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.971+0000 7f9e85495640 1 -- 172.21.15.7:0/1418120905 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e6c0a2c10 msgr2=0x7f9e6c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.971+0000 7f9e85495640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e6c0a2c10 0x7f9e6c0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e80102500 tx=0x7f9e68008040 comp rx=0 tx=0).stop 2024-09-10T10:45:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.971+0000 7f9e85495640 1 -- 172.21.15.7:0/1418120905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e80102e10 msgr2=0x7f9e80100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.971+0000 7f9e85495640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e80102e10 0x7f9e80100a70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000b4a0 tx=0x7f9e7000b960 comp rx=0 tx=0).stop 2024-09-10T10:45:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.971+0000 7f9e85495640 1 -- 172.21.15.7:0/1418120905 shutdown_connections 2024-09-10T10:45:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.971+0000 7f9e85495640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f9e6c0a2c10 0x7f9e6c0a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.971+0000 7f9e85495640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e8010c1b0 0x7f9e801014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.971+0000 7f9e85495640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e80103730 0x7f9e80100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.971+0000 7f9e85495640 1 --2- 172.21.15.7:0/1418120905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e80102e10 0x7f9e80100a70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.971+0000 7f9e85495640 1 -- 172.21.15.7:0/1418120905 >> 172.21.15.7:0/1418120905 conn(0x7f9e800fc7d0 msgr2=0x7f9e800fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.972+0000 7f9e85495640 1 -- 172.21.15.7:0/1418120905 shutdown_connections 2024-09-10T10:45:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:01.972+0000 7f9e85495640 1 -- 172.21.15.7:0/1418120905 wait complete. 2024-09-10T10:45:02.437 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574921 for osd.1 2024-09-10T10:45:02.437 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:02.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:02 smithi007 ceph-mon[30873]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:02.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:02 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4030662004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T10:45:02.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:02 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1418120905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T10:45:02.749 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:02 smithi103 ceph-mon[35160]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:02.749 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:02 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4030662004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T10:45:02.749 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:02 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1418120905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T10:45:02.777 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:02 smithi133 ceph-mon[35282]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:02.777 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:02 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4030662004' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T10:45:02.778 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:02 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1418120905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T10:45:02.927 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:02.928 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:02.928 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:02.928 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:03.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.163+0000 7f0a01c33640 1 -- 172.21.15.7:0/322962875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc101de0 msgr2=0x7f09fc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.163+0000 7f0a01c33640 1 --2- 172.21.15.7:0/322962875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc101de0 0x7f09fc1021c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009900 tx=0x7f09ec02a410 comp rx=0 tx=0).stop 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.163+0000 7f0a01c33640 1 -- 172.21.15.7:0/322962875 shutdown_connections 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.163+0000 7f0a01c33640 1 --2- 172.21.15.7:0/322962875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09fc106f10 0x7f09fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.163+0000 7f0a01c33640 1 --2- 172.21.15.7:0/322962875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc102790 0x7f09fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.163+0000 7f0a01c33640 1 --2- 172.21.15.7:0/322962875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc101de0 0x7f09fc1021c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.163+0000 7f0a01c33640 1 -- 172.21.15.7:0/322962875 >> 172.21.15.7:0/322962875 conn(0x7f09fc0fd8c0 msgr2=0x7f09fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.163+0000 7f0a01c33640 1 -- 172.21.15.7:0/322962875 shutdown_connections 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.163+0000 7f0a01c33640 1 -- 172.21.15.7:0/322962875 wait complete. 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f0a01c33640 1 Processor -- start 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f0a01c33640 1 -- start start 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f0a01c33640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc101de0 0x7f09fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:03.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f0a01c33640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc102790 0x7f09fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f0a01c33640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09fc106f10 0x7f09fc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f0a01c33640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc111f70 con 0x7f09fc101de0 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f0a01c33640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc111df0 con 0x7f09fc106f10 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f0a01c33640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1120f0 con 0x7f09fc102790 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc101de0 0x7f09fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc101de0 0x7f09fc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33020/0 (socket says 172.21.15.7:33020) 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f09fb7fe640 1 -- 172.21.15.7:0/2507561895 learned_addr learned my addr 172.21.15.7:0/2507561895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.164+0000 7f09fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09fc106f10 0x7f09fc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f09f2ffd640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc102790 0x7f09fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f09fbfff640 1 -- 172.21.15.7:0/2507561895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc102790 msgr2=0x7f09fc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f09fbfff640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc102790 0x7f09fc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f09fbfff640 1 -- 172.21.15.7:0/2507561895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc101de0 msgr2=0x7f09fc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f09fbfff640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc101de0 0x7f09fc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f09fbfff640 1 -- 172.21.15.7:0/2507561895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc1a6900 con 0x7f09fc106f10 2024-09-10T10:45:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f09fbfff640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09fc106f10 0x7f09fc074f70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09f400e940 tx=0x7f09f400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f09f97fa640 1 -- 172.21.15.7:0/2507561895 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4004d20 con 0x7f09fc106f10 2024-09-10T10:45:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f0a01c33640 1 -- 172.21.15.7:0/2507561895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc1a6bf0 con 0x7f09fc106f10 2024-09-10T10:45:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f09f97fa640 1 -- 172.21.15.7:0/2507561895 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f4005020 con 0x7f09fc106f10 2024-09-10T10:45:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f0a01c33640 1 -- 172.21.15.7:0/2507561895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc1a7130 con 0x7f09fc106f10 2024-09-10T10:45:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.165+0000 7f09f97fa640 1 -- 172.21.15.7:0/2507561895 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f40116b0 con 0x7f09fc106f10 2024-09-10T10:45:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.166+0000 7f0a01c33640 1 -- 172.21.15.7:0/2507561895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09dc005290 con 0x7f09fc106f10 2024-09-10T10:45:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.168+0000 7f09f97fa640 1 -- 172.21.15.7:0/2507561895 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f09f400cb20 con 0x7f09fc106f10 2024-09-10T10:45:03.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.170+0000 7f09f97fa640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09e80a2d10 0x7f09e80a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:03.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.170+0000 7f09fb7fe640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09e80a2d10 0x7f09e80a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:03.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.170+0000 7f09f97fa640 1 -- 172.21.15.7:0/2507561895 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f09f40cf0e0 con 0x7f09fc106f10 2024-09-10T10:45:03.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.170+0000 7f09fb7fe640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09e80a2d10 0x7f09e80a51d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0096c0 tx=0x7f09ec005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:03.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.173+0000 7f09f97fa640 1 -- 172.21.15.7:0/2507561895 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09f4016030 con 0x7f09fc106f10 2024-09-10T10:45:03.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.352+0000 7f0a01c33640 1 -- 172.21.15.7:0/2507561895 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f09dc005850 con 0x7f09fc106f10 2024-09-10T10:45:03.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.352+0000 7f09f97fa640 1 -- 172.21.15.7:0/2507561895 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f09f409be90 con 0x7f09fc106f10 2024-09-10T10:45:03.354 INFO:teuthology.orchestra.run.smithi007.stdout:283467841563 2024-09-10T10:45:03.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 -- 172.21.15.7:0/2507561895 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09e80a2d10 msgr2=0x7f09e80a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:03.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09e80a2d10 0x7f09e80a51d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0096c0 tx=0x7f09ec005a20 comp rx=0 tx=0).stop 2024-09-10T10:45:03.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 -- 172.21.15.7:0/2507561895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09fc106f10 msgr2=0x7f09fc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:03.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09fc106f10 0x7f09fc074f70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09f400e940 tx=0x7f09f400ee00 comp rx=0 tx=0).stop 2024-09-10T10:45:03.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 -- 172.21.15.7:0/2507561895 shutdown_connections 2024-09-10T10:45:03.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f09e80a2d10 0x7f09e80a51d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09fc106f10 0x7f09fc074f70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09fc102790 0x7f09fc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 --2- 172.21.15.7:0/2507561895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc101de0 0x7f09fc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:03.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 -- 172.21.15.7:0/2507561895 >> 172.21.15.7:0/2507561895 conn(0x7f09fc0fd8c0 msgr2=0x7f09fc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:03.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 -- 172.21.15.7:0/2507561895 shutdown_connections 2024-09-10T10:45:03.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:03.355+0000 7f0a01c33640 1 -- 172.21.15.7:0/2507561895 wait complete. 2024-09-10T10:45:03.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:03 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2507561895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T10:45:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:03 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2507561895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T10:45:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:03 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2507561895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T10:45:04.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.147+0000 7f5f12234640 1 -- 172.21.15.7:0/2007056532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c101bc0 msgr2=0x7f5f0c10d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.147+0000 7f5f12234640 1 --2- 172.21.15.7:0/2007056532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c101bc0 0x7f5f0c10d910 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009ca0 tx=0x7f5f0402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.147+0000 7f5f12234640 1 -- 172.21.15.7:0/2007056532 shutdown_connections 2024-09-10T10:45:04.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.147+0000 7f5f12234640 1 --2- 172.21.15.7:0/2007056532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c101bc0 0x7f5f0c10d910 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.147+0000 7f5f12234640 1 --2- 172.21.15.7:0/2007056532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f0c101240 0x7f5f0c101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.147+0000 7f5f12234640 1 --2- 172.21.15.7:0/2007056532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0c105a90 0x7f5f0c105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.147+0000 7f5f12234640 1 -- 172.21.15.7:0/2007056532 >> 172.21.15.7:0/2007056532 conn(0x7f5f0c077210 msgr2=0x7f5f0c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.147+0000 7f5f12234640 1 -- 172.21.15.7:0/2007056532 shutdown_connections 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.147+0000 7f5f12234640 1 -- 172.21.15.7:0/2007056532 wait complete. 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.148+0000 7f5f12234640 1 Processor -- start 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.148+0000 7f5f12234640 1 -- start start 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.148+0000 7f5f12234640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c101240 0x7f5f0c10e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.148+0000 7f5f12234640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0c101bc0 0x7f5f0c10e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.148+0000 7f5f12234640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f0c105a90 0x7f5f0c10eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.148+0000 7f5f12234640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c112090 con 0x7f5f0c101240 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.148+0000 7f5f12234640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c111f10 con 0x7f5f0c105a90 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.148+0000 7f5f12234640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c112210 con 0x7f5f0c101bc0 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c101240 0x7f5f0c10e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c101240 0x7f5f0c10e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33038/0 (socket says 172.21.15.7:33038) 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0b7fe640 1 -- 172.21.15.7:0/2606717605 learned_addr learned my addr 172.21.15.7:0/2606717605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0bfff640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f0c105a90 0x7f5f0c10eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0affd640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0c101bc0 0x7f5f0c10e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0b7fe640 1 -- 172.21.15.7:0/2606717605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0c101bc0 msgr2=0x7f5f0c10e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0b7fe640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0c101bc0 0x7f5f0c10e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0b7fe640 1 -- 172.21.15.7:0/2606717605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f0c105a90 msgr2=0x7f5f0c10eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0b7fe640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f0c105a90 0x7f5f0c10eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0b7fe640 1 -- 172.21.15.7:0/2606717605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c1a68f0 con 0x7f5f0c101240 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0bfff640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f0c105a90 0x7f5f0c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0affd640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0c101bc0 0x7f5f0c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f0b7fe640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c101240 0x7f5f0c10e420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00d770 tx=0x7f5efc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:04.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f08ff9640 1 -- 172.21.15.7:0/2606717605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc017070 con 0x7f5f0c101240 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f08ff9640 1 -- 172.21.15.7:0/2606717605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5efc005500 con 0x7f5f0c101240 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f08ff9640 1 -- 172.21.15.7:0/2606717605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc004e00 con 0x7f5f0c101240 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f12234640 1 -- 172.21.15.7:0/2606717605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0c1a6be0 con 0x7f5f0c101240 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.149+0000 7f5f12234640 1 -- 172.21.15.7:0/2606717605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0c1a70d0 con 0x7f5f0c101240 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.151+0000 7f5f12234640 1 -- 172.21.15.7:0/2606717605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0c101680 con 0x7f5f0c101240 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.152+0000 7f5f08ff9640 1 -- 172.21.15.7:0/2606717605 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5efc004fa0 con 0x7f5f0c101240 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.152+0000 7f5f08ff9640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5ef40a2c10 0x7f5ef40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.152+0000 7f5f0affd640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5ef40a2c10 0x7f5ef40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.152+0000 7f5f08ff9640 1 -- 172.21.15.7:0/2606717605 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f5efc0cec40 con 0x7f5f0c101240 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.153+0000 7f5f0affd640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5ef40a2c10 0x7f5ef40a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f00005d00 tx=0x7f5f00009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:04.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.157+0000 7f5f08ff9640 1 -- 172.21.15.7:0/2606717605 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5efc0d5080 con 0x7f5f0c101240 2024-09-10T10:45:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.339+0000 7f5f12234640 1 -- 172.21.15.7:0/2606717605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5f0c1a7340 con 0x7f5f0c101240 2024-09-10T10:45:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.340+0000 7f5f08ff9640 1 -- 172.21.15.7:0/2606717605 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5efc09ba50 con 0x7f5f0c101240 2024-09-10T10:45:04.341 INFO:teuthology.orchestra.run.smithi007.stdout:313532612631 2024-09-10T10:45:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.252+0000 7f2f0ffba640 1 -- 172.21.15.7:0/997742638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f081082a0 msgr2=0x7f2f0810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.252+0000 7f2f0ffba640 1 --2- 172.21.15.7:0/997742638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f081082a0 0x7f2f0810ec40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009cd0 tx=0x7f2f0002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.253+0000 7f2f0ffba640 1 -- 172.21.15.7:0/997742638 shutdown_connections 2024-09-10T10:45:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.253+0000 7f2f0ffba640 1 --2- 172.21.15.7:0/997742638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f081082a0 0x7f2f0810ec40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.253+0000 7f2f0ffba640 1 --2- 172.21.15.7:0/997742638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f081036e0 0x7f2f08107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.253+0000 7f2f0ffba640 1 --2- 172.21.15.7:0/997742638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f08102d30 0x7f2f08103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.253+0000 7f2f0ffba640 1 -- 172.21.15.7:0/997742638 >> 172.21.15.7:0/997742638 conn(0x7f2f080fc7d0 msgr2=0x7f2f080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.253+0000 7f2f0ffba640 1 -- 172.21.15.7:0/997742638 shutdown_connections 2024-09-10T10:45:04.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.253+0000 7f2f0ffba640 1 -- 172.21.15.7:0/997742638 wait complete. 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.253+0000 7f2f0ffba640 1 Processor -- start 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.253+0000 7f2f0ffba640 1 -- start start 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0ffba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f08102d30 0x7f2f08198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0ffba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f081036e0 0x7f2f08198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0ffba640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f081082a0 0x7f2f0819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0ffba640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f081120a0 con 0x7f2f081036e0 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0ffba640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08111f20 con 0x7f2f081082a0 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0ffba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08112220 con 0x7f2f08102d30 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0d555640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f081036e0 0x7f2f08198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0d555640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f081036e0 0x7f2f08198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33048/0 (socket says 172.21.15.7:33048) 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0d555640 1 -- 172.21.15.7:0/4075192062 learned_addr learned my addr 172.21.15.7:0/4075192062 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0e557640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f081082a0 0x7f2f0819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0dd56640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f08102d30 0x7f2f08198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0d555640 1 -- 172.21.15.7:0/4075192062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f08102d30 msgr2=0x7f2f08198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0d555640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f08102d30 0x7f2f08198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0d555640 1 -- 172.21.15.7:0/4075192062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f081082a0 msgr2=0x7f2f0819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0d555640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f081082a0 0x7f2f0819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0d555640 1 -- 172.21.15.7:0/4075192062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0819d4e0 con 0x7f2f081036e0 2024-09-10T10:45:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0e557640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f081082a0 0x7f2f0819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0d555640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f081036e0 0x7f2f08198a70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800d740 tx=0x7f2ef800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2efeffd640 1 -- 172.21.15.7:0/4075192062 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8017070 con 0x7f2f081036e0 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2efeffd640 1 -- 172.21.15.7:0/4075192062 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef800de50 con 0x7f2f081036e0 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0ffba640 1 -- 172.21.15.7:0/4075192062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0819d7d0 con 0x7f2f081036e0 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2efeffd640 1 -- 172.21.15.7:0/4075192062 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800f6b0 con 0x7f2f081036e0 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.254+0000 7f2f0ffba640 1 -- 172.21.15.7:0/4075192062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f081a50b0 con 0x7f2f081036e0 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.256+0000 7f2f0ffba640 1 -- 172.21.15.7:0/4075192062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f08103110 con 0x7f2f081036e0 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.257+0000 7f2efeffd640 1 -- 172.21.15.7:0/4075192062 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2ef801a070 con 0x7f2f081036e0 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.258+0000 7f2efeffd640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2ef40a2c90 0x7f2ef40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.258+0000 7f2f0dd56640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2ef40a2c90 0x7f2ef40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.258+0000 7f2efeffd640 1 -- 172.21.15.7:0/4075192062 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f2ef80cebb0 con 0x7f2f081036e0 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.258+0000 7f2f0dd56640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2ef40a2c90 0x7f2ef40a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f04005fd0 tx=0x7f2f04005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:04.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.262+0000 7f2efeffd640 1 -- 172.21.15.7:0/4075192062 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ef809b9c0 con 0x7f2f081036e0 2024-09-10T10:45:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 -- 172.21.15.7:0/2606717605 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5ef40a2c10 msgr2=0x7f5ef40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5ef40a2c10 0x7f5ef40a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f00005d00 tx=0x7f5f00009550 comp rx=0 tx=0).stop 2024-09-10T10:45:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 -- 172.21.15.7:0/2606717605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c101240 msgr2=0x7f5f0c10e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c101240 0x7f5f0c10e420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00d770 tx=0x7f5efc00dc30 comp rx=0 tx=0).stop 2024-09-10T10:45:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 -- 172.21.15.7:0/2606717605 shutdown_connections 2024-09-10T10:45:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f5ef40a2c10 0x7f5ef40a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5f0c105a90 0x7f5f0c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0c101bc0 0x7f5f0c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 --2- 172.21.15.7:0/2606717605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c101240 0x7f5f0c10e420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 -- 172.21.15.7:0/2606717605 >> 172.21.15.7:0/2606717605 conn(0x7f5f0c077210 msgr2=0x7f5f0c103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 -- 172.21.15.7:0/2606717605 shutdown_connections 2024-09-10T10:45:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.342+0000 7f5f12234640 1 -- 172.21.15.7:0/2606717605 wait complete. 2024-09-10T10:45:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.440+0000 7f2f0ffba640 1 -- 172.21.15.7:0/4075192062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2f0810a2e0 con 0x7f2f081036e0 2024-09-10T10:45:04.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.440+0000 7f2efeffd640 1 -- 172.21.15.7:0/4075192062 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2ef809b2c0 con 0x7f2f081036e0 2024-09-10T10:45:04.441 INFO:teuthology.orchestra.run.smithi007.stdout:103079215167 2024-09-10T10:45:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 -- 172.21.15.7:0/4075192062 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2ef40a2c90 msgr2=0x7f2ef40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2ef40a2c90 0x7f2ef40a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f04005fd0 tx=0x7f2f04005850 comp rx=0 tx=0).stop 2024-09-10T10:45:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 -- 172.21.15.7:0/4075192062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f081036e0 msgr2=0x7f2f08198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f081036e0 0x7f2f08198a70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800d740 tx=0x7f2ef800dc00 comp rx=0 tx=0).stop 2024-09-10T10:45:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 -- 172.21.15.7:0/4075192062 shutdown_connections 2024-09-10T10:45:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2ef40a2c90 0x7f2ef40a5150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2f081082a0 0x7f2f0819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f081036e0 0x7f2f08198a70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 --2- 172.21.15.7:0/4075192062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f08102d30 0x7f2f08198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 -- 172.21.15.7:0/4075192062 >> 172.21.15.7:0/4075192062 conn(0x7f2f080fc7d0 msgr2=0x7f2f08105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 -- 172.21.15.7:0/4075192062 shutdown_connections 2024-09-10T10:45:04.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:04.442+0000 7f2f0ffba640 1 -- 172.21.15.7:0/4075192062 wait complete. 2024-09-10T10:45:04.778 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:04 smithi007 ceph-mon[30873]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:04.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:04 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2606717605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T10:45:04.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:04 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4075192062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T10:45:04.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:04 smithi103 ceph-mon[35160]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:04.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:04 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2606717605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T10:45:04.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:04 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4075192062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T10:45:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:04 smithi133 ceph-mon[35282]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:04 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2606717605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T10:45:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:04 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4075192062' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T10:45:05.201 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215164 got 103079215167 for osd.3 2024-09-10T10:45:05.202 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:05.621 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234019 got 231928234022 for osd.8 2024-09-10T10:45:05.622 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:06.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612628 got 313532612631 for osd.11 2024-09-10T10:45:06.667 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:06.667 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:06.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:06 smithi133 ceph-mon[35282]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:06.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:06 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:06.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:06 smithi007 ceph-mon[30873]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:06.780 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:06 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.863+0000 7f19560aa640 1 -- 172.21.15.7:0/1712904668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1950106f10 msgr2=0x7f195010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.863+0000 7f19560aa640 1 --2- 172.21.15.7:0/1712904668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1950106f10 0x7f195010ea90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1948009ca0 tx=0x7f194802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.863+0000 7f19560aa640 1 -- 172.21.15.7:0/1712904668 shutdown_connections 2024-09-10T10:45:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.863+0000 7f19560aa640 1 --2- 172.21.15.7:0/1712904668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1950106f10 0x7f195010ea90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.863+0000 7f19560aa640 1 --2- 172.21.15.7:0/1712904668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950102790 0x7f19501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.863+0000 7f19560aa640 1 --2- 172.21.15.7:0/1712904668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1950101de0 0x7f19501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.863+0000 7f19560aa640 1 -- 172.21.15.7:0/1712904668 >> 172.21.15.7:0/1712904668 conn(0x7f19500fd8c0 msgr2=0x7f19500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.863+0000 7f19560aa640 1 -- 172.21.15.7:0/1712904668 shutdown_connections 2024-09-10T10:45:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.863+0000 7f19560aa640 1 -- 172.21.15.7:0/1712904668 wait complete. 2024-09-10T10:45:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f19560aa640 1 Processor -- start 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f19560aa640 1 -- start start 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f19560aa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1950101de0 0x7f1950198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f19560aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1950102790 0x7f19501988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f19560aa640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950106f10 0x7f195019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f19560aa640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1950111f70 con 0x7f1950101de0 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f19560aa640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1950111df0 con 0x7f1950106f10 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f19560aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f19501120f0 con 0x7f1950102790 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f194f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1950101de0 0x7f1950198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f194f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1950101de0 0x7f1950198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33074/0 (socket says 172.21.15.7:33074) 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.864+0000 7f194f7fe640 1 -- 172.21.15.7:0/4241370217 learned_addr learned my addr 172.21.15.7:0/4241370217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194effd640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1950102790 0x7f19501988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194ffff640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950106f10 0x7f195019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194f7fe640 1 -- 172.21.15.7:0/4241370217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1950102790 msgr2=0x7f19501988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194f7fe640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1950102790 0x7f19501988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194f7fe640 1 -- 172.21.15.7:0/4241370217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950106f10 msgr2=0x7f195019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194f7fe640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950106f10 0x7f195019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194f7fe640 1 -- 172.21.15.7:0/4241370217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195019d340 con 0x7f1950101de0 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194f7fe640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1950101de0 0x7f1950198390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f194000d950 tx=0x7f194000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194cff9640 1 -- 172.21.15.7:0/4241370217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940017030 con 0x7f1950101de0 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194cff9640 1 -- 172.21.15.7:0/4241370217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1940004020 con 0x7f1950101de0 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f19560aa640 1 -- 172.21.15.7:0/4241370217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195019d630 con 0x7f1950101de0 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f194cff9640 1 -- 172.21.15.7:0/4241370217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19400047a0 con 0x7f1950101de0 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.865+0000 7f19560aa640 1 -- 172.21.15.7:0/4241370217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19501a4f10 con 0x7f1950101de0 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.866+0000 7f19560aa640 1 -- 172.21.15.7:0/4241370217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19501021c0 con 0x7f1950101de0 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.867+0000 7f194cff9640 1 -- 172.21.15.7:0/4241370217 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f19400041d0 con 0x7f1950101de0 2024-09-10T10:45:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.868+0000 7f194cff9640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f19380a2c10 0x7f19380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.869+0000 7f194effd640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f19380a2c10 0x7f19380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.869+0000 7f194cff9640 1 -- 172.21.15.7:0/4241370217 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f19400ceae0 con 0x7f1950101de0 2024-09-10T10:45:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.869+0000 7f194effd640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f19380a2c10 0x7f19380a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19500fdce0 tx=0x7f1944005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:06.873+0000 7f194cff9640 1 -- 172.21.15.7:0/4241370217 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f194009b8f0 con 0x7f1950101de0 2024-09-10T10:45:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:06 smithi103 ceph-mon[35160]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:06 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.055+0000 7f19560aa640 1 -- 172.21.15.7:0/4241370217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f19501996f0 con 0x7f1950101de0 2024-09-10T10:45:07.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.056+0000 7f194cff9640 1 -- 172.21.15.7:0/4241370217 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f194009b1f0 con 0x7f1950101de0 2024-09-10T10:45:07.057 INFO:teuthology.orchestra.run.smithi007.stdout:77309411397 2024-09-10T10:45:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 -- 172.21.15.7:0/4241370217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f19380a2c10 msgr2=0x7f19380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f19380a2c10 0x7f19380a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19500fdce0 tx=0x7f1944005f90 comp rx=0 tx=0).stop 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 -- 172.21.15.7:0/4241370217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1950101de0 msgr2=0x7f1950198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1950101de0 0x7f1950198390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f194000d950 tx=0x7f194000de10 comp rx=0 tx=0).stop 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 -- 172.21.15.7:0/4241370217 shutdown_connections 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f19380a2c10 0x7f19380a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1950106f10 0x7f195019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1950102790 0x7f19501988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 --2- 172.21.15.7:0/4241370217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1950101de0 0x7f1950198390 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 -- 172.21.15.7:0/4241370217 >> 172.21.15.7:0/4241370217 conn(0x7f19500fd8c0 msgr2=0x7f19500fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 -- 172.21.15.7:0/4241370217 shutdown_connections 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:07.058+0000 7f19560aa640 1 -- 172.21.15.7:0/4241370217 wait complete. 2024-09-10T10:45:07.288 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841562 got 283467841563 for osd.10 2024-09-10T10:45:07.288 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:07.846 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:07 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/4241370217' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T10:45:07.846 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.846 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.846 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.846 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:07 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.873 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411397 for osd.2 2024-09-10T10:45:07.873 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:07 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/4241370217' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T10:45:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:07 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:07 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/4241370217' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T10:45:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:07 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.115+0000 7fe8bc46f640 1 -- 172.21.15.7:0/75336023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b4100e20 msgr2=0x7fe8b4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.115+0000 7fe8bc46f640 1 --2- 172.21.15.7:0/75336023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b4100e20 0x7fe8b4101200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009900 tx=0x7fe8b002a480 comp rx=0 tx=0).stop 2024-09-10T10:45:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.115+0000 7fe8bc46f640 1 -- 172.21.15.7:0/75336023 shutdown_connections 2024-09-10T10:45:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.115+0000 7fe8bc46f640 1 --2- 172.21.15.7:0/75336023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b4102ef0 0x7fe8b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.115+0000 7fe8bc46f640 1 --2- 172.21.15.7:0/75336023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b4102570 0x7fe8b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.115+0000 7fe8bc46f640 1 --2- 172.21.15.7:0/75336023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b4100e20 0x7fe8b4101200 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.115+0000 7fe8bc46f640 1 -- 172.21.15.7:0/75336023 >> 172.21.15.7:0/75336023 conn(0x7fe8b40fc7d0 msgr2=0x7fe8b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.115+0000 7fe8bc46f640 1 -- 172.21.15.7:0/75336023 shutdown_connections 2024-09-10T10:45:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.115+0000 7fe8bc46f640 1 -- 172.21.15.7:0/75336023 wait complete. 2024-09-10T10:45:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.116+0000 7fe8bc46f640 1 Processor -- start 2024-09-10T10:45:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.116+0000 7fe8bc46f640 1 -- start start 2024-09-10T10:45:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.116+0000 7fe8bc46f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b4100e20 0x7fe8b40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.116+0000 7fe8bc46f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b4102570 0x7fe8b40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.116+0000 7fe8bc46f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b4102ef0 0x7fe8b40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.116+0000 7fe8bc46f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b4110f80 con 0x7fe8b4102ef0 2024-09-10T10:45:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.116+0000 7fe8bc46f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b4110e00 con 0x7fe8b4102570 2024-09-10T10:45:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.116+0000 7fe8bc46f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b4111100 con 0x7fe8b4100e20 2024-09-10T10:45:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8baa0c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b4102ef0 0x7fe8b40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8baa0c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b4102ef0 0x7fe8b40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33100/0 (socket says 172.21.15.7:33100) 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8baa0c640 1 -- 172.21.15.7:0/758776675 learned_addr learned my addr 172.21.15.7:0/758776675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8ba20b640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b4100e20 0x7fe8b40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8b9a0a640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b4102570 0x7fe8b40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8ba20b640 1 -- 172.21.15.7:0/758776675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b4102570 msgr2=0x7fe8b40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8ba20b640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b4102570 0x7fe8b40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8ba20b640 1 -- 172.21.15.7:0/758776675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b4102ef0 msgr2=0x7fe8b40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8ba20b640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b4102ef0 0x7fe8b40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8ba20b640 1 -- 172.21.15.7:0/758776675 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b41a69c0 con 0x7fe8b4100e20 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8baa0c640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b4102ef0 0x7fe8b40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8ba20b640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b4100e20 0x7fe8b40fcc80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0005940 tx=0x7fe8b0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8ab7fe640 1 -- 172.21.15.7:0/758776675 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b003f070 con 0x7fe8b4100e20 2024-09-10T10:45:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8bc46f640 1 -- 172.21.15.7:0/758776675 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b41a6c50 con 0x7fe8b4100e20 2024-09-10T10:45:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.117+0000 7fe8ab7fe640 1 -- 172.21.15.7:0/758776675 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b0031c30 con 0x7fe8b4100e20 2024-09-10T10:45:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.118+0000 7fe8ab7fe640 1 -- 172.21.15.7:0/758776675 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0035030 con 0x7fe8b4100e20 2024-09-10T10:45:08.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.118+0000 7fe8bc46f640 1 -- 172.21.15.7:0/758776675 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b41a7180 con 0x7fe8b4100e20 2024-09-10T10:45:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.118+0000 7fe8bc46f640 1 -- 172.21.15.7:0/758776675 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b4101200 con 0x7fe8b4100e20 2024-09-10T10:45:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.120+0000 7fe8ab7fe640 1 -- 172.21.15.7:0/758776675 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe8b0042070 con 0x7fe8b4100e20 2024-09-10T10:45:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.120+0000 7fe8ab7fe640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe8a00a2c10 0x7fe8a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.121+0000 7fe8b9a0a640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe8a00a2c10 0x7fe8a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.121+0000 7fe8ab7fe640 1 -- 172.21.15.7:0/758776675 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fe8b00f00a0 con 0x7fe8b4100e20 2024-09-10T10:45:08.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.121+0000 7fe8b9a0a640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe8a00a2c10 0x7fe8a00a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40fe710 tx=0x7fe8a400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:08.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.124+0000 7fe8ab7fe640 1 -- 172.21.15.7:0/758776675 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8b0031de0 con 0x7fe8b4100e20 2024-09-10T10:45:08.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.303+0000 7fe8bc46f640 1 -- 172.21.15.7:0/758776675 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe8b410a0a0 con 0x7fe8b4100e20 2024-09-10T10:45:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.303+0000 7fe8ab7fe640 1 -- 172.21.15.7:0/758776675 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe8b00bc860 con 0x7fe8b4100e20 2024-09-10T10:45:08.305 INFO:teuthology.orchestra.run.smithi007.stdout:197568495660 2024-09-10T10:45:08.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 -- 172.21.15.7:0/758776675 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe8a00a2c10 msgr2=0x7fe8a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe8a00a2c10 0x7fe8a00a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40fe710 tx=0x7fe8a400a2d0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 -- 172.21.15.7:0/758776675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b4100e20 msgr2=0x7fe8b40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b4100e20 0x7fe8b40fcc80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0005940 tx=0x7fe8b0003c10 comp rx=0 tx=0).stop 2024-09-10T10:45:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 -- 172.21.15.7:0/758776675 shutdown_connections 2024-09-10T10:45:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe8a00a2c10 0x7fe8a00a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b4102ef0 0x7fe8b40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8b4102570 0x7fe8b40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 --2- 172.21.15.7:0/758776675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b4100e20 0x7fe8b40fcc80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 -- 172.21.15.7:0/758776675 >> 172.21.15.7:0/758776675 conn(0x7fe8b40fc7d0 msgr2=0x7fe8b406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 -- 172.21.15.7:0/758776675 shutdown_connections 2024-09-10T10:45:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.306+0000 7fe8bc46f640 1 -- 172.21.15.7:0/758776675 wait complete. 2024-09-10T10:45:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.308+0000 7fa24cd29640 1 -- 172.21.15.7:0/4061521917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248101dd0 msgr2=0x7fa2481021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.308+0000 7fa24cd29640 1 --2- 172.21.15.7:0/4061521917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248101dd0 0x7fa2481021b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa238009900 tx=0x7fa23802a410 comp rx=0 tx=0).stop 2024-09-10T10:45:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.308+0000 7fa24cd29640 1 -- 172.21.15.7:0/4061521917 shutdown_connections 2024-09-10T10:45:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.308+0000 7fa24cd29640 1 --2- 172.21.15.7:0/4061521917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa248106f00 0x7fa24810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.308+0000 7fa24cd29640 1 --2- 172.21.15.7:0/4061521917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa248102780 0x7fa2481067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.308+0000 7fa24cd29640 1 --2- 172.21.15.7:0/4061521917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248101dd0 0x7fa2481021b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.308+0000 7fa24cd29640 1 -- 172.21.15.7:0/4061521917 >> 172.21.15.7:0/4061521917 conn(0x7fa2480fd8b0 msgr2=0x7fa2480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.308+0000 7fa24cd29640 1 -- 172.21.15.7:0/4061521917 shutdown_connections 2024-09-10T10:45:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.308+0000 7fa24cd29640 1 -- 172.21.15.7:0/4061521917 wait complete. 2024-09-10T10:45:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.309+0000 7fa24cd29640 1 Processor -- start 2024-09-10T10:45:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.309+0000 7fa24cd29640 1 -- start start 2024-09-10T10:45:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.309+0000 7fa24cd29640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa248101dd0 0x7fa2480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.309+0000 7fa24cd29640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248102780 0x7fa248074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.309+0000 7fa24cd29640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa248106f00 0x7fa248074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.309+0000 7fa24cd29640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa248111f20 con 0x7fa248102780 2024-09-10T10:45:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.309+0000 7fa24cd29640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa248111da0 con 0x7fa248106f00 2024-09-10T10:45:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.309+0000 7fa24cd29640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481120a0 con 0x7fa248101dd0 2024-09-10T10:45:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.310+0000 7fa245d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248102780 0x7fa248074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.310+0000 7fa245d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248102780 0x7fa248074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33130/0 (socket says 172.21.15.7:33130) 2024-09-10T10:45:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.310+0000 7fa245d9b640 1 -- 172.21.15.7:0/2926792944 learned_addr learned my addr 172.21.15.7:0/2926792944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.310+0000 7fa246d9d640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa248106f00 0x7fa248074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.310+0000 7fa245d9b640 1 -- 172.21.15.7:0/2926792944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa248101dd0 msgr2=0x7fa2480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.310+0000 7fa245d9b640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa248101dd0 0x7fa2480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.310+0000 7fa245d9b640 1 -- 172.21.15.7:0/2926792944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa248106f00 msgr2=0x7fa248074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.310+0000 7fa245d9b640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa248106f00 0x7fa248074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.310+0000 7fa245d9b640 1 -- 172.21.15.7:0/2926792944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2481aae10 con 0x7fa248102780 2024-09-10T10:45:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.310+0000 7fa245d9b640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248102780 0x7fa248074a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa24000ebc0 tx=0x7fa24000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.311+0000 7fa2377fe640 1 -- 172.21.15.7:0/2926792944 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240010630 con 0x7fa248102780 2024-09-10T10:45:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.311+0000 7fa2377fe640 1 -- 172.21.15.7:0/2926792944 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa240010cb0 con 0x7fa248102780 2024-09-10T10:45:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.311+0000 7fa24cd29640 1 -- 172.21.15.7:0/2926792944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2481aafe0 con 0x7fa248102780 2024-09-10T10:45:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.311+0000 7fa24cd29640 1 -- 172.21.15.7:0/2926792944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2481ab4c0 con 0x7fa248102780 2024-09-10T10:45:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.311+0000 7fa2377fe640 1 -- 172.21.15.7:0/2926792944 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240018bd0 con 0x7fa248102780 2024-09-10T10:45:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.312+0000 7fa24cd29640 1 -- 172.21.15.7:0/2926792944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2481032b0 con 0x7fa248102780 2024-09-10T10:45:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.313+0000 7fa2377fe640 1 -- 172.21.15.7:0/2926792944 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa2400107d0 con 0x7fa248102780 2024-09-10T10:45:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.314+0000 7fa2377fe640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2300a2d60 0x7fa2300a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.314+0000 7fa24659c640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2300a2d60 0x7fa2300a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.314+0000 7fa2377fe640 1 -- 172.21.15.7:0/2926792944 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fa2400cf7e0 con 0x7fa248102780 2024-09-10T10:45:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.315+0000 7fa24659c640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2300a2d60 0x7fa2300a5220 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa23802a930 tx=0x7fa238000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:08.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.318+0000 7fa2377fe640 1 -- 172.21.15.7:0/2926792944 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa24009c5f0 con 0x7fa248102780 2024-09-10T10:45:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.507+0000 7fa24cd29640 1 -- 172.21.15.7:0/2926792944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fa24810a0e0 con 0x7fa248102780 2024-09-10T10:45:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.507+0000 7fa2377fe640 1 -- 172.21.15.7:0/2926792944 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa24009bef0 con 0x7fa248102780 2024-09-10T10:45:08.509 INFO:teuthology.orchestra.run.smithi007.stdout:261993005090 2024-09-10T10:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 -- 172.21.15.7:0/2926792944 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2300a2d60 msgr2=0x7fa2300a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2300a2d60 0x7fa2300a5220 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa23802a930 tx=0x7fa238000bc0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 -- 172.21.15.7:0/2926792944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248102780 msgr2=0x7fa248074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248102780 0x7fa248074a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa24000ebc0 tx=0x7fa24000c6a0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 -- 172.21.15.7:0/2926792944 shutdown_connections 2024-09-10T10:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2300a2d60 0x7fa2300a5220 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa248106f00 0x7fa248074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa248102780 0x7fa248074a30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 --2- 172.21.15.7:0/2926792944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa248101dd0 0x7fa2480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 -- 172.21.15.7:0/2926792944 >> 172.21.15.7:0/2926792944 conn(0x7fa2480fd8b0 msgr2=0x7fa2480fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 -- 172.21.15.7:0/2926792944 shutdown_connections 2024-09-10T10:45:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.510+0000 7fa24cd29640 1 -- 172.21.15.7:0/2926792944 wait complete. 2024-09-10T10:45:08.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:08 smithi007 ceph-mon[30873]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:08.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:08 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/758776675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T10:45:08.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.903+0000 7fd39a2f7640 1 -- 172.21.15.7:0/2186245000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd394106f10 msgr2=0x7fd39410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.903+0000 7fd39a2f7640 1 --2- 172.21.15.7:0/2186245000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd394106f10 0x7fd39410ea90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd38c009ca0 tx=0x7fd38c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.903+0000 7fd39a2f7640 1 -- 172.21.15.7:0/2186245000 shutdown_connections 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.903+0000 7fd39a2f7640 1 --2- 172.21.15.7:0/2186245000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd394106f10 0x7fd39410ea90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.903+0000 7fd39a2f7640 1 --2- 172.21.15.7:0/2186245000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd394102790 0x7fd3941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.903+0000 7fd39a2f7640 1 --2- 172.21.15.7:0/2186245000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd394101de0 0x7fd3941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.903+0000 7fd39a2f7640 1 -- 172.21.15.7:0/2186245000 >> 172.21.15.7:0/2186245000 conn(0x7fd3940fd8c0 msgr2=0x7fd3940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.903+0000 7fd39a2f7640 1 -- 172.21.15.7:0/2186245000 shutdown_connections 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.903+0000 7fd39a2f7640 1 -- 172.21.15.7:0/2186245000 wait complete. 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd39a2f7640 1 Processor -- start 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd39a2f7640 1 -- start start 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd39a2f7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd394101de0 0x7fd394198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd39a2f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd394102790 0x7fd394198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd39a2f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd394106f10 0x7fd39419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd39a2f7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd394111f90 con 0x7fd394102790 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd39a2f7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd394111e10 con 0x7fd394106f10 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd39a2f7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd394112110 con 0x7fd394101de0 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd394102790 0x7fd394198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd394102790 0x7fd394198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33142/0 (socket says 172.21.15.7:33142) 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd38bfff640 1 -- 172.21.15.7:0/190371176 learned_addr learned my addr 172.21.15.7:0/190371176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.904+0000 7fd398894640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd394106f10 0x7fd39419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd393fff640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd394101de0 0x7fd394198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd38bfff640 1 -- 172.21.15.7:0/190371176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd394101de0 msgr2=0x7fd394198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd38bfff640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd394101de0 0x7fd394198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd38bfff640 1 -- 172.21.15.7:0/190371176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd394106f10 msgr2=0x7fd39419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd38bfff640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd394106f10 0x7fd39419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd38bfff640 1 -- 172.21.15.7:0/190371176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39419d3c0 con 0x7fd394102790 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd393fff640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd394101de0 0x7fd394198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd398894640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd394106f10 0x7fd39419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd38bfff640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd394102790 0x7fd394198950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b4a0 tx=0x7fd37c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd391ffb640 1 -- 172.21.15.7:0/190371176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c014070 con 0x7fd394102790 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd391ffb640 1 -- 172.21.15.7:0/190371176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd37c010af0 con 0x7fd394102790 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd39a2f7640 1 -- 172.21.15.7:0/190371176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39419d6b0 con 0x7fd394102790 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd391ffb640 1 -- 172.21.15.7:0/190371176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c017d10 con 0x7fd394102790 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.905+0000 7fd39a2f7640 1 -- 172.21.15.7:0/190371176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3941a4f90 con 0x7fd394102790 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.906+0000 7fd39a2f7640 1 -- 172.21.15.7:0/190371176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3941021c0 con 0x7fd394102790 2024-09-10T10:45:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.907+0000 7fd391ffb640 1 -- 172.21.15.7:0/190371176 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd37c017070 con 0x7fd394102790 2024-09-10T10:45:08.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.908+0000 7fd391ffb640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd3800a2c90 0x7fd3800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:08.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.908+0000 7fd393fff640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd3800a2c90 0x7fd3800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:08.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.909+0000 7fd391ffb640 1 -- 172.21.15.7:0/190371176 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fd37c0ce450 con 0x7fd394102790 2024-09-10T10:45:08.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.909+0000 7fd393fff640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd3800a2c90 0x7fd3800a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd38400baa0 tx=0x7fd384005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:08.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:08.913+0000 7fd391ffb640 1 -- 172.21.15.7:0/190371176 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd37c09b370 con 0x7fd394102790 2024-09-10T10:45:08.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:08 smithi103 ceph-mon[35160]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:08.926 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:08 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/758776675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T10:45:08.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:08 smithi133 ceph-mon[35282]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:08.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:08 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/758776675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T10:45:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.090+0000 7fd39a2f7640 1 -- 172.21.15.7:0/190371176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd3941a52d0 con 0x7fd394102790 2024-09-10T10:45:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.090+0000 7fd391ffb640 1 -- 172.21.15.7:0/190371176 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd37c09ac70 con 0x7fd394102790 2024-09-10T10:45:09.091 INFO:teuthology.orchestra.run.smithi007.stdout:180388626481 2024-09-10T10:45:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 -- 172.21.15.7:0/190371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd3800a2c90 msgr2=0x7fd3800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd3800a2c90 0x7fd3800a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd38400baa0 tx=0x7fd384005d90 comp rx=0 tx=0).stop 2024-09-10T10:45:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 -- 172.21.15.7:0/190371176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd394102790 msgr2=0x7fd394198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd394102790 0x7fd394198950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b4a0 tx=0x7fd37c00b960 comp rx=0 tx=0).stop 2024-09-10T10:45:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 -- 172.21.15.7:0/190371176 shutdown_connections 2024-09-10T10:45:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd3800a2c90 0x7fd3800a5150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd394106f10 0x7fd39419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd394102790 0x7fd394198950 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 --2- 172.21.15.7:0/190371176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd394101de0 0x7fd394198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 -- 172.21.15.7:0/190371176 >> 172.21.15.7:0/190371176 conn(0x7fd3940fd8c0 msgr2=0x7fd3940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 -- 172.21.15.7:0/190371176 shutdown_connections 2024-09-10T10:45:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:09.092+0000 7fd39a2f7640 1 -- 172.21.15.7:0/190371176 wait complete. 2024-09-10T10:45:09.803 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:09 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2926792944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T10:45:09.804 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:09 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/190371176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T10:45:09.834 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626478 got 180388626481 for osd.6 2024-09-10T10:45:09.834 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:09.910 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:09 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2926792944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T10:45:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:09 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/190371176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T10:45:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:09 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2926792944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T10:45:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:09 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/190371176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T10:45:10.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495656 got 197568495660 for osd.7 2024-09-10T10:45:10.320 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:10.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:10 smithi007 ceph-mon[30873]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-09-10T10:45:10.719 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005087 got 261993005090 for osd.9 2024-09-10T10:45:10.720 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:10.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:10 smithi103 ceph-mon[35160]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-09-10T10:45:10.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:10 smithi133 ceph-mon[35282]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-09-10T10:45:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.261+0000 7f924fe21640 1 -- 172.21.15.7:0/3182037751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9248101de0 msgr2=0x7f92481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.261+0000 7f924fe21640 1 --2- 172.21.15.7:0/3182037751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9248101de0 0x7f92481021c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9244009900 tx=0x7f924402a410 comp rx=0 tx=0).stop 2024-09-10T10:45:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.261+0000 7f924fe21640 1 -- 172.21.15.7:0/3182037751 shutdown_connections 2024-09-10T10:45:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.261+0000 7f924fe21640 1 --2- 172.21.15.7:0/3182037751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248106f10 0x7f924810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.261+0000 7f924fe21640 1 --2- 172.21.15.7:0/3182037751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9248102790 0x7f92481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.261+0000 7f924fe21640 1 --2- 172.21.15.7:0/3182037751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9248101de0 0x7f92481021c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.261+0000 7f924fe21640 1 -- 172.21.15.7:0/3182037751 >> 172.21.15.7:0/3182037751 conn(0x7f92480fd8c0 msgr2=0x7f92480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.261+0000 7f924fe21640 1 -- 172.21.15.7:0/3182037751 shutdown_connections 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.262+0000 7f924fe21640 1 -- 172.21.15.7:0/3182037751 wait complete. 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.262+0000 7f924fe21640 1 Processor -- start 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.262+0000 7f924fe21640 1 -- start start 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.262+0000 7f924fe21640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9248101de0 0x7f92481983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.262+0000 7f924fe21640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9248102790 0x7f9248198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.262+0000 7f924fe21640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248106f10 0x7f924819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.262+0000 7f924fe21640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9248111f70 con 0x7f9248102790 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.262+0000 7f924fe21640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9248111df0 con 0x7f9248106f10 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.262+0000 7f924fe21640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92481120f0 con 0x7f9248101de0 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924d3bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9248102790 0x7f9248198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924d3bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9248102790 0x7f9248198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35872/0 (socket says 172.21.15.7:35872) 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924d3bc640 1 -- 172.21.15.7:0/1748691539 learned_addr learned my addr 172.21.15.7:0/1748691539 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924dbbd640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9248101de0 0x7f92481983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924e3be640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248106f10 0x7f924819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924d3bc640 1 -- 172.21.15.7:0/1748691539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9248101de0 msgr2=0x7f92481983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924d3bc640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9248101de0 0x7f92481983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924d3bc640 1 -- 172.21.15.7:0/1748691539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248106f10 msgr2=0x7f924819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924d3bc640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248106f10 0x7f924819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924d3bc640 1 -- 172.21.15.7:0/1748691539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924819d380 con 0x7f9248102790 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924e3be640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248106f10 0x7f924819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924d3bc640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9248102790 0x7f9248198910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f923800d770 tx=0x7f923800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f923effd640 1 -- 172.21.15.7:0/1748691539 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238017070 con 0x7f9248102790 2024-09-10T10:45:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f923effd640 1 -- 172.21.15.7:0/1748691539 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9238005500 con 0x7f9248102790 2024-09-10T10:45:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924fe21640 1 -- 172.21.15.7:0/1748691539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924819d670 con 0x7f9248102790 2024-09-10T10:45:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f923effd640 1 -- 172.21.15.7:0/1748691539 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238004e00 con 0x7f9248102790 2024-09-10T10:45:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.263+0000 7f924fe21640 1 -- 172.21.15.7:0/1748691539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92481a4f50 con 0x7f9248102790 2024-09-10T10:45:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.264+0000 7f924fe21640 1 -- 172.21.15.7:0/1748691539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92481021c0 con 0x7f9248102790 2024-09-10T10:45:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.266+0000 7f923effd640 1 -- 172.21.15.7:0/1748691539 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9238004fa0 con 0x7f9248102790 2024-09-10T10:45:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.266+0000 7f923effd640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f92340a2c90 0x7f92340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.266+0000 7f923effd640 1 -- 172.21.15.7:0/1748691539 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f92380cf000 con 0x7f9248102790 2024-09-10T10:45:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.266+0000 7f924dbbd640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f92340a2c90 0x7f92340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.267+0000 7f924dbbd640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f92340a2c90 0x7f92340a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9244009230 tx=0x7f92440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.269+0000 7f923effd640 1 -- 172.21.15.7:0/1748691539 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f923809be90 con 0x7f9248102790 2024-09-10T10:45:11.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.446+0000 7f924fe21640 1 -- 172.21.15.7:0/1748691539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f924810a150 con 0x7f9248102790 2024-09-10T10:45:11.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.446+0000 7f923effd640 1 -- 172.21.15.7:0/1748691539 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f923809b790 con 0x7f9248102790 2024-09-10T10:45:11.448 INFO:teuthology.orchestra.run.smithi007.stdout:133143986236 2024-09-10T10:45:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 -- 172.21.15.7:0/1748691539 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f92340a2c90 msgr2=0x7f92340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f92340a2c90 0x7f92340a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9244009230 tx=0x7f92440058d0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 -- 172.21.15.7:0/1748691539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9248102790 msgr2=0x7f9248198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9248102790 0x7f9248198910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f923800d770 tx=0x7f923800dc30 comp rx=0 tx=0).stop 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 -- 172.21.15.7:0/1748691539 shutdown_connections 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f92340a2c90 0x7f92340a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9248106f10 0x7f924819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9248102790 0x7f9248198910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 --2- 172.21.15.7:0/1748691539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9248101de0 0x7f92481983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 -- 172.21.15.7:0/1748691539 >> 172.21.15.7:0/1748691539 conn(0x7f92480fd8c0 msgr2=0x7f92480febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 -- 172.21.15.7:0/1748691539 shutdown_connections 2024-09-10T10:45:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:11.448+0000 7f924fe21640 1 -- 172.21.15.7:0/1748691539 wait complete. 2024-09-10T10:45:11.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:11 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1748691539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T10:45:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:11 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1748691539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T10:45:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:11 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1748691539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T10:45:12.216 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986232 got 133143986236 for osd.4 2024-09-10T10:45:12.216 DEBUG:teuthology.parallel:result is None 2024-09-10T10:45:12.216 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T10:45:12.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:45:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:12.779 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:12 smithi007 ceph-mon[30873]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:12.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:12 smithi103 ceph-mon[35160]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:12.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:12 smithi133 ceph-mon[35282]: pgmap v306: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.097+0000 7fa234c85640 1 -- 172.21.15.7:0/3311311313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa230102570 msgr2=0x7fa2301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.097+0000 7fa234c85640 1 --2- 172.21.15.7:0/3311311313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa230102570 0x7fa2301029b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa224009900 tx=0x7fa22402a480 comp rx=0 tx=0).stop 2024-09-10T10:45:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.097+0000 7fa234c85640 1 -- 172.21.15.7:0/3311311313 shutdown_connections 2024-09-10T10:45:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.098+0000 7fa234c85640 1 --2- 172.21.15.7:0/3311311313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa230102ef0 0x7fa23010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.098+0000 7fa234c85640 1 --2- 172.21.15.7:0/3311311313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa230102570 0x7fa2301029b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.098+0000 7fa234c85640 1 --2- 172.21.15.7:0/3311311313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa230100e20 0x7fa230101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.098+0000 7fa234c85640 1 -- 172.21.15.7:0/3311311313 >> 172.21.15.7:0/3311311313 conn(0x7fa2300fc7d0 msgr2=0x7fa2300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.098+0000 7fa234c85640 1 -- 172.21.15.7:0/3311311313 shutdown_connections 2024-09-10T10:45:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.098+0000 7fa234c85640 1 -- 172.21.15.7:0/3311311313 wait complete. 2024-09-10T10:45:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.098+0000 7fa234c85640 1 Processor -- start 2024-09-10T10:45:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.098+0000 7fa234c85640 1 -- start start 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa234c85640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa230100e20 0x7fa230199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa234c85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa230102570 0x7fa2301998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa234c85640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa230102ef0 0x7fa2301a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa234c85640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa230110fa0 con 0x7fa230102570 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa234c85640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa230110e20 con 0x7fa230100e20 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa234c85640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa230111120 con 0x7fa230102ef0 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa230102570 0x7fa2301998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa230102570 0x7fa2301998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35894/0 (socket says 172.21.15.7:35894) 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22dd9b640 1 -- 172.21.15.7:0/2667323640 learned_addr learned my addr 172.21.15.7:0/2667323640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22e59c640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa230100e20 0x7fa230199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22ed9d640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa230102ef0 0x7fa2301a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22dd9b640 1 -- 172.21.15.7:0/2667323640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa230102ef0 msgr2=0x7fa2301a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22dd9b640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa230102ef0 0x7fa2301a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22dd9b640 1 -- 172.21.15.7:0/2667323640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa230100e20 msgr2=0x7fa230199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22dd9b640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa230100e20 0x7fa230199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22dd9b640 1 -- 172.21.15.7:0/2667323640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2301a4950 con 0x7fa230102570 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22ed9d640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa230102ef0 0x7fa2301a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa22dd9b640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa230102570 0x7fa2301998d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa224006010 tx=0x7fa224003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa21f7fe640 1 -- 172.21.15.7:0/2667323640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22403f070 con 0x7fa230102570 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.099+0000 7fa21f7fe640 1 -- 172.21.15.7:0/2667323640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa224031e30 con 0x7fa230102570 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.100+0000 7fa234c85640 1 -- 172.21.15.7:0/2667323640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2301a4b20 con 0x7fa230102570 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.100+0000 7fa21f7fe640 1 -- 172.21.15.7:0/2667323640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224035030 con 0x7fa230102570 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.100+0000 7fa234c85640 1 -- 172.21.15.7:0/2667323640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2301a4fd0 con 0x7fa230102570 2024-09-10T10:45:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.101+0000 7fa234c85640 1 -- 172.21.15.7:0/2667323640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa230101200 con 0x7fa230102570 2024-09-10T10:45:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.102+0000 7fa21f7fe640 1 -- 172.21.15.7:0/2667323640 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa224042070 con 0x7fa230102570 2024-09-10T10:45:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.103+0000 7fa21f7fe640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2180a2c10 0x7fa2180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.103+0000 7fa22e59c640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2180a2c10 0x7fa2180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.103+0000 7fa21f7fe640 1 -- 172.21.15.7:0/2667323640 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fa2240f0160 con 0x7fa230102570 2024-09-10T10:45:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.103+0000 7fa22e59c640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2180a2c10 0x7fa2180a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa22000a400 tx=0x7fa22000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.106+0000 7fa21f7fe640 1 -- 172.21.15.7:0/2667323640 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2240bcfa0 con 0x7fa230102570 2024-09-10T10:45:13.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.290+0000 7fa234c85640 1 -- 172.21.15.7:0/2667323640 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa230109100 con 0x7fa2180a2c10 2024-09-10T10:45:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.297+0000 7fa21f7fe640 1 -- 172.21.15.7:0/2667323640 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258709 (secure 0 0 0) 0x7fa220013340 con 0x7fa2180a2c10 2024-09-10T10:45:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T10:45:13.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 -- 172.21.15.7:0/2667323640 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2180a2c10 msgr2=0x7fa2180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2180a2c10 0x7fa2180a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa22000a400 tx=0x7fa22000d040 comp rx=0 tx=0).stop 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 -- 172.21.15.7:0/2667323640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa230102570 msgr2=0x7fa2301998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa230102570 0x7fa2301998d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa224006010 tx=0x7fa224003c80 comp rx=0 tx=0).stop 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 -- 172.21.15.7:0/2667323640 shutdown_connections 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa2180a2c10 0x7fa2180a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa230102ef0 0x7fa2301a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa230102570 0x7fa2301998d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 --2- 172.21.15.7:0/2667323640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa230100e20 0x7fa230199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 -- 172.21.15.7:0/2667323640 >> 172.21.15.7:0/2667323640 conn(0x7fa2300fc7d0 msgr2=0x7fa2300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 -- 172.21.15.7:0/2667323640 shutdown_connections 2024-09-10T10:45:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:13.300+0000 7fa234c85640 1 -- 172.21.15.7:0/2667323640 wait complete. 2024-09-10T10:45:14.097 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-10T10:45:12.698344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655379,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":450,"num_read_kb":344,"num_write":347,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399152,"kb_used_data":12444,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542672,"statfs":{"total":1151940427776,"available":1151531696128,"internally_reserved":0,"allocated":12742656,"data_stored":7125501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003246"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078402+0000","last_change":"2024-09-10T10:44:11.040610+0000","last_active":"2024-09-10T10:44:22.078402+0000","last_peered":"2024-09-10T10:44:22.078402+0000","last_clean":"2024-09-10T10:44:22.078402+0000","last_became_active":"2024-09-10T10:44:11.040250+0000","last_became_peered":"2024-09-10T10:44:11.040250+0000","last_unstale":"2024-09-10T10:44:22.078402+0000","last_undegraded":"2024-09-10T10:44:22.078402+0000","last_fullsized":"2024-09-10T10:44:22.078402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:25.577181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079163+0000","last_change":"2024-09-10T10:44:11.041944+0000","last_active":"2024-09-10T10:44:22.079163+0000","last_peered":"2024-09-10T10:44:22.079163+0000","last_clean":"2024-09-10T10:44:22.079163+0000","last_became_active":"2024-09-10T10:44:11.041587+0000","last_became_peered":"2024-09-10T10:44:11.041587+0000","last_unstale":"2024-09-10T10:44:22.079163+0000","last_undegraded":"2024-09-10T10:44:22.079163+0000","last_fullsized":"2024-09-10T10:44:22.079163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:50.923879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079144+0000","last_change":"2024-09-10T10:44:11.038730+0000","last_active":"2024-09-10T10:44:22.079144+0000","last_peered":"2024-09-10T10:44:22.079144+0000","last_clean":"2024-09-10T10:44:22.079144+0000","last_became_active":"2024-09-10T10:44:11.038153+0000","last_became_peered":"2024-09-10T10:44:11.038153+0000","last_unstale":"2024-09-10T10:44:22.079144+0000","last_undegraded":"2024-09-10T10:44:22.079144+0000","last_fullsized":"2024-09-10T10:44:22.079144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:47.283582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092089+0000","last_change":"2024-09-10T10:44:11.038739+0000","last_active":"2024-09-10T10:44:22.092089+0000","last_peered":"2024-09-10T10:44:22.092089+0000","last_clean":"2024-09-10T10:44:22.092089+0000","last_became_active":"2024-09-10T10:44:11.038002+0000","last_became_peered":"2024-09-10T10:44:11.038002+0000","last_unstale":"2024-09-10T10:44:22.092089+0000","last_undegraded":"2024-09-10T10:44:22.092089+0000","last_fullsized":"2024-09-10T10:44:22.092089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:40.552127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083116+0000","last_change":"2024-09-10T10:44:11.037026+0000","last_active":"2024-09-10T10:44:22.083116+0000","last_peered":"2024-09-10T10:44:22.083116+0000","last_clean":"2024-09-10T10:44:22.083116+0000","last_became_active":"2024-09-10T10:44:11.036575+0000","last_became_peered":"2024-09-10T10:44:11.036575+0000","last_unstale":"2024-09-10T10:44:22.083116+0000","last_undegraded":"2024-09-10T10:44:22.083116+0000","last_fullsized":"2024-09-10T10:44:22.083116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:38.994882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082953+0000","last_change":"2024-09-10T10:44:11.036454+0000","last_active":"2024-09-10T10:44:22.082953+0000","last_peered":"2024-09-10T10:44:22.082953+0000","last_clean":"2024-09-10T10:44:22.082953+0000","last_became_active":"2024-09-10T10:44:11.034870+0000","last_became_peered":"2024-09-10T10:44:11.034870+0000","last_unstale":"2024-09-10T10:44:22.082953+0000","last_undegraded":"2024-09-10T10:44:22.082953+0000","last_fullsized":"2024-09-10T10:44:22.082953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:09.390465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081505+0000","last_change":"2024-09-10T10:44:11.042061+0000","last_active":"2024-09-10T10:44:22.081505+0000","last_peered":"2024-09-10T10:44:22.081505+0000","last_clean":"2024-09-10T10:44:22.081505+0000","last_became_active":"2024-09-10T10:44:11.041740+0000","last_became_peered":"2024-09-10T10:44:11.041740+0000","last_unstale":"2024-09-10T10:44:22.081505+0000","last_undegraded":"2024-09-10T10:44:22.081505+0000","last_fullsized":"2024-09-10T10:44:22.081505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:27.305586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153429+0000","last_change":"2024-09-10T10:44:11.041426+0000","last_active":"2024-09-10T10:44:22.153429+0000","last_peered":"2024-09-10T10:44:22.153429+0000","last_clean":"2024-09-10T10:44:22.153429+0000","last_became_active":"2024-09-10T10:44:11.041094+0000","last_became_peered":"2024-09-10T10:44:11.041094+0000","last_unstale":"2024-09-10T10:44:22.153429+0000","last_undegraded":"2024-09-10T10:44:22.153429+0000","last_fullsized":"2024-09-10T10:44:22.153429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:48.448394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085711+0000","last_change":"2024-09-10T10:44:11.035830+0000","last_active":"2024-09-10T10:44:22.085711+0000","last_peered":"2024-09-10T10:44:22.085711+0000","last_clean":"2024-09-10T10:44:22.085711+0000","last_became_active":"2024-09-10T10:44:11.034736+0000","last_became_peered":"2024-09-10T10:44:11.034736+0000","last_unstale":"2024-09-10T10:44:22.085711+0000","last_undegraded":"2024-09-10T10:44:22.085711+0000","last_fullsized":"2024-09-10T10:44:22.085711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:25.988368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082569+0000","last_change":"2024-09-10T10:44:11.040292+0000","last_active":"2024-09-10T10:44:22.082569+0000","last_peered":"2024-09-10T10:44:22.082569+0000","last_clean":"2024-09-10T10:44:22.082569+0000","last_became_active":"2024-09-10T10:44:11.039979+0000","last_became_peered":"2024-09-10T10:44:11.039979+0000","last_unstale":"2024-09-10T10:44:22.082569+0000","last_undegraded":"2024-09-10T10:44:22.082569+0000","last_fullsized":"2024-09-10T10:44:22.082569+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:02.539946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082219+0000","last_change":"2024-09-10T10:44:11.043683+0000","last_active":"2024-09-10T10:44:22.082219+0000","last_peered":"2024-09-10T10:44:22.082219+0000","last_clean":"2024-09-10T10:44:22.082219+0000","last_became_active":"2024-09-10T10:44:11.043038+0000","last_became_peered":"2024-09-10T10:44:11.043038+0000","last_unstale":"2024-09-10T10:44:22.082219+0000","last_undegraded":"2024-09-10T10:44:22.082219+0000","last_fullsized":"2024-09-10T10:44:22.082219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:25.548780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092135+0000","last_change":"2024-09-10T10:44:11.041776+0000","last_active":"2024-09-10T10:44:22.092135+0000","last_peered":"2024-09-10T10:44:22.092135+0000","last_clean":"2024-09-10T10:44:22.092135+0000","last_became_active":"2024-09-10T10:44:11.041450+0000","last_became_peered":"2024-09-10T10:44:11.041450+0000","last_unstale":"2024-09-10T10:44:22.092135+0000","last_undegraded":"2024-09-10T10:44:22.092135+0000","last_fullsized":"2024-09-10T10:44:22.092135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:01.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077361+0000","last_change":"2024-09-10T10:44:11.033973+0000","last_active":"2024-09-10T10:44:22.077361+0000","last_peered":"2024-09-10T10:44:22.077361+0000","last_clean":"2024-09-10T10:44:22.077361+0000","last_became_active":"2024-09-10T10:44:11.033649+0000","last_became_peered":"2024-09-10T10:44:11.033649+0000","last_unstale":"2024-09-10T10:44:22.077361+0000","last_undegraded":"2024-09-10T10:44:22.077361+0000","last_fullsized":"2024-09-10T10:44:22.077361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:16.671491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078786+0000","last_change":"2024-09-10T10:44:11.036931+0000","last_active":"2024-09-10T10:44:22.078786+0000","last_peered":"2024-09-10T10:44:22.078786+0000","last_clean":"2024-09-10T10:44:22.078786+0000","last_became_active":"2024-09-10T10:44:11.036574+0000","last_became_peered":"2024-09-10T10:44:11.036574+0000","last_unstale":"2024-09-10T10:44:22.078786+0000","last_undegraded":"2024-09-10T10:44:22.078786+0000","last_fullsized":"2024-09-10T10:44:22.078786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:57.908420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078875+0000","last_change":"2024-09-10T10:44:11.037701+0000","last_active":"2024-09-10T10:44:22.078875+0000","last_peered":"2024-09-10T10:44:22.078875+0000","last_clean":"2024-09-10T10:44:22.078875+0000","last_became_active":"2024-09-10T10:44:11.037417+0000","last_became_peered":"2024-09-10T10:44:11.037417+0000","last_unstale":"2024-09-10T10:44:22.078875+0000","last_undegraded":"2024-09-10T10:44:22.078875+0000","last_fullsized":"2024-09-10T10:44:22.078875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:35.584226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084980+0000","last_change":"2024-09-10T10:44:11.038545+0000","last_active":"2024-09-10T10:44:22.084980+0000","last_peered":"2024-09-10T10:44:22.084980+0000","last_clean":"2024-09-10T10:44:22.084980+0000","last_became_active":"2024-09-10T10:44:11.037574+0000","last_became_peered":"2024-09-10T10:44:11.037574+0000","last_unstale":"2024-09-10T10:44:22.084980+0000","last_undegraded":"2024-09-10T10:44:22.084980+0000","last_fullsized":"2024-09-10T10:44:22.084980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:13.471862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078510+0000","last_change":"2024-09-10T10:44:11.042231+0000","last_active":"2024-09-10T10:44:22.078510+0000","last_peered":"2024-09-10T10:44:22.078510+0000","last_clean":"2024-09-10T10:44:22.078510+0000","last_became_active":"2024-09-10T10:44:11.041764+0000","last_became_peered":"2024-09-10T10:44:11.041764+0000","last_unstale":"2024-09-10T10:44:22.078510+0000","last_undegraded":"2024-09-10T10:44:22.078510+0000","last_fullsized":"2024-09-10T10:44:22.078510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:38.301271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080361+0000","last_change":"2024-09-10T10:44:11.044492+0000","last_active":"2024-09-10T10:44:22.080361+0000","last_peered":"2024-09-10T10:44:22.080361+0000","last_clean":"2024-09-10T10:44:22.080361+0000","last_became_active":"2024-09-10T10:44:11.044135+0000","last_became_peered":"2024-09-10T10:44:11.044135+0000","last_unstale":"2024-09-10T10:44:22.080361+0000","last_undegraded":"2024-09-10T10:44:22.080361+0000","last_fullsized":"2024-09-10T10:44:22.080361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:47.054076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081417+0000","last_change":"2024-09-10T10:44:11.040349+0000","last_active":"2024-09-10T10:44:22.081417+0000","last_peered":"2024-09-10T10:44:22.081417+0000","last_clean":"2024-09-10T10:44:22.081417+0000","last_became_active":"2024-09-10T10:44:11.040051+0000","last_became_peered":"2024-09-10T10:44:11.040051+0000","last_unstale":"2024-09-10T10:44:22.081417+0000","last_undegraded":"2024-09-10T10:44:22.081417+0000","last_fullsized":"2024-09-10T10:44:22.081417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:33.777529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085576+0000","last_change":"2024-09-10T10:44:22.085576+0000","last_active":"2024-09-10T10:44:22.085576+0000","last_peered":"2024-09-10T10:44:22.085576+0000","last_clean":"2024-09-10T10:44:22.085576+0000","last_became_active":"2024-09-10T10:44:22.085135+0000","last_became_peered":"2024-09-10T10:44:22.085135+0000","last_unstale":"2024-09-10T10:44:22.085576+0000","last_undegraded":"2024-09-10T10:44:22.085576+0000","last_fullsized":"2024-09-10T10:44:22.085576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077458+0000","last_change":"2024-09-10T10:44:11.030324+0000","last_active":"2024-09-10T10:44:22.077458+0000","last_peered":"2024-09-10T10:44:22.077458+0000","last_clean":"2024-09-10T10:44:22.077458+0000","last_became_active":"2024-09-10T10:44:11.029652+0000","last_became_peered":"2024-09-10T10:44:11.029652+0000","last_unstale":"2024-09-10T10:44:22.077458+0000","last_undegraded":"2024-09-10T10:44:22.077458+0000","last_fullsized":"2024-09-10T10:44:22.077458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:51.332548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085112+0000","last_change":"2024-09-10T10:44:11.040637+0000","last_active":"2024-09-10T10:44:22.085112+0000","last_peered":"2024-09-10T10:44:22.085112+0000","last_clean":"2024-09-10T10:44:22.085112+0000","last_became_active":"2024-09-10T10:44:11.040345+0000","last_became_peered":"2024-09-10T10:44:11.040345+0000","last_unstale":"2024-09-10T10:44:22.085112+0000","last_undegraded":"2024-09-10T10:44:22.085112+0000","last_fullsized":"2024-09-10T10:44:22.085112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:18.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153476+0000","last_change":"2024-09-10T10:44:11.043157+0000","last_active":"2024-09-10T10:44:22.153476+0000","last_peered":"2024-09-10T10:44:22.153476+0000","last_clean":"2024-09-10T10:44:22.153476+0000","last_became_active":"2024-09-10T10:44:11.042823+0000","last_became_peered":"2024-09-10T10:44:11.042823+0000","last_unstale":"2024-09-10T10:44:22.153476+0000","last_undegraded":"2024-09-10T10:44:22.153476+0000","last_fullsized":"2024-09-10T10:44:22.153476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:23.988072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080356+0000","last_change":"2024-09-10T10:44:11.041918+0000","last_active":"2024-09-10T10:44:22.080356+0000","last_peered":"2024-09-10T10:44:22.080356+0000","last_clean":"2024-09-10T10:44:22.080356+0000","last_became_active":"2024-09-10T10:44:11.040111+0000","last_became_peered":"2024-09-10T10:44:11.040111+0000","last_unstale":"2024-09-10T10:44:22.080356+0000","last_undegraded":"2024-09-10T10:44:22.080356+0000","last_fullsized":"2024-09-10T10:44:22.080356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:24.999485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092270+0000","last_change":"2024-09-10T10:44:22.091959+0000","last_active":"2024-09-10T10:44:22.092270+0000","last_peered":"2024-09-10T10:44:22.092270+0000","last_clean":"2024-09-10T10:44:22.092270+0000","last_became_active":"2024-09-10T10:44:22.091707+0000","last_became_peered":"2024-09-10T10:44:22.091707+0000","last_unstale":"2024-09-10T10:44:22.092270+0000","last_undegraded":"2024-09-10T10:44:22.092270+0000","last_fullsized":"2024-09-10T10:44:22.092270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:38.454868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081432+0000","last_change":"2024-09-10T10:44:11.034339+0000","last_active":"2024-09-10T10:44:22.081432+0000","last_peered":"2024-09-10T10:44:22.081432+0000","last_clean":"2024-09-10T10:44:22.081432+0000","last_became_active":"2024-09-10T10:44:11.034022+0000","last_became_peered":"2024-09-10T10:44:11.034022+0000","last_unstale":"2024-09-10T10:44:22.081432+0000","last_undegraded":"2024-09-10T10:44:22.081432+0000","last_fullsized":"2024-09-10T10:44:22.081432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:54.122244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078068+0000","last_change":"2024-09-10T10:44:11.037871+0000","last_active":"2024-09-10T10:44:22.078068+0000","last_peered":"2024-09-10T10:44:22.078068+0000","last_clean":"2024-09-10T10:44:22.078068+0000","last_became_active":"2024-09-10T10:44:11.037372+0000","last_became_peered":"2024-09-10T10:44:11.037372+0000","last_unstale":"2024-09-10T10:44:22.078068+0000","last_undegraded":"2024-09-10T10:44:22.078068+0000","last_fullsized":"2024-09-10T10:44:22.078068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:22.296078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077383+0000","last_change":"2024-09-10T10:44:11.034826+0000","last_active":"2024-09-10T10:44:22.077383+0000","last_peered":"2024-09-10T10:44:22.077383+0000","last_clean":"2024-09-10T10:44:22.077383+0000","last_became_active":"2024-09-10T10:44:11.034520+0000","last_became_peered":"2024-09-10T10:44:11.034520+0000","last_unstale":"2024-09-10T10:44:22.077383+0000","last_undegraded":"2024-09-10T10:44:22.077383+0000","last_fullsized":"2024-09-10T10:44:22.077383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:13.131060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085136+0000","last_change":"2024-09-10T10:44:22.085136+0000","last_active":"2024-09-10T10:44:22.085136+0000","last_peered":"2024-09-10T10:44:22.085136+0000","last_clean":"2024-09-10T10:44:22.085136+0000","last_became_active":"2024-09-10T10:44:22.084718+0000","last_became_peered":"2024-09-10T10:44:22.084718+0000","last_unstale":"2024-09-10T10:44:22.085136+0000","last_undegraded":"2024-09-10T10:44:22.085136+0000","last_fullsized":"2024-09-10T10:44:22.085136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:31.165424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081062+0000","last_change":"2024-09-10T10:44:11.042336+0000","last_active":"2024-09-10T10:44:22.081062+0000","last_peered":"2024-09-10T10:44:22.081062+0000","last_clean":"2024-09-10T10:44:22.081062+0000","last_became_active":"2024-09-10T10:44:11.041127+0000","last_became_peered":"2024-09-10T10:44:11.041127+0000","last_unstale":"2024-09-10T10:44:22.081062+0000","last_undegraded":"2024-09-10T10:44:22.081062+0000","last_fullsized":"2024-09-10T10:44:22.081062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:50.349201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081158+0000","last_change":"2024-09-10T10:44:11.043188+0000","last_active":"2024-09-10T10:44:22.081158+0000","last_peered":"2024-09-10T10:44:22.081158+0000","last_clean":"2024-09-10T10:44:22.081158+0000","last_became_active":"2024-09-10T10:44:11.042735+0000","last_became_peered":"2024-09-10T10:44:11.042735+0000","last_unstale":"2024-09-10T10:44:22.081158+0000","last_undegraded":"2024-09-10T10:44:22.081158+0000","last_fullsized":"2024-09-10T10:44:22.081158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:50.090810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077961+0000","last_change":"2024-09-10T10:44:11.041339+0000","last_active":"2024-09-10T10:44:22.077961+0000","last_peered":"2024-09-10T10:44:22.077961+0000","last_clean":"2024-09-10T10:44:22.077961+0000","last_became_active":"2024-09-10T10:44:11.038011+0000","last_became_peered":"2024-09-10T10:44:11.038011+0000","last_unstale":"2024-09-10T10:44:22.077961+0000","last_undegraded":"2024-09-10T10:44:22.077961+0000","last_fullsized":"2024-09-10T10:44:22.077961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:47.521465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084488+0000","last_change":"2024-09-10T10:44:22.084488+0000","last_active":"2024-09-10T10:44:22.084488+0000","last_peered":"2024-09-10T10:44:22.084488+0000","last_clean":"2024-09-10T10:44:22.084488+0000","last_became_active":"2024-09-10T10:44:22.084062+0000","last_became_peered":"2024-09-10T10:44:22.084062+0000","last_unstale":"2024-09-10T10:44:22.084488+0000","last_undegraded":"2024-09-10T10:44:22.084488+0000","last_fullsized":"2024-09-10T10:44:22.084488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:57.739261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":77,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:45:07.154564+0000","last_change":"2024-09-10T10:44:09.024716+0000","last_active":"2024-09-10T10:45:07.154564+0000","last_peered":"2024-09-10T10:45:07.154564+0000","last_clean":"2024-09-10T10:45:07.154564+0000","last_became_active":"2024-09-10T10:44:09.023901+0000","last_became_peered":"2024-09-10T10:44:09.023901+0000","last_unstale":"2024-09-10T10:45:07.154564+0000","last_undegraded":"2024-09-10T10:45:07.154564+0000","last_fullsized":"2024-09-10T10:45:07.154564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:28.901896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080568+0000","last_change":"2024-09-10T10:44:11.044838+0000","last_active":"2024-09-10T10:44:22.080568+0000","last_peered":"2024-09-10T10:44:22.080568+0000","last_clean":"2024-09-10T10:44:22.080568+0000","last_became_active":"2024-09-10T10:44:11.044542+0000","last_became_peered":"2024-09-10T10:44:11.044542+0000","last_unstale":"2024-09-10T10:44:22.080568+0000","last_undegraded":"2024-09-10T10:44:22.080568+0000","last_fullsized":"2024-09-10T10:44:22.080568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:02.753914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082979+0000","last_change":"2024-09-10T10:44:09.025799+0000","last_active":"2024-09-10T10:44:22.082979+0000","last_peered":"2024-09-10T10:44:22.082979+0000","last_clean":"2024-09-10T10:44:22.082979+0000","last_became_active":"2024-09-10T10:44:09.025421+0000","last_became_peered":"2024-09-10T10:44:09.025421+0000","last_unstale":"2024-09-10T10:44:22.082979+0000","last_undegraded":"2024-09-10T10:44:22.082979+0000","last_fullsized":"2024-09-10T10:44:22.082979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:11.928663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077144+0000","last_change":"2024-09-10T10:44:09.017640+0000","last_active":"2024-09-10T10:44:22.077144+0000","last_peered":"2024-09-10T10:44:22.077144+0000","last_clean":"2024-09-10T10:44:22.077144+0000","last_became_active":"2024-09-10T10:44:09.017286+0000","last_became_peered":"2024-09-10T10:44:09.017286+0000","last_unstale":"2024-09-10T10:44:22.077144+0000","last_undegraded":"2024-09-10T10:44:22.077144+0000","last_fullsized":"2024-09-10T10:44:22.077144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:59.678607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153461+0000","last_change":"2024-09-10T10:44:11.038225+0000","last_active":"2024-09-10T10:44:22.153461+0000","last_peered":"2024-09-10T10:44:22.153461+0000","last_clean":"2024-09-10T10:44:22.153461+0000","last_became_active":"2024-09-10T10:44:11.037924+0000","last_became_peered":"2024-09-10T10:44:11.037924+0000","last_unstale":"2024-09-10T10:44:22.153461+0000","last_undegraded":"2024-09-10T10:44:22.153461+0000","last_fullsized":"2024-09-10T10:44:22.153461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:58.787166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080948+0000","last_change":"2024-09-10T10:44:09.024676+0000","last_active":"2024-09-10T10:44:22.080948+0000","last_peered":"2024-09-10T10:44:22.080948+0000","last_clean":"2024-09-10T10:44:22.080948+0000","last_became_active":"2024-09-10T10:44:09.023970+0000","last_became_peered":"2024-09-10T10:44:09.023970+0000","last_unstale":"2024-09-10T10:44:22.080948+0000","last_undegraded":"2024-09-10T10:44:22.080948+0000","last_fullsized":"2024-09-10T10:44:22.080948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:40.296260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080161+0000","last_change":"2024-09-10T10:44:11.040330+0000","last_active":"2024-09-10T10:44:22.080161+0000","last_peered":"2024-09-10T10:44:22.080161+0000","last_clean":"2024-09-10T10:44:22.080161+0000","last_became_active":"2024-09-10T10:44:11.040018+0000","last_became_peered":"2024-09-10T10:44:11.040018+0000","last_unstale":"2024-09-10T10:44:22.080161+0000","last_undegraded":"2024-09-10T10:44:22.080161+0000","last_fullsized":"2024-09-10T10:44:22.080161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:24.403070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":73,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:45:07.153989+0000","last_change":"2024-09-10T10:44:09.021488+0000","last_active":"2024-09-10T10:45:07.153989+0000","last_peered":"2024-09-10T10:45:07.153989+0000","last_clean":"2024-09-10T10:45:07.153989+0000","last_became_active":"2024-09-10T10:44:09.021088+0000","last_became_peered":"2024-09-10T10:44:09.021088+0000","last_unstale":"2024-09-10T10:45:07.153989+0000","last_undegraded":"2024-09-10T10:45:07.153989+0000","last_fullsized":"2024-09-10T10:45:07.153989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:37.990090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085498+0000","last_change":"2024-09-10T10:44:11.045142+0000","last_active":"2024-09-10T10:44:22.085498+0000","last_peered":"2024-09-10T10:44:22.085498+0000","last_clean":"2024-09-10T10:44:22.085498+0000","last_became_active":"2024-09-10T10:44:11.044770+0000","last_became_peered":"2024-09-10T10:44:11.044770+0000","last_unstale":"2024-09-10T10:44:22.085498+0000","last_undegraded":"2024-09-10T10:44:22.085498+0000","last_fullsized":"2024-09-10T10:44:22.085498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:10.731504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.096348+0000","last_change":"2024-09-10T10:44:22.096348+0000","last_active":"2024-09-10T10:44:22.096348+0000","last_peered":"2024-09-10T10:44:22.096348+0000","last_clean":"2024-09-10T10:44:22.096348+0000","last_became_active":"2024-09-10T10:44:22.092921+0000","last_became_peered":"2024-09-10T10:44:22.092921+0000","last_unstale":"2024-09-10T10:44:22.096348+0000","last_undegraded":"2024-09-10T10:44:22.096348+0000","last_fullsized":"2024-09-10T10:44:22.096348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:15.721849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079428+0000","last_change":"2024-09-10T10:44:11.034499+0000","last_active":"2024-09-10T10:44:22.079428+0000","last_peered":"2024-09-10T10:44:22.079428+0000","last_clean":"2024-09-10T10:44:22.079428+0000","last_became_active":"2024-09-10T10:44:11.034030+0000","last_became_peered":"2024-09-10T10:44:11.034030+0000","last_unstale":"2024-09-10T10:44:22.079428+0000","last_undegraded":"2024-09-10T10:44:22.079428+0000","last_fullsized":"2024-09-10T10:44:22.079428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:57.313000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080739+0000","last_change":"2024-09-10T10:44:09.142509+0000","last_active":"2024-09-10T10:44:22.080739+0000","last_peered":"2024-09-10T10:44:22.080739+0000","last_clean":"2024-09-10T10:44:22.080739+0000","last_became_active":"2024-09-10T10:44:09.142048+0000","last_became_peered":"2024-09-10T10:44:09.142048+0000","last_unstale":"2024-09-10T10:44:22.080739+0000","last_undegraded":"2024-09-10T10:44:22.080739+0000","last_fullsized":"2024-09-10T10:44:22.080739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:57.937241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085180+0000","last_change":"2024-09-10T10:44:11.034851+0000","last_active":"2024-09-10T10:44:22.085180+0000","last_peered":"2024-09-10T10:44:22.085180+0000","last_clean":"2024-09-10T10:44:22.085180+0000","last_became_active":"2024-09-10T10:44:11.034524+0000","last_became_peered":"2024-09-10T10:44:11.034524+0000","last_unstale":"2024-09-10T10:44:22.085180+0000","last_undegraded":"2024-09-10T10:44:22.085180+0000","last_fullsized":"2024-09-10T10:44:22.085180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:06.356958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":154,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:45:08.166744+0000","last_change":"2024-09-10T10:44:09.024313+0000","last_active":"2024-09-10T10:45:08.166744+0000","last_peered":"2024-09-10T10:45:08.166744+0000","last_clean":"2024-09-10T10:45:08.166744+0000","last_became_active":"2024-09-10T10:44:09.023933+0000","last_became_peered":"2024-09-10T10:44:09.023933+0000","last_unstale":"2024-09-10T10:45:08.166744+0000","last_undegraded":"2024-09-10T10:45:08.166744+0000","last_fullsized":"2024-09-10T10:45:08.166744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:37.522786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61167,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":103,"num_read_kb":110,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083275+0000","last_change":"2024-09-10T10:44:11.039171+0000","last_active":"2024-09-10T10:44:22.083275+0000","last_peered":"2024-09-10T10:44:22.083275+0000","last_clean":"2024-09-10T10:44:22.083275+0000","last_became_active":"2024-09-10T10:44:11.038833+0000","last_became_peered":"2024-09-10T10:44:11.038833+0000","last_unstale":"2024-09-10T10:44:22.083275+0000","last_undegraded":"2024-09-10T10:44:22.083275+0000","last_fullsized":"2024-09-10T10:44:22.083275+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:37.115626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081645+0000","last_change":"2024-09-10T10:44:09.021368+0000","last_active":"2024-09-10T10:44:22.081645+0000","last_peered":"2024-09-10T10:44:22.081645+0000","last_clean":"2024-09-10T10:44:22.081645+0000","last_became_active":"2024-09-10T10:44:09.020805+0000","last_became_peered":"2024-09-10T10:44:09.020805+0000","last_unstale":"2024-09-10T10:44:22.081645+0000","last_undegraded":"2024-09-10T10:44:22.081645+0000","last_fullsized":"2024-09-10T10:44:22.081645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:20.408776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077226+0000","last_change":"2024-09-10T10:44:11.037539+0000","last_active":"2024-09-10T10:44:22.077226+0000","last_peered":"2024-09-10T10:44:22.077226+0000","last_clean":"2024-09-10T10:44:22.077226+0000","last_became_active":"2024-09-10T10:44:11.036607+0000","last_became_peered":"2024-09-10T10:44:11.036607+0000","last_unstale":"2024-09-10T10:44:22.077226+0000","last_undegraded":"2024-09-10T10:44:22.077226+0000","last_fullsized":"2024-09-10T10:44:22.077226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:01:46.029665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080495+0000","last_change":"2024-09-10T10:44:09.023766+0000","last_active":"2024-09-10T10:44:22.080495+0000","last_peered":"2024-09-10T10:44:22.080495+0000","last_clean":"2024-09-10T10:44:22.080495+0000","last_became_active":"2024-09-10T10:44:09.023422+0000","last_became_peered":"2024-09-10T10:44:09.023422+0000","last_unstale":"2024-09-10T10:44:22.080495+0000","last_undegraded":"2024-09-10T10:44:22.080495+0000","last_fullsized":"2024-09-10T10:44:22.080495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:56.009610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082817+0000","last_change":"2024-09-10T10:44:11.038846+0000","last_active":"2024-09-10T10:44:22.082817+0000","last_peered":"2024-09-10T10:44:22.082817+0000","last_clean":"2024-09-10T10:44:22.082817+0000","last_became_active":"2024-09-10T10:44:11.038477+0000","last_became_peered":"2024-09-10T10:44:11.038477+0000","last_unstale":"2024-09-10T10:44:22.082817+0000","last_undegraded":"2024-09-10T10:44:22.082817+0000","last_fullsized":"2024-09-10T10:44:22.082817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:26.255612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'3","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:45:07.161116+0000","last_change":"2024-09-10T10:44:09.023394+0000","last_active":"2024-09-10T10:45:07.161116+0000","last_peered":"2024-09-10T10:45:07.161116+0000","last_clean":"2024-09-10T10:45:07.161116+0000","last_became_active":"2024-09-10T10:44:09.023049+0000","last_became_peered":"2024-09-10T10:44:09.023049+0000","last_unstale":"2024-09-10T10:45:07.161116+0000","last_undegraded":"2024-09-10T10:45:07.161116+0000","last_fullsized":"2024-09-10T10:45:07.161116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:00.024671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081712+0000","last_change":"2024-09-10T10:44:11.041162+0000","last_active":"2024-09-10T10:44:22.081712+0000","last_peered":"2024-09-10T10:44:22.081712+0000","last_clean":"2024-09-10T10:44:22.081712+0000","last_became_active":"2024-09-10T10:44:11.040848+0000","last_became_peered":"2024-09-10T10:44:11.040848+0000","last_unstale":"2024-09-10T10:44:22.081712+0000","last_undegraded":"2024-09-10T10:44:22.081712+0000","last_fullsized":"2024-09-10T10:44:22.081712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:13.044447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082834+0000","last_change":"2024-09-10T10:44:09.142490+0000","last_active":"2024-09-10T10:44:22.082834+0000","last_peered":"2024-09-10T10:44:22.082834+0000","last_clean":"2024-09-10T10:44:22.082834+0000","last_became_active":"2024-09-10T10:44:09.141983+0000","last_became_peered":"2024-09-10T10:44:09.141983+0000","last_unstale":"2024-09-10T10:44:22.082834+0000","last_undegraded":"2024-09-10T10:44:22.082834+0000","last_fullsized":"2024-09-10T10:44:22.082834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:34.481241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080333+0000","last_change":"2024-09-10T10:44:11.042109+0000","last_active":"2024-09-10T10:44:22.080333+0000","last_peered":"2024-09-10T10:44:22.080333+0000","last_clean":"2024-09-10T10:44:22.080333+0000","last_became_active":"2024-09-10T10:44:11.041154+0000","last_became_peered":"2024-09-10T10:44:11.041154+0000","last_unstale":"2024-09-10T10:44:22.080333+0000","last_undegraded":"2024-09-10T10:44:22.080333+0000","last_fullsized":"2024-09-10T10:44:22.080333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:50.134773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083174+0000","last_change":"2024-09-10T10:44:09.019117+0000","last_active":"2024-09-10T10:44:22.083174+0000","last_peered":"2024-09-10T10:44:22.083174+0000","last_clean":"2024-09-10T10:44:22.083174+0000","last_became_active":"2024-09-10T10:44:09.018733+0000","last_became_peered":"2024-09-10T10:44:09.018733+0000","last_unstale":"2024-09-10T10:44:22.083174+0000","last_undegraded":"2024-09-10T10:44:22.083174+0000","last_fullsized":"2024-09-10T10:44:22.083174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:40.049342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078300+0000","last_change":"2024-09-10T10:44:11.040381+0000","last_active":"2024-09-10T10:44:22.078300+0000","last_peered":"2024-09-10T10:44:22.078300+0000","last_clean":"2024-09-10T10:44:22.078300+0000","last_became_active":"2024-09-10T10:44:11.040053+0000","last_became_peered":"2024-09-10T10:44:11.040053+0000","last_unstale":"2024-09-10T10:44:22.078300+0000","last_undegraded":"2024-09-10T10:44:22.078300+0000","last_fullsized":"2024-09-10T10:44:22.078300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:29.043171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077403+0000","last_change":"2024-09-10T10:44:09.018173+0000","last_active":"2024-09-10T10:44:22.077403+0000","last_peered":"2024-09-10T10:44:22.077403+0000","last_clean":"2024-09-10T10:44:22.077403+0000","last_became_active":"2024-09-10T10:44:09.017790+0000","last_became_peered":"2024-09-10T10:44:09.017790+0000","last_unstale":"2024-09-10T10:44:22.077403+0000","last_undegraded":"2024-09-10T10:44:22.077403+0000","last_fullsized":"2024-09-10T10:44:22.077403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:43.515112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082583+0000","last_change":"2024-09-10T10:44:11.035293+0000","last_active":"2024-09-10T10:44:22.082583+0000","last_peered":"2024-09-10T10:44:22.082583+0000","last_clean":"2024-09-10T10:44:22.082583+0000","last_became_active":"2024-09-10T10:44:11.034958+0000","last_became_peered":"2024-09-10T10:44:11.034958+0000","last_unstale":"2024-09-10T10:44:22.082583+0000","last_undegraded":"2024-09-10T10:44:22.082583+0000","last_fullsized":"2024-09-10T10:44:22.082583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:09.397229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080789+0000","last_change":"2024-09-10T10:44:09.021322+0000","last_active":"2024-09-10T10:44:22.080789+0000","last_peered":"2024-09-10T10:44:22.080789+0000","last_clean":"2024-09-10T10:44:22.080789+0000","last_became_active":"2024-09-10T10:44:09.021018+0000","last_became_peered":"2024-09-10T10:44:09.021018+0000","last_unstale":"2024-09-10T10:44:22.080789+0000","last_undegraded":"2024-09-10T10:44:22.080789+0000","last_fullsized":"2024-09-10T10:44:22.080789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:48.999125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082090+0000","last_change":"2024-09-10T10:44:11.040545+0000","last_active":"2024-09-10T10:44:22.082090+0000","last_peered":"2024-09-10T10:44:22.082090+0000","last_clean":"2024-09-10T10:44:22.082090+0000","last_became_active":"2024-09-10T10:44:11.040042+0000","last_became_peered":"2024-09-10T10:44:11.040042+0000","last_unstale":"2024-09-10T10:44:22.082090+0000","last_undegraded":"2024-09-10T10:44:22.082090+0000","last_fullsized":"2024-09-10T10:44:22.082090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:14.367771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081581+0000","last_change":"2024-09-10T10:44:09.142944+0000","last_active":"2024-09-10T10:44:22.081581+0000","last_peered":"2024-09-10T10:44:22.081581+0000","last_clean":"2024-09-10T10:44:22.081581+0000","last_became_active":"2024-09-10T10:44:09.142550+0000","last_became_peered":"2024-09-10T10:44:09.142550+0000","last_unstale":"2024-09-10T10:44:22.081581+0000","last_undegraded":"2024-09-10T10:44:22.081581+0000","last_fullsized":"2024-09-10T10:44:22.081581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:56.247251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082331+0000","last_change":"2024-09-10T10:44:11.038205+0000","last_active":"2024-09-10T10:44:22.082331+0000","last_peered":"2024-09-10T10:44:22.082331+0000","last_clean":"2024-09-10T10:44:22.082331+0000","last_became_active":"2024-09-10T10:44:11.037817+0000","last_became_peered":"2024-09-10T10:44:11.037817+0000","last_unstale":"2024-09-10T10:44:22.082331+0000","last_undegraded":"2024-09-10T10:44:22.082331+0000","last_fullsized":"2024-09-10T10:44:22.082331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:39.221897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":343,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081904+0000","last_change":"2024-09-10T10:43:01.771132+0000","last_active":"2024-09-10T10:44:22.081904+0000","last_peered":"2024-09-10T10:44:22.081904+0000","last_clean":"2024-09-10T10:44:22.081904+0000","last_became_active":"2024-09-10T10:43:01.770568+0000","last_became_peered":"2024-09-10T10:43:01.770568+0000","last_unstale":"2024-09-10T10:44:22.081904+0000","last_undegraded":"2024-09-10T10:44:22.081904+0000","last_fullsized":"2024-09-10T10:44:22.081904+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:39:37.547474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:39:37.547474+0000","last_clean_scrub_stamp":"2024-09-10T10:39:37.547474+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:09.699899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.091961+0000","last_change":"2024-09-10T10:44:09.021115+0000","last_active":"2024-09-10T10:44:22.091961+0000","last_peered":"2024-09-10T10:44:22.091961+0000","last_clean":"2024-09-10T10:44:22.091961+0000","last_became_active":"2024-09-10T10:44:09.020747+0000","last_became_peered":"2024-09-10T10:44:09.020747+0000","last_unstale":"2024-09-10T10:44:22.091961+0000","last_undegraded":"2024-09-10T10:44:22.091961+0000","last_fullsized":"2024-09-10T10:44:22.091961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:15.828535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153652+0000","last_change":"2024-09-10T10:44:11.046484+0000","last_active":"2024-09-10T10:44:22.153652+0000","last_peered":"2024-09-10T10:44:22.153652+0000","last_clean":"2024-09-10T10:44:22.153652+0000","last_became_active":"2024-09-10T10:44:11.046142+0000","last_became_peered":"2024-09-10T10:44:11.046142+0000","last_unstale":"2024-09-10T10:44:22.153652+0000","last_undegraded":"2024-09-10T10:44:22.153652+0000","last_fullsized":"2024-09-10T10:44:22.153652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:15.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.088392+0000","last_change":"2024-09-10T10:44:22.087926+0000","last_active":"2024-09-10T10:44:22.088392+0000","last_peered":"2024-09-10T10:44:22.088392+0000","last_clean":"2024-09-10T10:44:22.088392+0000","last_became_active":"2024-09-10T10:44:22.084248+0000","last_became_peered":"2024-09-10T10:44:22.084248+0000","last_unstale":"2024-09-10T10:44:22.088392+0000","last_undegraded":"2024-09-10T10:44:22.088392+0000","last_fullsized":"2024-09-10T10:44:22.088392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:15.774609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077128+0000","last_change":"2024-09-10T10:44:11.034105+0000","last_active":"2024-09-10T10:44:22.077128+0000","last_peered":"2024-09-10T10:44:22.077128+0000","last_clean":"2024-09-10T10:44:22.077128+0000","last_became_active":"2024-09-10T10:44:11.033792+0000","last_became_peered":"2024-09-10T10:44:11.033792+0000","last_unstale":"2024-09-10T10:44:22.077128+0000","last_undegraded":"2024-09-10T10:44:22.077128+0000","last_fullsized":"2024-09-10T10:44:22.077128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.131963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:45:08.161859+0000","last_change":"2024-09-10T10:44:09.142514+0000","last_active":"2024-09-10T10:45:08.161859+0000","last_peered":"2024-09-10T10:45:08.161859+0000","last_clean":"2024-09-10T10:45:08.161859+0000","last_became_active":"2024-09-10T10:44:09.142115+0000","last_became_peered":"2024-09-10T10:44:09.142115+0000","last_unstale":"2024-09-10T10:45:08.161859+0000","last_undegraded":"2024-09-10T10:45:08.161859+0000","last_fullsized":"2024-09-10T10:45:08.161859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:37.520105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080531+0000","last_change":"2024-09-10T10:44:11.040122+0000","last_active":"2024-09-10T10:44:22.080531+0000","last_peered":"2024-09-10T10:44:22.080531+0000","last_clean":"2024-09-10T10:44:22.080531+0000","last_became_active":"2024-09-10T10:44:11.039649+0000","last_became_peered":"2024-09-10T10:44:11.039649+0000","last_unstale":"2024-09-10T10:44:22.080531+0000","last_undegraded":"2024-09-10T10:44:22.080531+0000","last_fullsized":"2024-09-10T10:44:22.080531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:26.401981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083151+0000","last_change":"2024-09-10T10:44:09.024933+0000","last_active":"2024-09-10T10:44:22.083151+0000","last_peered":"2024-09-10T10:44:22.083151+0000","last_clean":"2024-09-10T10:44:22.083151+0000","last_became_active":"2024-09-10T10:44:09.024579+0000","last_became_peered":"2024-09-10T10:44:09.024579+0000","last_unstale":"2024-09-10T10:44:22.083151+0000","last_undegraded":"2024-09-10T10:44:22.083151+0000","last_fullsized":"2024-09-10T10:44:22.083151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:45.644932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084217+0000","last_change":"2024-09-10T10:44:22.084217+0000","last_active":"2024-09-10T10:44:22.084217+0000","last_peered":"2024-09-10T10:44:22.084217+0000","last_clean":"2024-09-10T10:44:22.084217+0000","last_became_active":"2024-09-10T10:44:22.083707+0000","last_became_peered":"2024-09-10T10:44:22.083707+0000","last_unstale":"2024-09-10T10:44:22.084217+0000","last_undegraded":"2024-09-10T10:44:22.084217+0000","last_fullsized":"2024-09-10T10:44:22.084217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:30.124690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080402+0000","last_change":"2024-09-10T10:44:09.142462+0000","last_active":"2024-09-10T10:44:22.080402+0000","last_peered":"2024-09-10T10:44:22.080402+0000","last_clean":"2024-09-10T10:44:22.080402+0000","last_became_active":"2024-09-10T10:44:09.142086+0000","last_became_peered":"2024-09-10T10:44:09.142086+0000","last_unstale":"2024-09-10T10:44:22.080402+0000","last_undegraded":"2024-09-10T10:44:22.080402+0000","last_fullsized":"2024-09-10T10:44:22.080402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:28.019899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082676+0000","last_change":"2024-09-10T10:44:11.036928+0000","last_active":"2024-09-10T10:44:22.082676+0000","last_peered":"2024-09-10T10:44:22.082676+0000","last_clean":"2024-09-10T10:44:22.082676+0000","last_became_active":"2024-09-10T10:44:11.036626+0000","last_became_peered":"2024-09-10T10:44:11.036626+0000","last_unstale":"2024-09-10T10:44:22.082676+0000","last_undegraded":"2024-09-10T10:44:22.082676+0000","last_fullsized":"2024-09-10T10:44:22.082676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:42.624867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083407+0000","last_change":"2024-09-10T10:44:09.142521+0000","last_active":"2024-09-10T10:44:22.083407+0000","last_peered":"2024-09-10T10:44:22.083407+0000","last_clean":"2024-09-10T10:44:22.083407+0000","last_became_active":"2024-09-10T10:44:09.142061+0000","last_became_peered":"2024-09-10T10:44:09.142061+0000","last_unstale":"2024-09-10T10:44:22.083407+0000","last_undegraded":"2024-09-10T10:44:22.083407+0000","last_fullsized":"2024-09-10T10:44:22.083407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:14.054388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077489+0000","last_change":"2024-09-10T10:44:11.031953+0000","last_active":"2024-09-10T10:44:22.077489+0000","last_peered":"2024-09-10T10:44:22.077489+0000","last_clean":"2024-09-10T10:44:22.077489+0000","last_became_active":"2024-09-10T10:44:11.031610+0000","last_became_peered":"2024-09-10T10:44:11.031610+0000","last_unstale":"2024-09-10T10:44:22.077489+0000","last_undegraded":"2024-09-10T10:44:22.077489+0000","last_fullsized":"2024-09-10T10:44:22.077489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:03.305651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:35.030830+0000","last_change":"2024-09-10T10:44:09.016752+0000","last_active":"2024-09-10T10:44:35.030830+0000","last_peered":"2024-09-10T10:44:35.030830+0000","last_clean":"2024-09-10T10:44:35.030830+0000","last_became_active":"2024-09-10T10:44:09.016464+0000","last_became_peered":"2024-09-10T10:44:09.016464+0000","last_unstale":"2024-09-10T10:44:35.030830+0000","last_undegraded":"2024-09-10T10:44:35.030830+0000","last_fullsized":"2024-09-10T10:44:35.030830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:25.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153747+0000","last_change":"2024-09-10T10:44:11.044793+0000","last_active":"2024-09-10T10:44:22.153747+0000","last_peered":"2024-09-10T10:44:22.153747+0000","last_clean":"2024-09-10T10:44:22.153747+0000","last_became_active":"2024-09-10T10:44:11.043611+0000","last_became_peered":"2024-09-10T10:44:11.043611+0000","last_unstale":"2024-09-10T10:44:22.153747+0000","last_undegraded":"2024-09-10T10:44:22.153747+0000","last_fullsized":"2024-09-10T10:44:22.153747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:20.235527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.091934+0000","last_change":"2024-09-10T10:44:09.017385+0000","last_active":"2024-09-10T10:44:22.091934+0000","last_peered":"2024-09-10T10:44:22.091934+0000","last_clean":"2024-09-10T10:44:22.091934+0000","last_became_active":"2024-09-10T10:44:09.017033+0000","last_became_peered":"2024-09-10T10:44:09.017033+0000","last_unstale":"2024-09-10T10:44:22.091934+0000","last_undegraded":"2024-09-10T10:44:22.091934+0000","last_fullsized":"2024-09-10T10:44:22.091934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:57.963765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079263+0000","last_change":"2024-09-10T10:44:11.039194+0000","last_active":"2024-09-10T10:44:22.079263+0000","last_peered":"2024-09-10T10:44:22.079263+0000","last_clean":"2024-09-10T10:44:22.079263+0000","last_became_active":"2024-09-10T10:44:11.038290+0000","last_became_peered":"2024-09-10T10:44:11.038290+0000","last_unstale":"2024-09-10T10:44:22.079263+0000","last_undegraded":"2024-09-10T10:44:22.079263+0000","last_fullsized":"2024-09-10T10:44:22.079263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:50.938124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085070+0000","last_change":"2024-09-10T10:44:09.142618+0000","last_active":"2024-09-10T10:44:22.085070+0000","last_peered":"2024-09-10T10:44:22.085070+0000","last_clean":"2024-09-10T10:44:22.085070+0000","last_became_active":"2024-09-10T10:44:09.142232+0000","last_became_peered":"2024-09-10T10:44:09.142232+0000","last_unstale":"2024-09-10T10:44:22.085070+0000","last_undegraded":"2024-09-10T10:44:22.085070+0000","last_fullsized":"2024-09-10T10:44:22.085070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:18.797080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082392+0000","last_change":"2024-09-10T10:44:11.039288+0000","last_active":"2024-09-10T10:44:22.082392+0000","last_peered":"2024-09-10T10:44:22.082392+0000","last_clean":"2024-09-10T10:44:22.082392+0000","last_became_active":"2024-09-10T10:44:11.038955+0000","last_became_peered":"2024-09-10T10:44:11.038955+0000","last_unstale":"2024-09-10T10:44:22.082392+0000","last_undegraded":"2024-09-10T10:44:22.082392+0000","last_fullsized":"2024-09-10T10:44:22.082392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:29.564274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153587+0000","last_change":"2024-09-10T10:44:09.020638+0000","last_active":"2024-09-10T10:44:22.153587+0000","last_peered":"2024-09-10T10:44:22.153587+0000","last_clean":"2024-09-10T10:44:22.153587+0000","last_became_active":"2024-09-10T10:44:09.020320+0000","last_became_peered":"2024-09-10T10:44:09.020320+0000","last_unstale":"2024-09-10T10:44:22.153587+0000","last_undegraded":"2024-09-10T10:44:22.153587+0000","last_fullsized":"2024-09-10T10:44:22.153587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:00.793392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078085+0000","last_change":"2024-09-10T10:44:11.034010+0000","last_active":"2024-09-10T10:44:22.078085+0000","last_peered":"2024-09-10T10:44:22.078085+0000","last_clean":"2024-09-10T10:44:22.078085+0000","last_became_active":"2024-09-10T10:44:11.033639+0000","last_became_peered":"2024-09-10T10:44:11.033639+0000","last_unstale":"2024-09-10T10:44:22.078085+0000","last_undegraded":"2024-09-10T10:44:22.078085+0000","last_fullsized":"2024-09-10T10:44:22.078085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:20.127131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081171+0000","last_change":"2024-09-10T10:44:09.019478+0000","last_active":"2024-09-10T10:44:22.081171+0000","last_peered":"2024-09-10T10:44:22.081171+0000","last_clean":"2024-09-10T10:44:22.081171+0000","last_became_active":"2024-09-10T10:44:09.019033+0000","last_became_peered":"2024-09-10T10:44:09.019033+0000","last_unstale":"2024-09-10T10:44:22.081171+0000","last_undegraded":"2024-09-10T10:44:22.081171+0000","last_fullsized":"2024-09-10T10:44:22.081171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:17.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153647+0000","last_change":"2024-09-10T10:44:11.036118+0000","last_active":"2024-09-10T10:44:22.153647+0000","last_peered":"2024-09-10T10:44:22.153647+0000","last_clean":"2024-09-10T10:44:22.153647+0000","last_became_active":"2024-09-10T10:44:11.035648+0000","last_became_peered":"2024-09-10T10:44:11.035648+0000","last_unstale":"2024-09-10T10:44:22.153647+0000","last_undegraded":"2024-09-10T10:44:22.153647+0000","last_fullsized":"2024-09-10T10:44:22.153647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:52.134207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.100063+0000","last_change":"2024-09-10T10:44:22.099643+0000","last_active":"2024-09-10T10:44:22.100063+0000","last_peered":"2024-09-10T10:44:22.100063+0000","last_clean":"2024-09-10T10:44:22.100063+0000","last_became_active":"2024-09-10T10:44:22.093961+0000","last_became_peered":"2024-09-10T10:44:22.093961+0000","last_unstale":"2024-09-10T10:44:22.100063+0000","last_undegraded":"2024-09-10T10:44:22.100063+0000","last_fullsized":"2024-09-10T10:44:22.100063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:11.958415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082997+0000","last_change":"2024-09-10T10:44:11.035067+0000","last_active":"2024-09-10T10:44:22.082997+0000","last_peered":"2024-09-10T10:44:22.082997+0000","last_clean":"2024-09-10T10:44:22.082997+0000","last_became_active":"2024-09-10T10:44:11.034756+0000","last_became_peered":"2024-09-10T10:44:11.034756+0000","last_unstale":"2024-09-10T10:44:22.082997+0000","last_undegraded":"2024-09-10T10:44:22.082997+0000","last_fullsized":"2024-09-10T10:44:22.082997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:20.425948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080613+0000","last_change":"2024-09-10T10:44:11.039774+0000","last_active":"2024-09-10T10:44:22.080613+0000","last_peered":"2024-09-10T10:44:22.080613+0000","last_clean":"2024-09-10T10:44:22.080613+0000","last_became_active":"2024-09-10T10:44:11.039433+0000","last_became_peered":"2024-09-10T10:44:11.039433+0000","last_unstale":"2024-09-10T10:44:22.080613+0000","last_undegraded":"2024-09-10T10:44:22.080613+0000","last_fullsized":"2024-09-10T10:44:22.080613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:17.743436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083623+0000","last_change":"2024-09-10T10:44:09.022130+0000","last_active":"2024-09-10T10:44:22.083623+0000","last_peered":"2024-09-10T10:44:22.083623+0000","last_clean":"2024-09-10T10:44:22.083623+0000","last_became_active":"2024-09-10T10:44:09.021769+0000","last_became_peered":"2024-09-10T10:44:09.021769+0000","last_unstale":"2024-09-10T10:44:22.083623+0000","last_undegraded":"2024-09-10T10:44:22.083623+0000","last_fullsized":"2024-09-10T10:44:22.083623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:03.215955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085154+0000","last_change":"2024-09-10T10:44:11.044147+0000","last_active":"2024-09-10T10:44:22.085154+0000","last_peered":"2024-09-10T10:44:22.085154+0000","last_clean":"2024-09-10T10:44:22.085154+0000","last_became_active":"2024-09-10T10:44:11.042545+0000","last_became_peered":"2024-09-10T10:44:11.042545+0000","last_unstale":"2024-09-10T10:44:22.085154+0000","last_undegraded":"2024-09-10T10:44:22.085154+0000","last_fullsized":"2024-09-10T10:44:22.085154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:59.360624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083361+0000","last_change":"2024-09-10T10:44:09.023396+0000","last_active":"2024-09-10T10:44:22.083361+0000","last_peered":"2024-09-10T10:44:22.083361+0000","last_clean":"2024-09-10T10:44:22.083361+0000","last_became_active":"2024-09-10T10:44:09.022876+0000","last_became_peered":"2024-09-10T10:44:09.022876+0000","last_unstale":"2024-09-10T10:44:22.083361+0000","last_undegraded":"2024-09-10T10:44:22.083361+0000","last_fullsized":"2024-09-10T10:44:22.083361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:01.901011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079435+0000","last_change":"2024-09-10T10:44:11.033338+0000","last_active":"2024-09-10T10:44:22.079435+0000","last_peered":"2024-09-10T10:44:22.079435+0000","last_clean":"2024-09-10T10:44:22.079435+0000","last_became_active":"2024-09-10T10:44:11.032941+0000","last_became_peered":"2024-09-10T10:44:11.032941+0000","last_unstale":"2024-09-10T10:44:22.079435+0000","last_undegraded":"2024-09-10T10:44:22.079435+0000","last_fullsized":"2024-09-10T10:44:22.079435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:13.787799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.154016+0000","last_change":"2024-09-10T10:44:09.143814+0000","last_active":"2024-09-10T10:44:22.154016+0000","last_peered":"2024-09-10T10:44:22.154016+0000","last_clean":"2024-09-10T10:44:22.154016+0000","last_became_active":"2024-09-10T10:44:09.141942+0000","last_became_peered":"2024-09-10T10:44:09.141942+0000","last_unstale":"2024-09-10T10:44:22.154016+0000","last_undegraded":"2024-09-10T10:44:22.154016+0000","last_fullsized":"2024-09-10T10:44:22.154016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:31.839457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153764+0000","last_change":"2024-09-10T10:44:11.041386+0000","last_active":"2024-09-10T10:44:22.153764+0000","last_peered":"2024-09-10T10:44:22.153764+0000","last_clean":"2024-09-10T10:44:22.153764+0000","last_became_active":"2024-09-10T10:44:11.041060+0000","last_became_peered":"2024-09-10T10:44:11.041060+0000","last_unstale":"2024-09-10T10:44:22.153764+0000","last_undegraded":"2024-09-10T10:44:22.153764+0000","last_fullsized":"2024-09-10T10:44:22.153764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:38.528977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080586+0000","last_change":"2024-09-10T10:44:09.023375+0000","last_active":"2024-09-10T10:44:22.080586+0000","last_peered":"2024-09-10T10:44:22.080586+0000","last_clean":"2024-09-10T10:44:22.080586+0000","last_became_active":"2024-09-10T10:44:09.023082+0000","last_became_peered":"2024-09-10T10:44:09.023082+0000","last_unstale":"2024-09-10T10:44:22.080586+0000","last_undegraded":"2024-09-10T10:44:22.080586+0000","last_fullsized":"2024-09-10T10:44:22.080586+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:51.685194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64777,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":314,"num_read_kb":225,"num_write":164,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612633,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":880640,"data_stored":430529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841565,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1576960,"data_stored":1083177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005091,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":430765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234024,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":430685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495661,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1490944,"data_stored":1022224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40368,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704784,"statfs":{"total":95995035648,"available":95953698816,"internally_reserved":0,"allocated":884736,"data_stored":430154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855415,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1021908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":888832,"data_stored":430591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":430465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":492464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":491114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":905216,"data_stored":431425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:14.102 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:45:14.365 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:14.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:14 smithi007 ceph-mon[30873]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:14.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:14 smithi007 ceph-mon[30873]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:14.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:14.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:14.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:14.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:14.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:14 smithi007 ceph-mon[30873]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4101079282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f4106f10 msgr2=0x7fd6f410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 --2- 172.21.15.7:0/4101079282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f4106f10 0x7fd6f410ea90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009900 tx=0x7fd6ec02a410 comp rx=0 tx=0).stop 2024-09-10T10:45:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4101079282 shutdown_connections 2024-09-10T10:45:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 --2- 172.21.15.7:0/4101079282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f4106f10 0x7fd6f410ea90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 --2- 172.21.15.7:0/4101079282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f4102790 0x7fd6f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 --2- 172.21.15.7:0/4101079282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4101de0 0x7fd6f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4101079282 >> 172.21.15.7:0/4101079282 conn(0x7fd6f40fd8c0 msgr2=0x7fd6f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4101079282 shutdown_connections 2024-09-10T10:45:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4101079282 wait complete. 2024-09-10T10:45:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 Processor -- start 2024-09-10T10:45:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.780+0000 7fd6fb5da640 1 -- start start 2024-09-10T10:45:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6fb5da640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f4101de0 0x7fd6f4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6fb5da640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f4102790 0x7fd6f4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6fb5da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4106f10 0x7fd6f419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6fb5da640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4111fc0 con 0x7fd6f4101de0 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6fb5da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4111e40 con 0x7fd6f4106f10 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6fb5da640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f4112140 con 0x7fd6f4102790 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f9376640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f4101de0 0x7fd6f4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f9376640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f4101de0 0x7fd6f4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35914/0 (socket says 172.21.15.7:35914) 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f9376640 1 -- 172.21.15.7:0/4152692812 learned_addr learned my addr 172.21.15.7:0/4152692812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f8b75640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f4102790 0x7fd6f4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f9b77640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4106f10 0x7fd6f419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f9376640 1 -- 172.21.15.7:0/4152692812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f4102790 msgr2=0x7fd6f4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:14.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f9376640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f4102790 0x7fd6f4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f9376640 1 -- 172.21.15.7:0/4152692812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4106f10 msgr2=0x7fd6f419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f9376640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4106f10 0x7fd6f419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f9376640 1 -- 172.21.15.7:0/4152692812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f419d3b0 con 0x7fd6f4101de0 2024-09-10T10:45:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f9376640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f4101de0 0x7fd6f4198400 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400eab0 tx=0x7fd6e400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f27fc640 1 -- 172.21.15.7:0/4152692812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e400ce40 con 0x7fd6f4101de0 2024-09-10T10:45:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f27fc640 1 -- 172.21.15.7:0/4152692812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6e4011950 con 0x7fd6f4101de0 2024-09-10T10:45:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4152692812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f419d6a0 con 0x7fd6f4101de0 2024-09-10T10:45:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.781+0000 7fd6f27fc640 1 -- 172.21.15.7:0/4152692812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4010ae0 con 0x7fd6f4101de0 2024-09-10T10:45:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.782+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4152692812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f41a4f80 con 0x7fd6f4101de0 2024-09-10T10:45:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.783+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4152692812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f41021c0 con 0x7fd6f4101de0 2024-09-10T10:45:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.784+0000 7fd6f27fc640 1 -- 172.21.15.7:0/4152692812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd6e4011b00 con 0x7fd6f4101de0 2024-09-10T10:45:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.784+0000 7fd6f27fc640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd6dc0a2c10 0x7fd6dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.785+0000 7fd6f8b75640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd6dc0a2c10 0x7fd6dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.785+0000 7fd6f27fc640 1 -- 172.21.15.7:0/4152692812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fd6e40cf890 con 0x7fd6f4101de0 2024-09-10T10:45:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.785+0000 7fd6f8b75640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd6dc0a2c10 0x7fd6dc0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd6f41088e0 tx=0x7fd6e800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.789+0000 7fd6f27fc640 1 -- 172.21.15.7:0/4152692812 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6e4016030 con 0x7fd6f4101de0 2024-09-10T10:45:14.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:14 smithi103 ceph-mon[35160]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:14.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:14 smithi103 ceph-mon[35160]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:14.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:14.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:14.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:14.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:14.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:14 smithi103 ceph-mon[35160]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:14.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.962+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4152692812 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6f4062630 con 0x7fd6dc0a2c10 2024-09-10T10:45:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.967+0000 7fd6f27fc640 1 -- 172.21.15.7:0/4152692812 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258709 (secure 0 0 0) 0x7fd6f4062630 con 0x7fd6dc0a2c10 2024-09-10T10:45:14.971 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4152692812 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd6dc0a2c10 msgr2=0x7fd6dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd6dc0a2c10 0x7fd6dc0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd6f41088e0 tx=0x7fd6e800b410 comp rx=0 tx=0).stop 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4152692812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f4101de0 msgr2=0x7fd6f4198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f4101de0 0x7fd6f4198400 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400eab0 tx=0x7fd6e400ef70 comp rx=0 tx=0).stop 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4152692812 shutdown_connections 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fd6dc0a2c10 0x7fd6dc0a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6f4106f10 0x7fd6f419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6f4102790 0x7fd6f4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 --2- 172.21.15.7:0/4152692812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f4101de0 0x7fd6f4198400 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4152692812 >> 172.21.15.7:0/4152692812 conn(0x7fd6f40fd8c0 msgr2=0x7fd6f40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4152692812 shutdown_connections 2024-09-10T10:45:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:14.970+0000 7fd6fb5da640 1 -- 172.21.15.7:0/4152692812 wait complete. 2024-09-10T10:45:14.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:14 smithi133 ceph-mon[35282]: pgmap v307: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:14.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:14 smithi133 ceph-mon[35282]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:14.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:14.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:14.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:14.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:14.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:14 smithi133 ceph-mon[35282]: from='mgr.14150 172.21.15.7:0/3159029858' entity='mgr.x' 2024-09-10T10:45:15.565 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-10T10:45:14.698725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655379,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":480,"num_read_kb":359,"num_write":347,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":399152,"kb_used_data":12444,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124542672,"statfs":{"total":1151940427776,"available":1151531696128,"internally_reserved":0,"allocated":12742656,"data_stored":7125501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":395752762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002977"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078402+0000","last_change":"2024-09-10T10:44:11.040610+0000","last_active":"2024-09-10T10:44:22.078402+0000","last_peered":"2024-09-10T10:44:22.078402+0000","last_clean":"2024-09-10T10:44:22.078402+0000","last_became_active":"2024-09-10T10:44:11.040250+0000","last_became_peered":"2024-09-10T10:44:11.040250+0000","last_unstale":"2024-09-10T10:44:22.078402+0000","last_undegraded":"2024-09-10T10:44:22.078402+0000","last_fullsized":"2024-09-10T10:44:22.078402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:25.577181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079163+0000","last_change":"2024-09-10T10:44:11.041944+0000","last_active":"2024-09-10T10:44:22.079163+0000","last_peered":"2024-09-10T10:44:22.079163+0000","last_clean":"2024-09-10T10:44:22.079163+0000","last_became_active":"2024-09-10T10:44:11.041587+0000","last_became_peered":"2024-09-10T10:44:11.041587+0000","last_unstale":"2024-09-10T10:44:22.079163+0000","last_undegraded":"2024-09-10T10:44:22.079163+0000","last_fullsized":"2024-09-10T10:44:22.079163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:50.923879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079144+0000","last_change":"2024-09-10T10:44:11.038730+0000","last_active":"2024-09-10T10:44:22.079144+0000","last_peered":"2024-09-10T10:44:22.079144+0000","last_clean":"2024-09-10T10:44:22.079144+0000","last_became_active":"2024-09-10T10:44:11.038153+0000","last_became_peered":"2024-09-10T10:44:11.038153+0000","last_unstale":"2024-09-10T10:44:22.079144+0000","last_undegraded":"2024-09-10T10:44:22.079144+0000","last_fullsized":"2024-09-10T10:44:22.079144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:47.283582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092089+0000","last_change":"2024-09-10T10:44:11.038739+0000","last_active":"2024-09-10T10:44:22.092089+0000","last_peered":"2024-09-10T10:44:22.092089+0000","last_clean":"2024-09-10T10:44:22.092089+0000","last_became_active":"2024-09-10T10:44:11.038002+0000","last_became_peered":"2024-09-10T10:44:11.038002+0000","last_unstale":"2024-09-10T10:44:22.092089+0000","last_undegraded":"2024-09-10T10:44:22.092089+0000","last_fullsized":"2024-09-10T10:44:22.092089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:40.552127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083116+0000","last_change":"2024-09-10T10:44:11.037026+0000","last_active":"2024-09-10T10:44:22.083116+0000","last_peered":"2024-09-10T10:44:22.083116+0000","last_clean":"2024-09-10T10:44:22.083116+0000","last_became_active":"2024-09-10T10:44:11.036575+0000","last_became_peered":"2024-09-10T10:44:11.036575+0000","last_unstale":"2024-09-10T10:44:22.083116+0000","last_undegraded":"2024-09-10T10:44:22.083116+0000","last_fullsized":"2024-09-10T10:44:22.083116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:38.994882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082953+0000","last_change":"2024-09-10T10:44:11.036454+0000","last_active":"2024-09-10T10:44:22.082953+0000","last_peered":"2024-09-10T10:44:22.082953+0000","last_clean":"2024-09-10T10:44:22.082953+0000","last_became_active":"2024-09-10T10:44:11.034870+0000","last_became_peered":"2024-09-10T10:44:11.034870+0000","last_unstale":"2024-09-10T10:44:22.082953+0000","last_undegraded":"2024-09-10T10:44:22.082953+0000","last_fullsized":"2024-09-10T10:44:22.082953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:09.390465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081505+0000","last_change":"2024-09-10T10:44:11.042061+0000","last_active":"2024-09-10T10:44:22.081505+0000","last_peered":"2024-09-10T10:44:22.081505+0000","last_clean":"2024-09-10T10:44:22.081505+0000","last_became_active":"2024-09-10T10:44:11.041740+0000","last_became_peered":"2024-09-10T10:44:11.041740+0000","last_unstale":"2024-09-10T10:44:22.081505+0000","last_undegraded":"2024-09-10T10:44:22.081505+0000","last_fullsized":"2024-09-10T10:44:22.081505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:27.305586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153429+0000","last_change":"2024-09-10T10:44:11.041426+0000","last_active":"2024-09-10T10:44:22.153429+0000","last_peered":"2024-09-10T10:44:22.153429+0000","last_clean":"2024-09-10T10:44:22.153429+0000","last_became_active":"2024-09-10T10:44:11.041094+0000","last_became_peered":"2024-09-10T10:44:11.041094+0000","last_unstale":"2024-09-10T10:44:22.153429+0000","last_undegraded":"2024-09-10T10:44:22.153429+0000","last_fullsized":"2024-09-10T10:44:22.153429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:48.448394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085711+0000","last_change":"2024-09-10T10:44:11.035830+0000","last_active":"2024-09-10T10:44:22.085711+0000","last_peered":"2024-09-10T10:44:22.085711+0000","last_clean":"2024-09-10T10:44:22.085711+0000","last_became_active":"2024-09-10T10:44:11.034736+0000","last_became_peered":"2024-09-10T10:44:11.034736+0000","last_unstale":"2024-09-10T10:44:22.085711+0000","last_undegraded":"2024-09-10T10:44:22.085711+0000","last_fullsized":"2024-09-10T10:44:22.085711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:25.988368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082569+0000","last_change":"2024-09-10T10:44:11.040292+0000","last_active":"2024-09-10T10:44:22.082569+0000","last_peered":"2024-09-10T10:44:22.082569+0000","last_clean":"2024-09-10T10:44:22.082569+0000","last_became_active":"2024-09-10T10:44:11.039979+0000","last_became_peered":"2024-09-10T10:44:11.039979+0000","last_unstale":"2024-09-10T10:44:22.082569+0000","last_undegraded":"2024-09-10T10:44:22.082569+0000","last_fullsized":"2024-09-10T10:44:22.082569+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:02.539946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082219+0000","last_change":"2024-09-10T10:44:11.043683+0000","last_active":"2024-09-10T10:44:22.082219+0000","last_peered":"2024-09-10T10:44:22.082219+0000","last_clean":"2024-09-10T10:44:22.082219+0000","last_became_active":"2024-09-10T10:44:11.043038+0000","last_became_peered":"2024-09-10T10:44:11.043038+0000","last_unstale":"2024-09-10T10:44:22.082219+0000","last_undegraded":"2024-09-10T10:44:22.082219+0000","last_fullsized":"2024-09-10T10:44:22.082219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:43:25.548780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092135+0000","last_change":"2024-09-10T10:44:11.041776+0000","last_active":"2024-09-10T10:44:22.092135+0000","last_peered":"2024-09-10T10:44:22.092135+0000","last_clean":"2024-09-10T10:44:22.092135+0000","last_became_active":"2024-09-10T10:44:11.041450+0000","last_became_peered":"2024-09-10T10:44:11.041450+0000","last_unstale":"2024-09-10T10:44:22.092135+0000","last_undegraded":"2024-09-10T10:44:22.092135+0000","last_fullsized":"2024-09-10T10:44:22.092135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:01.442578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077361+0000","last_change":"2024-09-10T10:44:11.033973+0000","last_active":"2024-09-10T10:44:22.077361+0000","last_peered":"2024-09-10T10:44:22.077361+0000","last_clean":"2024-09-10T10:44:22.077361+0000","last_became_active":"2024-09-10T10:44:11.033649+0000","last_became_peered":"2024-09-10T10:44:11.033649+0000","last_unstale":"2024-09-10T10:44:22.077361+0000","last_undegraded":"2024-09-10T10:44:22.077361+0000","last_fullsized":"2024-09-10T10:44:22.077361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:16.671491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078786+0000","last_change":"2024-09-10T10:44:11.036931+0000","last_active":"2024-09-10T10:44:22.078786+0000","last_peered":"2024-09-10T10:44:22.078786+0000","last_clean":"2024-09-10T10:44:22.078786+0000","last_became_active":"2024-09-10T10:44:11.036574+0000","last_became_peered":"2024-09-10T10:44:11.036574+0000","last_unstale":"2024-09-10T10:44:22.078786+0000","last_undegraded":"2024-09-10T10:44:22.078786+0000","last_fullsized":"2024-09-10T10:44:22.078786+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:57.908420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078875+0000","last_change":"2024-09-10T10:44:11.037701+0000","last_active":"2024-09-10T10:44:22.078875+0000","last_peered":"2024-09-10T10:44:22.078875+0000","last_clean":"2024-09-10T10:44:22.078875+0000","last_became_active":"2024-09-10T10:44:11.037417+0000","last_became_peered":"2024-09-10T10:44:11.037417+0000","last_unstale":"2024-09-10T10:44:22.078875+0000","last_undegraded":"2024-09-10T10:44:22.078875+0000","last_fullsized":"2024-09-10T10:44:22.078875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:35.584226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084980+0000","last_change":"2024-09-10T10:44:11.038545+0000","last_active":"2024-09-10T10:44:22.084980+0000","last_peered":"2024-09-10T10:44:22.084980+0000","last_clean":"2024-09-10T10:44:22.084980+0000","last_became_active":"2024-09-10T10:44:11.037574+0000","last_became_peered":"2024-09-10T10:44:11.037574+0000","last_unstale":"2024-09-10T10:44:22.084980+0000","last_undegraded":"2024-09-10T10:44:22.084980+0000","last_fullsized":"2024-09-10T10:44:22.084980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:13.471862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078510+0000","last_change":"2024-09-10T10:44:11.042231+0000","last_active":"2024-09-10T10:44:22.078510+0000","last_peered":"2024-09-10T10:44:22.078510+0000","last_clean":"2024-09-10T10:44:22.078510+0000","last_became_active":"2024-09-10T10:44:11.041764+0000","last_became_peered":"2024-09-10T10:44:11.041764+0000","last_unstale":"2024-09-10T10:44:22.078510+0000","last_undegraded":"2024-09-10T10:44:22.078510+0000","last_fullsized":"2024-09-10T10:44:22.078510+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:38.301271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080361+0000","last_change":"2024-09-10T10:44:11.044492+0000","last_active":"2024-09-10T10:44:22.080361+0000","last_peered":"2024-09-10T10:44:22.080361+0000","last_clean":"2024-09-10T10:44:22.080361+0000","last_became_active":"2024-09-10T10:44:11.044135+0000","last_became_peered":"2024-09-10T10:44:11.044135+0000","last_unstale":"2024-09-10T10:44:22.080361+0000","last_undegraded":"2024-09-10T10:44:22.080361+0000","last_fullsized":"2024-09-10T10:44:22.080361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:47.054076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081417+0000","last_change":"2024-09-10T10:44:11.040349+0000","last_active":"2024-09-10T10:44:22.081417+0000","last_peered":"2024-09-10T10:44:22.081417+0000","last_clean":"2024-09-10T10:44:22.081417+0000","last_became_active":"2024-09-10T10:44:11.040051+0000","last_became_peered":"2024-09-10T10:44:11.040051+0000","last_unstale":"2024-09-10T10:44:22.081417+0000","last_undegraded":"2024-09-10T10:44:22.081417+0000","last_fullsized":"2024-09-10T10:44:22.081417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:33.777529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085576+0000","last_change":"2024-09-10T10:44:22.085576+0000","last_active":"2024-09-10T10:44:22.085576+0000","last_peered":"2024-09-10T10:44:22.085576+0000","last_clean":"2024-09-10T10:44:22.085576+0000","last_became_active":"2024-09-10T10:44:22.085135+0000","last_became_peered":"2024-09-10T10:44:22.085135+0000","last_unstale":"2024-09-10T10:44:22.085576+0000","last_undegraded":"2024-09-10T10:44:22.085576+0000","last_fullsized":"2024-09-10T10:44:22.085576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:23.871319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077458+0000","last_change":"2024-09-10T10:44:11.030324+0000","last_active":"2024-09-10T10:44:22.077458+0000","last_peered":"2024-09-10T10:44:22.077458+0000","last_clean":"2024-09-10T10:44:22.077458+0000","last_became_active":"2024-09-10T10:44:11.029652+0000","last_became_peered":"2024-09-10T10:44:11.029652+0000","last_unstale":"2024-09-10T10:44:22.077458+0000","last_undegraded":"2024-09-10T10:44:22.077458+0000","last_fullsized":"2024-09-10T10:44:22.077458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:47:51.332548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085112+0000","last_change":"2024-09-10T10:44:11.040637+0000","last_active":"2024-09-10T10:44:22.085112+0000","last_peered":"2024-09-10T10:44:22.085112+0000","last_clean":"2024-09-10T10:44:22.085112+0000","last_became_active":"2024-09-10T10:44:11.040345+0000","last_became_peered":"2024-09-10T10:44:11.040345+0000","last_unstale":"2024-09-10T10:44:22.085112+0000","last_undegraded":"2024-09-10T10:44:22.085112+0000","last_fullsized":"2024-09-10T10:44:22.085112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:18.166658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153476+0000","last_change":"2024-09-10T10:44:11.043157+0000","last_active":"2024-09-10T10:44:22.153476+0000","last_peered":"2024-09-10T10:44:22.153476+0000","last_clean":"2024-09-10T10:44:22.153476+0000","last_became_active":"2024-09-10T10:44:11.042823+0000","last_became_peered":"2024-09-10T10:44:11.042823+0000","last_unstale":"2024-09-10T10:44:22.153476+0000","last_undegraded":"2024-09-10T10:44:22.153476+0000","last_fullsized":"2024-09-10T10:44:22.153476+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:23.988072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080356+0000","last_change":"2024-09-10T10:44:11.041918+0000","last_active":"2024-09-10T10:44:22.080356+0000","last_peered":"2024-09-10T10:44:22.080356+0000","last_clean":"2024-09-10T10:44:22.080356+0000","last_became_active":"2024-09-10T10:44:11.040111+0000","last_became_peered":"2024-09-10T10:44:11.040111+0000","last_unstale":"2024-09-10T10:44:22.080356+0000","last_undegraded":"2024-09-10T10:44:22.080356+0000","last_fullsized":"2024-09-10T10:44:22.080356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:24.999485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.092270+0000","last_change":"2024-09-10T10:44:22.091959+0000","last_active":"2024-09-10T10:44:22.092270+0000","last_peered":"2024-09-10T10:44:22.092270+0000","last_clean":"2024-09-10T10:44:22.092270+0000","last_became_active":"2024-09-10T10:44:22.091707+0000","last_became_peered":"2024-09-10T10:44:22.091707+0000","last_unstale":"2024-09-10T10:44:22.092270+0000","last_undegraded":"2024-09-10T10:44:22.092270+0000","last_fullsized":"2024-09-10T10:44:22.092270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:38.454868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081432+0000","last_change":"2024-09-10T10:44:11.034339+0000","last_active":"2024-09-10T10:44:22.081432+0000","last_peered":"2024-09-10T10:44:22.081432+0000","last_clean":"2024-09-10T10:44:22.081432+0000","last_became_active":"2024-09-10T10:44:11.034022+0000","last_became_peered":"2024-09-10T10:44:11.034022+0000","last_unstale":"2024-09-10T10:44:22.081432+0000","last_undegraded":"2024-09-10T10:44:22.081432+0000","last_fullsized":"2024-09-10T10:44:22.081432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:54.122244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078068+0000","last_change":"2024-09-10T10:44:11.037871+0000","last_active":"2024-09-10T10:44:22.078068+0000","last_peered":"2024-09-10T10:44:22.078068+0000","last_clean":"2024-09-10T10:44:22.078068+0000","last_became_active":"2024-09-10T10:44:11.037372+0000","last_became_peered":"2024-09-10T10:44:11.037372+0000","last_unstale":"2024-09-10T10:44:22.078068+0000","last_undegraded":"2024-09-10T10:44:22.078068+0000","last_fullsized":"2024-09-10T10:44:22.078068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:22.296078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077383+0000","last_change":"2024-09-10T10:44:11.034826+0000","last_active":"2024-09-10T10:44:22.077383+0000","last_peered":"2024-09-10T10:44:22.077383+0000","last_clean":"2024-09-10T10:44:22.077383+0000","last_became_active":"2024-09-10T10:44:11.034520+0000","last_became_peered":"2024-09-10T10:44:11.034520+0000","last_unstale":"2024-09-10T10:44:22.077383+0000","last_undegraded":"2024-09-10T10:44:22.077383+0000","last_fullsized":"2024-09-10T10:44:22.077383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:13.131060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085136+0000","last_change":"2024-09-10T10:44:22.085136+0000","last_active":"2024-09-10T10:44:22.085136+0000","last_peered":"2024-09-10T10:44:22.085136+0000","last_clean":"2024-09-10T10:44:22.085136+0000","last_became_active":"2024-09-10T10:44:22.084718+0000","last_became_peered":"2024-09-10T10:44:22.084718+0000","last_unstale":"2024-09-10T10:44:22.085136+0000","last_undegraded":"2024-09-10T10:44:22.085136+0000","last_fullsized":"2024-09-10T10:44:22.085136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:31.165424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081062+0000","last_change":"2024-09-10T10:44:11.042336+0000","last_active":"2024-09-10T10:44:22.081062+0000","last_peered":"2024-09-10T10:44:22.081062+0000","last_clean":"2024-09-10T10:44:22.081062+0000","last_became_active":"2024-09-10T10:44:11.041127+0000","last_became_peered":"2024-09-10T10:44:11.041127+0000","last_unstale":"2024-09-10T10:44:22.081062+0000","last_undegraded":"2024-09-10T10:44:22.081062+0000","last_fullsized":"2024-09-10T10:44:22.081062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:50.349201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081158+0000","last_change":"2024-09-10T10:44:11.043188+0000","last_active":"2024-09-10T10:44:22.081158+0000","last_peered":"2024-09-10T10:44:22.081158+0000","last_clean":"2024-09-10T10:44:22.081158+0000","last_became_active":"2024-09-10T10:44:11.042735+0000","last_became_peered":"2024-09-10T10:44:11.042735+0000","last_unstale":"2024-09-10T10:44:22.081158+0000","last_undegraded":"2024-09-10T10:44:22.081158+0000","last_fullsized":"2024-09-10T10:44:22.081158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:50.090810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077961+0000","last_change":"2024-09-10T10:44:11.041339+0000","last_active":"2024-09-10T10:44:22.077961+0000","last_peered":"2024-09-10T10:44:22.077961+0000","last_clean":"2024-09-10T10:44:22.077961+0000","last_became_active":"2024-09-10T10:44:11.038011+0000","last_became_peered":"2024-09-10T10:44:11.038011+0000","last_unstale":"2024-09-10T10:44:22.077961+0000","last_undegraded":"2024-09-10T10:44:22.077961+0000","last_fullsized":"2024-09-10T10:44:22.077961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:47.521465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084488+0000","last_change":"2024-09-10T10:44:22.084488+0000","last_active":"2024-09-10T10:44:22.084488+0000","last_peered":"2024-09-10T10:44:22.084488+0000","last_clean":"2024-09-10T10:44:22.084488+0000","last_became_active":"2024-09-10T10:44:22.084062+0000","last_became_peered":"2024-09-10T10:44:22.084062+0000","last_unstale":"2024-09-10T10:44:22.084488+0000","last_undegraded":"2024-09-10T10:44:22.084488+0000","last_fullsized":"2024-09-10T10:44:22.084488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:57.739261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"79'5","reported_seq":82,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:45:12.162039+0000","last_change":"2024-09-10T10:44:09.024716+0000","last_active":"2024-09-10T10:45:12.162039+0000","last_peered":"2024-09-10T10:45:12.162039+0000","last_clean":"2024-09-10T10:45:12.162039+0000","last_became_active":"2024-09-10T10:44:09.023901+0000","last_became_peered":"2024-09-10T10:44:09.023901+0000","last_unstale":"2024-09-10T10:45:12.162039+0000","last_undegraded":"2024-09-10T10:45:12.162039+0000","last_fullsized":"2024-09-10T10:45:12.162039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:28.901896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080568+0000","last_change":"2024-09-10T10:44:11.044838+0000","last_active":"2024-09-10T10:44:22.080568+0000","last_peered":"2024-09-10T10:44:22.080568+0000","last_clean":"2024-09-10T10:44:22.080568+0000","last_became_active":"2024-09-10T10:44:11.044542+0000","last_became_peered":"2024-09-10T10:44:11.044542+0000","last_unstale":"2024-09-10T10:44:22.080568+0000","last_undegraded":"2024-09-10T10:44:22.080568+0000","last_fullsized":"2024-09-10T10:44:22.080568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:02.753914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082979+0000","last_change":"2024-09-10T10:44:09.025799+0000","last_active":"2024-09-10T10:44:22.082979+0000","last_peered":"2024-09-10T10:44:22.082979+0000","last_clean":"2024-09-10T10:44:22.082979+0000","last_became_active":"2024-09-10T10:44:09.025421+0000","last_became_peered":"2024-09-10T10:44:09.025421+0000","last_unstale":"2024-09-10T10:44:22.082979+0000","last_undegraded":"2024-09-10T10:44:22.082979+0000","last_fullsized":"2024-09-10T10:44:22.082979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:11.928663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077144+0000","last_change":"2024-09-10T10:44:09.017640+0000","last_active":"2024-09-10T10:44:22.077144+0000","last_peered":"2024-09-10T10:44:22.077144+0000","last_clean":"2024-09-10T10:44:22.077144+0000","last_became_active":"2024-09-10T10:44:09.017286+0000","last_became_peered":"2024-09-10T10:44:09.017286+0000","last_unstale":"2024-09-10T10:44:22.077144+0000","last_undegraded":"2024-09-10T10:44:22.077144+0000","last_fullsized":"2024-09-10T10:44:22.077144+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:59.678607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153461+0000","last_change":"2024-09-10T10:44:11.038225+0000","last_active":"2024-09-10T10:44:22.153461+0000","last_peered":"2024-09-10T10:44:22.153461+0000","last_clean":"2024-09-10T10:44:22.153461+0000","last_became_active":"2024-09-10T10:44:11.037924+0000","last_became_peered":"2024-09-10T10:44:11.037924+0000","last_unstale":"2024-09-10T10:44:22.153461+0000","last_undegraded":"2024-09-10T10:44:22.153461+0000","last_fullsized":"2024-09-10T10:44:22.153461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:58.787166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080948+0000","last_change":"2024-09-10T10:44:09.024676+0000","last_active":"2024-09-10T10:44:22.080948+0000","last_peered":"2024-09-10T10:44:22.080948+0000","last_clean":"2024-09-10T10:44:22.080948+0000","last_became_active":"2024-09-10T10:44:09.023970+0000","last_became_peered":"2024-09-10T10:44:09.023970+0000","last_unstale":"2024-09-10T10:44:22.080948+0000","last_undegraded":"2024-09-10T10:44:22.080948+0000","last_fullsized":"2024-09-10T10:44:22.080948+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:40.296260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080161+0000","last_change":"2024-09-10T10:44:11.040330+0000","last_active":"2024-09-10T10:44:22.080161+0000","last_peered":"2024-09-10T10:44:22.080161+0000","last_clean":"2024-09-10T10:44:22.080161+0000","last_became_active":"2024-09-10T10:44:11.040018+0000","last_became_peered":"2024-09-10T10:44:11.040018+0000","last_unstale":"2024-09-10T10:44:22.080161+0000","last_undegraded":"2024-09-10T10:44:22.080161+0000","last_fullsized":"2024-09-10T10:44:22.080161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:24.403070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":78,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:45:12.161368+0000","last_change":"2024-09-10T10:44:09.021488+0000","last_active":"2024-09-10T10:45:12.161368+0000","last_peered":"2024-09-10T10:45:12.161368+0000","last_clean":"2024-09-10T10:45:12.161368+0000","last_became_active":"2024-09-10T10:44:09.021088+0000","last_became_peered":"2024-09-10T10:44:09.021088+0000","last_unstale":"2024-09-10T10:45:12.161368+0000","last_undegraded":"2024-09-10T10:45:12.161368+0000","last_fullsized":"2024-09-10T10:45:12.161368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:37.990090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085498+0000","last_change":"2024-09-10T10:44:11.045142+0000","last_active":"2024-09-10T10:44:22.085498+0000","last_peered":"2024-09-10T10:44:22.085498+0000","last_clean":"2024-09-10T10:44:22.085498+0000","last_became_active":"2024-09-10T10:44:11.044770+0000","last_became_peered":"2024-09-10T10:44:11.044770+0000","last_unstale":"2024-09-10T10:44:22.085498+0000","last_undegraded":"2024-09-10T10:44:22.085498+0000","last_fullsized":"2024-09-10T10:44:22.085498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:10.731504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.096348+0000","last_change":"2024-09-10T10:44:22.096348+0000","last_active":"2024-09-10T10:44:22.096348+0000","last_peered":"2024-09-10T10:44:22.096348+0000","last_clean":"2024-09-10T10:44:22.096348+0000","last_became_active":"2024-09-10T10:44:22.092921+0000","last_became_peered":"2024-09-10T10:44:22.092921+0000","last_unstale":"2024-09-10T10:44:22.096348+0000","last_undegraded":"2024-09-10T10:44:22.096348+0000","last_fullsized":"2024-09-10T10:44:22.096348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:15.721849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079428+0000","last_change":"2024-09-10T10:44:11.034499+0000","last_active":"2024-09-10T10:44:22.079428+0000","last_peered":"2024-09-10T10:44:22.079428+0000","last_clean":"2024-09-10T10:44:22.079428+0000","last_became_active":"2024-09-10T10:44:11.034030+0000","last_became_peered":"2024-09-10T10:44:11.034030+0000","last_unstale":"2024-09-10T10:44:22.079428+0000","last_undegraded":"2024-09-10T10:44:22.079428+0000","last_fullsized":"2024-09-10T10:44:22.079428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:57.313000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080739+0000","last_change":"2024-09-10T10:44:09.142509+0000","last_active":"2024-09-10T10:44:22.080739+0000","last_peered":"2024-09-10T10:44:22.080739+0000","last_clean":"2024-09-10T10:44:22.080739+0000","last_became_active":"2024-09-10T10:44:09.142048+0000","last_became_peered":"2024-09-10T10:44:09.142048+0000","last_unstale":"2024-09-10T10:44:22.080739+0000","last_undegraded":"2024-09-10T10:44:22.080739+0000","last_fullsized":"2024-09-10T10:44:22.080739+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:57.937241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085180+0000","last_change":"2024-09-10T10:44:11.034851+0000","last_active":"2024-09-10T10:44:22.085180+0000","last_peered":"2024-09-10T10:44:22.085180+0000","last_clean":"2024-09-10T10:44:22.085180+0000","last_became_active":"2024-09-10T10:44:11.034524+0000","last_became_peered":"2024-09-10T10:44:11.034524+0000","last_unstale":"2024-09-10T10:44:22.085180+0000","last_undegraded":"2024-09-10T10:44:22.085180+0000","last_fullsized":"2024-09-10T10:44:22.085180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:49:06.356958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":164,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:45:13.174188+0000","last_change":"2024-09-10T10:44:09.024313+0000","last_active":"2024-09-10T10:45:13.174188+0000","last_peered":"2024-09-10T10:45:13.174188+0000","last_clean":"2024-09-10T10:45:13.174188+0000","last_became_active":"2024-09-10T10:44:09.023933+0000","last_became_peered":"2024-09-10T10:44:09.023933+0000","last_unstale":"2024-09-10T10:45:13.174188+0000","last_undegraded":"2024-09-10T10:45:13.174188+0000","last_fullsized":"2024-09-10T10:45:13.174188+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:37.522786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61167,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":113,"num_read_kb":115,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083275+0000","last_change":"2024-09-10T10:44:11.039171+0000","last_active":"2024-09-10T10:44:22.083275+0000","last_peered":"2024-09-10T10:44:22.083275+0000","last_clean":"2024-09-10T10:44:22.083275+0000","last_became_active":"2024-09-10T10:44:11.038833+0000","last_became_peered":"2024-09-10T10:44:11.038833+0000","last_unstale":"2024-09-10T10:44:22.083275+0000","last_undegraded":"2024-09-10T10:44:22.083275+0000","last_fullsized":"2024-09-10T10:44:22.083275+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:37.115626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081645+0000","last_change":"2024-09-10T10:44:09.021368+0000","last_active":"2024-09-10T10:44:22.081645+0000","last_peered":"2024-09-10T10:44:22.081645+0000","last_clean":"2024-09-10T10:44:22.081645+0000","last_became_active":"2024-09-10T10:44:09.020805+0000","last_became_peered":"2024-09-10T10:44:09.020805+0000","last_unstale":"2024-09-10T10:44:22.081645+0000","last_undegraded":"2024-09-10T10:44:22.081645+0000","last_fullsized":"2024-09-10T10:44:22.081645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:20.408776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077226+0000","last_change":"2024-09-10T10:44:11.037539+0000","last_active":"2024-09-10T10:44:22.077226+0000","last_peered":"2024-09-10T10:44:22.077226+0000","last_clean":"2024-09-10T10:44:22.077226+0000","last_became_active":"2024-09-10T10:44:11.036607+0000","last_became_peered":"2024-09-10T10:44:11.036607+0000","last_unstale":"2024-09-10T10:44:22.077226+0000","last_undegraded":"2024-09-10T10:44:22.077226+0000","last_fullsized":"2024-09-10T10:44:22.077226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:01:46.029665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080495+0000","last_change":"2024-09-10T10:44:09.023766+0000","last_active":"2024-09-10T10:44:22.080495+0000","last_peered":"2024-09-10T10:44:22.080495+0000","last_clean":"2024-09-10T10:44:22.080495+0000","last_became_active":"2024-09-10T10:44:09.023422+0000","last_became_peered":"2024-09-10T10:44:09.023422+0000","last_unstale":"2024-09-10T10:44:22.080495+0000","last_undegraded":"2024-09-10T10:44:22.080495+0000","last_fullsized":"2024-09-10T10:44:22.080495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:56.009610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082817+0000","last_change":"2024-09-10T10:44:11.038846+0000","last_active":"2024-09-10T10:44:22.082817+0000","last_peered":"2024-09-10T10:44:22.082817+0000","last_clean":"2024-09-10T10:44:22.082817+0000","last_became_active":"2024-09-10T10:44:11.038477+0000","last_became_peered":"2024-09-10T10:44:11.038477+0000","last_unstale":"2024-09-10T10:44:22.082817+0000","last_undegraded":"2024-09-10T10:44:22.082817+0000","last_fullsized":"2024-09-10T10:44:22.082817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:26.255612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"79'3","reported_seq":80,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:45:12.169145+0000","last_change":"2024-09-10T10:44:09.023394+0000","last_active":"2024-09-10T10:45:12.169145+0000","last_peered":"2024-09-10T10:45:12.169145+0000","last_clean":"2024-09-10T10:45:12.169145+0000","last_became_active":"2024-09-10T10:44:09.023049+0000","last_became_peered":"2024-09-10T10:44:09.023049+0000","last_unstale":"2024-09-10T10:45:12.169145+0000","last_undegraded":"2024-09-10T10:45:12.169145+0000","last_fullsized":"2024-09-10T10:45:12.169145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:00.024671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081712+0000","last_change":"2024-09-10T10:44:11.041162+0000","last_active":"2024-09-10T10:44:22.081712+0000","last_peered":"2024-09-10T10:44:22.081712+0000","last_clean":"2024-09-10T10:44:22.081712+0000","last_became_active":"2024-09-10T10:44:11.040848+0000","last_became_peered":"2024-09-10T10:44:11.040848+0000","last_unstale":"2024-09-10T10:44:22.081712+0000","last_undegraded":"2024-09-10T10:44:22.081712+0000","last_fullsized":"2024-09-10T10:44:22.081712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:13.044447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082834+0000","last_change":"2024-09-10T10:44:09.142490+0000","last_active":"2024-09-10T10:44:22.082834+0000","last_peered":"2024-09-10T10:44:22.082834+0000","last_clean":"2024-09-10T10:44:22.082834+0000","last_became_active":"2024-09-10T10:44:09.141983+0000","last_became_peered":"2024-09-10T10:44:09.141983+0000","last_unstale":"2024-09-10T10:44:22.082834+0000","last_undegraded":"2024-09-10T10:44:22.082834+0000","last_fullsized":"2024-09-10T10:44:22.082834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:34.481241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080333+0000","last_change":"2024-09-10T10:44:11.042109+0000","last_active":"2024-09-10T10:44:22.080333+0000","last_peered":"2024-09-10T10:44:22.080333+0000","last_clean":"2024-09-10T10:44:22.080333+0000","last_became_active":"2024-09-10T10:44:11.041154+0000","last_became_peered":"2024-09-10T10:44:11.041154+0000","last_unstale":"2024-09-10T10:44:22.080333+0000","last_undegraded":"2024-09-10T10:44:22.080333+0000","last_fullsized":"2024-09-10T10:44:22.080333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:50.134773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083174+0000","last_change":"2024-09-10T10:44:09.019117+0000","last_active":"2024-09-10T10:44:22.083174+0000","last_peered":"2024-09-10T10:44:22.083174+0000","last_clean":"2024-09-10T10:44:22.083174+0000","last_became_active":"2024-09-10T10:44:09.018733+0000","last_became_peered":"2024-09-10T10:44:09.018733+0000","last_unstale":"2024-09-10T10:44:22.083174+0000","last_undegraded":"2024-09-10T10:44:22.083174+0000","last_fullsized":"2024-09-10T10:44:22.083174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:40.049342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078300+0000","last_change":"2024-09-10T10:44:11.040381+0000","last_active":"2024-09-10T10:44:22.078300+0000","last_peered":"2024-09-10T10:44:22.078300+0000","last_clean":"2024-09-10T10:44:22.078300+0000","last_became_active":"2024-09-10T10:44:11.040053+0000","last_became_peered":"2024-09-10T10:44:11.040053+0000","last_unstale":"2024-09-10T10:44:22.078300+0000","last_undegraded":"2024-09-10T10:44:22.078300+0000","last_fullsized":"2024-09-10T10:44:22.078300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:29.043171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077403+0000","last_change":"2024-09-10T10:44:09.018173+0000","last_active":"2024-09-10T10:44:22.077403+0000","last_peered":"2024-09-10T10:44:22.077403+0000","last_clean":"2024-09-10T10:44:22.077403+0000","last_became_active":"2024-09-10T10:44:09.017790+0000","last_became_peered":"2024-09-10T10:44:09.017790+0000","last_unstale":"2024-09-10T10:44:22.077403+0000","last_undegraded":"2024-09-10T10:44:22.077403+0000","last_fullsized":"2024-09-10T10:44:22.077403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:43.515112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082583+0000","last_change":"2024-09-10T10:44:11.035293+0000","last_active":"2024-09-10T10:44:22.082583+0000","last_peered":"2024-09-10T10:44:22.082583+0000","last_clean":"2024-09-10T10:44:22.082583+0000","last_became_active":"2024-09-10T10:44:11.034958+0000","last_became_peered":"2024-09-10T10:44:11.034958+0000","last_unstale":"2024-09-10T10:44:22.082583+0000","last_undegraded":"2024-09-10T10:44:22.082583+0000","last_fullsized":"2024-09-10T10:44:22.082583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:09.397229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080789+0000","last_change":"2024-09-10T10:44:09.021322+0000","last_active":"2024-09-10T10:44:22.080789+0000","last_peered":"2024-09-10T10:44:22.080789+0000","last_clean":"2024-09-10T10:44:22.080789+0000","last_became_active":"2024-09-10T10:44:09.021018+0000","last_became_peered":"2024-09-10T10:44:09.021018+0000","last_unstale":"2024-09-10T10:44:22.080789+0000","last_undegraded":"2024-09-10T10:44:22.080789+0000","last_fullsized":"2024-09-10T10:44:22.080789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:48.999125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082090+0000","last_change":"2024-09-10T10:44:11.040545+0000","last_active":"2024-09-10T10:44:22.082090+0000","last_peered":"2024-09-10T10:44:22.082090+0000","last_clean":"2024-09-10T10:44:22.082090+0000","last_became_active":"2024-09-10T10:44:11.040042+0000","last_became_peered":"2024-09-10T10:44:11.040042+0000","last_unstale":"2024-09-10T10:44:22.082090+0000","last_undegraded":"2024-09-10T10:44:22.082090+0000","last_fullsized":"2024-09-10T10:44:22.082090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:14.367771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081581+0000","last_change":"2024-09-10T10:44:09.142944+0000","last_active":"2024-09-10T10:44:22.081581+0000","last_peered":"2024-09-10T10:44:22.081581+0000","last_clean":"2024-09-10T10:44:22.081581+0000","last_became_active":"2024-09-10T10:44:09.142550+0000","last_became_peered":"2024-09-10T10:44:09.142550+0000","last_unstale":"2024-09-10T10:44:22.081581+0000","last_undegraded":"2024-09-10T10:44:22.081581+0000","last_fullsized":"2024-09-10T10:44:22.081581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:56.247251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082331+0000","last_change":"2024-09-10T10:44:11.038205+0000","last_active":"2024-09-10T10:44:22.082331+0000","last_peered":"2024-09-10T10:44:22.082331+0000","last_clean":"2024-09-10T10:44:22.082331+0000","last_became_active":"2024-09-10T10:44:11.037817+0000","last_became_peered":"2024-09-10T10:44:11.037817+0000","last_unstale":"2024-09-10T10:44:22.082331+0000","last_undegraded":"2024-09-10T10:44:22.082331+0000","last_fullsized":"2024-09-10T10:44:22.082331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:39.221897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":343,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081904+0000","last_change":"2024-09-10T10:43:01.771132+0000","last_active":"2024-09-10T10:44:22.081904+0000","last_peered":"2024-09-10T10:44:22.081904+0000","last_clean":"2024-09-10T10:44:22.081904+0000","last_became_active":"2024-09-10T10:43:01.770568+0000","last_became_peered":"2024-09-10T10:43:01.770568+0000","last_unstale":"2024-09-10T10:44:22.081904+0000","last_undegraded":"2024-09-10T10:44:22.081904+0000","last_fullsized":"2024-09-10T10:44:22.081904+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:39:37.547474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:39:37.547474+0000","last_clean_scrub_stamp":"2024-09-10T10:39:37.547474+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:09.699899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.091961+0000","last_change":"2024-09-10T10:44:09.021115+0000","last_active":"2024-09-10T10:44:22.091961+0000","last_peered":"2024-09-10T10:44:22.091961+0000","last_clean":"2024-09-10T10:44:22.091961+0000","last_became_active":"2024-09-10T10:44:09.020747+0000","last_became_peered":"2024-09-10T10:44:09.020747+0000","last_unstale":"2024-09-10T10:44:22.091961+0000","last_undegraded":"2024-09-10T10:44:22.091961+0000","last_fullsized":"2024-09-10T10:44:22.091961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:15.828535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153652+0000","last_change":"2024-09-10T10:44:11.046484+0000","last_active":"2024-09-10T10:44:22.153652+0000","last_peered":"2024-09-10T10:44:22.153652+0000","last_clean":"2024-09-10T10:44:22.153652+0000","last_became_active":"2024-09-10T10:44:11.046142+0000","last_became_peered":"2024-09-10T10:44:11.046142+0000","last_unstale":"2024-09-10T10:44:22.153652+0000","last_undegraded":"2024-09-10T10:44:22.153652+0000","last_fullsized":"2024-09-10T10:44:22.153652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:15.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.088392+0000","last_change":"2024-09-10T10:44:22.087926+0000","last_active":"2024-09-10T10:44:22.088392+0000","last_peered":"2024-09-10T10:44:22.088392+0000","last_clean":"2024-09-10T10:44:22.088392+0000","last_became_active":"2024-09-10T10:44:22.084248+0000","last_became_peered":"2024-09-10T10:44:22.084248+0000","last_unstale":"2024-09-10T10:44:22.088392+0000","last_undegraded":"2024-09-10T10:44:22.088392+0000","last_fullsized":"2024-09-10T10:44:22.088392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:15.774609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077128+0000","last_change":"2024-09-10T10:44:11.034105+0000","last_active":"2024-09-10T10:44:22.077128+0000","last_peered":"2024-09-10T10:44:22.077128+0000","last_clean":"2024-09-10T10:44:22.077128+0000","last_became_active":"2024-09-10T10:44:11.033792+0000","last_became_peered":"2024-09-10T10:44:11.033792+0000","last_unstale":"2024-09-10T10:44:22.077128+0000","last_undegraded":"2024-09-10T10:44:22.077128+0000","last_fullsized":"2024-09-10T10:44:22.077128+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:41.131963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":80,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:45:13.170092+0000","last_change":"2024-09-10T10:44:09.142514+0000","last_active":"2024-09-10T10:45:13.170092+0000","last_peered":"2024-09-10T10:45:13.170092+0000","last_clean":"2024-09-10T10:45:13.170092+0000","last_became_active":"2024-09-10T10:44:09.142115+0000","last_became_peered":"2024-09-10T10:44:09.142115+0000","last_unstale":"2024-09-10T10:45:13.170092+0000","last_undegraded":"2024-09-10T10:45:13.170092+0000","last_fullsized":"2024-09-10T10:45:13.170092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:37.520105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080531+0000","last_change":"2024-09-10T10:44:11.040122+0000","last_active":"2024-09-10T10:44:22.080531+0000","last_peered":"2024-09-10T10:44:22.080531+0000","last_clean":"2024-09-10T10:44:22.080531+0000","last_became_active":"2024-09-10T10:44:11.039649+0000","last_became_peered":"2024-09-10T10:44:11.039649+0000","last_unstale":"2024-09-10T10:44:22.080531+0000","last_undegraded":"2024-09-10T10:44:22.080531+0000","last_fullsized":"2024-09-10T10:44:22.080531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:26.401981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083151+0000","last_change":"2024-09-10T10:44:09.024933+0000","last_active":"2024-09-10T10:44:22.083151+0000","last_peered":"2024-09-10T10:44:22.083151+0000","last_clean":"2024-09-10T10:44:22.083151+0000","last_became_active":"2024-09-10T10:44:09.024579+0000","last_became_peered":"2024-09-10T10:44:09.024579+0000","last_unstale":"2024-09-10T10:44:22.083151+0000","last_undegraded":"2024-09-10T10:44:22.083151+0000","last_fullsized":"2024-09-10T10:44:22.083151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:45.644932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.084217+0000","last_change":"2024-09-10T10:44:22.084217+0000","last_active":"2024-09-10T10:44:22.084217+0000","last_peered":"2024-09-10T10:44:22.084217+0000","last_clean":"2024-09-10T10:44:22.084217+0000","last_became_active":"2024-09-10T10:44:22.083707+0000","last_became_peered":"2024-09-10T10:44:22.083707+0000","last_unstale":"2024-09-10T10:44:22.084217+0000","last_undegraded":"2024-09-10T10:44:22.084217+0000","last_fullsized":"2024-09-10T10:44:22.084217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:20:30.124690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080402+0000","last_change":"2024-09-10T10:44:09.142462+0000","last_active":"2024-09-10T10:44:22.080402+0000","last_peered":"2024-09-10T10:44:22.080402+0000","last_clean":"2024-09-10T10:44:22.080402+0000","last_became_active":"2024-09-10T10:44:09.142086+0000","last_became_peered":"2024-09-10T10:44:09.142086+0000","last_unstale":"2024-09-10T10:44:22.080402+0000","last_undegraded":"2024-09-10T10:44:22.080402+0000","last_fullsized":"2024-09-10T10:44:22.080402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:28.019899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082676+0000","last_change":"2024-09-10T10:44:11.036928+0000","last_active":"2024-09-10T10:44:22.082676+0000","last_peered":"2024-09-10T10:44:22.082676+0000","last_clean":"2024-09-10T10:44:22.082676+0000","last_became_active":"2024-09-10T10:44:11.036626+0000","last_became_peered":"2024-09-10T10:44:11.036626+0000","last_unstale":"2024-09-10T10:44:22.082676+0000","last_undegraded":"2024-09-10T10:44:22.082676+0000","last_fullsized":"2024-09-10T10:44:22.082676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:42.624867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083407+0000","last_change":"2024-09-10T10:44:09.142521+0000","last_active":"2024-09-10T10:44:22.083407+0000","last_peered":"2024-09-10T10:44:22.083407+0000","last_clean":"2024-09-10T10:44:22.083407+0000","last_became_active":"2024-09-10T10:44:09.142061+0000","last_became_peered":"2024-09-10T10:44:09.142061+0000","last_unstale":"2024-09-10T10:44:22.083407+0000","last_undegraded":"2024-09-10T10:44:22.083407+0000","last_fullsized":"2024-09-10T10:44:22.083407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:14.054388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.077489+0000","last_change":"2024-09-10T10:44:11.031953+0000","last_active":"2024-09-10T10:44:22.077489+0000","last_peered":"2024-09-10T10:44:22.077489+0000","last_clean":"2024-09-10T10:44:22.077489+0000","last_became_active":"2024-09-10T10:44:11.031610+0000","last_became_peered":"2024-09-10T10:44:11.031610+0000","last_unstale":"2024-09-10T10:44:22.077489+0000","last_undegraded":"2024-09-10T10:44:22.077489+0000","last_fullsized":"2024-09-10T10:44:22.077489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:03.305651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:35.030830+0000","last_change":"2024-09-10T10:44:09.016752+0000","last_active":"2024-09-10T10:44:35.030830+0000","last_peered":"2024-09-10T10:44:35.030830+0000","last_clean":"2024-09-10T10:44:35.030830+0000","last_became_active":"2024-09-10T10:44:09.016464+0000","last_became_peered":"2024-09-10T10:44:09.016464+0000","last_unstale":"2024-09-10T10:44:35.030830+0000","last_undegraded":"2024-09-10T10:44:35.030830+0000","last_fullsized":"2024-09-10T10:44:35.030830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:25.136212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153747+0000","last_change":"2024-09-10T10:44:11.044793+0000","last_active":"2024-09-10T10:44:22.153747+0000","last_peered":"2024-09-10T10:44:22.153747+0000","last_clean":"2024-09-10T10:44:22.153747+0000","last_became_active":"2024-09-10T10:44:11.043611+0000","last_became_peered":"2024-09-10T10:44:11.043611+0000","last_unstale":"2024-09-10T10:44:22.153747+0000","last_undegraded":"2024-09-10T10:44:22.153747+0000","last_fullsized":"2024-09-10T10:44:22.153747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:20.235527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.091934+0000","last_change":"2024-09-10T10:44:09.017385+0000","last_active":"2024-09-10T10:44:22.091934+0000","last_peered":"2024-09-10T10:44:22.091934+0000","last_clean":"2024-09-10T10:44:22.091934+0000","last_became_active":"2024-09-10T10:44:09.017033+0000","last_became_peered":"2024-09-10T10:44:09.017033+0000","last_unstale":"2024-09-10T10:44:22.091934+0000","last_undegraded":"2024-09-10T10:44:22.091934+0000","last_fullsized":"2024-09-10T10:44:22.091934+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:57.963765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079263+0000","last_change":"2024-09-10T10:44:11.039194+0000","last_active":"2024-09-10T10:44:22.079263+0000","last_peered":"2024-09-10T10:44:22.079263+0000","last_clean":"2024-09-10T10:44:22.079263+0000","last_became_active":"2024-09-10T10:44:11.038290+0000","last_became_peered":"2024-09-10T10:44:11.038290+0000","last_unstale":"2024-09-10T10:44:22.079263+0000","last_undegraded":"2024-09-10T10:44:22.079263+0000","last_fullsized":"2024-09-10T10:44:22.079263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:50.938124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085070+0000","last_change":"2024-09-10T10:44:09.142618+0000","last_active":"2024-09-10T10:44:22.085070+0000","last_peered":"2024-09-10T10:44:22.085070+0000","last_clean":"2024-09-10T10:44:22.085070+0000","last_became_active":"2024-09-10T10:44:09.142232+0000","last_became_peered":"2024-09-10T10:44:09.142232+0000","last_unstale":"2024-09-10T10:44:22.085070+0000","last_undegraded":"2024-09-10T10:44:22.085070+0000","last_fullsized":"2024-09-10T10:44:22.085070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:18.797080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082392+0000","last_change":"2024-09-10T10:44:11.039288+0000","last_active":"2024-09-10T10:44:22.082392+0000","last_peered":"2024-09-10T10:44:22.082392+0000","last_clean":"2024-09-10T10:44:22.082392+0000","last_became_active":"2024-09-10T10:44:11.038955+0000","last_became_peered":"2024-09-10T10:44:11.038955+0000","last_unstale":"2024-09-10T10:44:22.082392+0000","last_undegraded":"2024-09-10T10:44:22.082392+0000","last_fullsized":"2024-09-10T10:44:22.082392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:29.564274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153587+0000","last_change":"2024-09-10T10:44:09.020638+0000","last_active":"2024-09-10T10:44:22.153587+0000","last_peered":"2024-09-10T10:44:22.153587+0000","last_clean":"2024-09-10T10:44:22.153587+0000","last_became_active":"2024-09-10T10:44:09.020320+0000","last_became_peered":"2024-09-10T10:44:09.020320+0000","last_unstale":"2024-09-10T10:44:22.153587+0000","last_undegraded":"2024-09-10T10:44:22.153587+0000","last_fullsized":"2024-09-10T10:44:22.153587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:00.793392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.078085+0000","last_change":"2024-09-10T10:44:11.034010+0000","last_active":"2024-09-10T10:44:22.078085+0000","last_peered":"2024-09-10T10:44:22.078085+0000","last_clean":"2024-09-10T10:44:22.078085+0000","last_became_active":"2024-09-10T10:44:11.033639+0000","last_became_peered":"2024-09-10T10:44:11.033639+0000","last_unstale":"2024-09-10T10:44:22.078085+0000","last_undegraded":"2024-09-10T10:44:22.078085+0000","last_fullsized":"2024-09-10T10:44:22.078085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:20.127131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.081171+0000","last_change":"2024-09-10T10:44:09.019478+0000","last_active":"2024-09-10T10:44:22.081171+0000","last_peered":"2024-09-10T10:44:22.081171+0000","last_clean":"2024-09-10T10:44:22.081171+0000","last_became_active":"2024-09-10T10:44:09.019033+0000","last_became_peered":"2024-09-10T10:44:09.019033+0000","last_unstale":"2024-09-10T10:44:22.081171+0000","last_undegraded":"2024-09-10T10:44:22.081171+0000","last_fullsized":"2024-09-10T10:44:22.081171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:17.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153647+0000","last_change":"2024-09-10T10:44:11.036118+0000","last_active":"2024-09-10T10:44:22.153647+0000","last_peered":"2024-09-10T10:44:22.153647+0000","last_clean":"2024-09-10T10:44:22.153647+0000","last_became_active":"2024-09-10T10:44:11.035648+0000","last_became_peered":"2024-09-10T10:44:11.035648+0000","last_unstale":"2024-09-10T10:44:22.153647+0000","last_undegraded":"2024-09-10T10:44:22.153647+0000","last_fullsized":"2024-09-10T10:44:22.153647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:52.134207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.100063+0000","last_change":"2024-09-10T10:44:22.099643+0000","last_active":"2024-09-10T10:44:22.100063+0000","last_peered":"2024-09-10T10:44:22.100063+0000","last_clean":"2024-09-10T10:44:22.100063+0000","last_became_active":"2024-09-10T10:44:22.093961+0000","last_became_peered":"2024-09-10T10:44:22.093961+0000","last_unstale":"2024-09-10T10:44:22.100063+0000","last_undegraded":"2024-09-10T10:44:22.100063+0000","last_fullsized":"2024-09-10T10:44:22.100063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:11.958415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.082997+0000","last_change":"2024-09-10T10:44:11.035067+0000","last_active":"2024-09-10T10:44:22.082997+0000","last_peered":"2024-09-10T10:44:22.082997+0000","last_clean":"2024-09-10T10:44:22.082997+0000","last_became_active":"2024-09-10T10:44:11.034756+0000","last_became_peered":"2024-09-10T10:44:11.034756+0000","last_unstale":"2024-09-10T10:44:22.082997+0000","last_undegraded":"2024-09-10T10:44:22.082997+0000","last_fullsized":"2024-09-10T10:44:22.082997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:20.425948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080613+0000","last_change":"2024-09-10T10:44:11.039774+0000","last_active":"2024-09-10T10:44:22.080613+0000","last_peered":"2024-09-10T10:44:22.080613+0000","last_clean":"2024-09-10T10:44:22.080613+0000","last_became_active":"2024-09-10T10:44:11.039433+0000","last_became_peered":"2024-09-10T10:44:11.039433+0000","last_unstale":"2024-09-10T10:44:22.080613+0000","last_undegraded":"2024-09-10T10:44:22.080613+0000","last_fullsized":"2024-09-10T10:44:22.080613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:17.743436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083623+0000","last_change":"2024-09-10T10:44:09.022130+0000","last_active":"2024-09-10T10:44:22.083623+0000","last_peered":"2024-09-10T10:44:22.083623+0000","last_clean":"2024-09-10T10:44:22.083623+0000","last_became_active":"2024-09-10T10:44:09.021769+0000","last_became_peered":"2024-09-10T10:44:09.021769+0000","last_unstale":"2024-09-10T10:44:22.083623+0000","last_undegraded":"2024-09-10T10:44:22.083623+0000","last_fullsized":"2024-09-10T10:44:22.083623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:03.215955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.085154+0000","last_change":"2024-09-10T10:44:11.044147+0000","last_active":"2024-09-10T10:44:22.085154+0000","last_peered":"2024-09-10T10:44:22.085154+0000","last_clean":"2024-09-10T10:44:22.085154+0000","last_became_active":"2024-09-10T10:44:11.042545+0000","last_became_peered":"2024-09-10T10:44:11.042545+0000","last_unstale":"2024-09-10T10:44:22.085154+0000","last_undegraded":"2024-09-10T10:44:22.085154+0000","last_fullsized":"2024-09-10T10:44:22.085154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:59.360624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.083361+0000","last_change":"2024-09-10T10:44:09.023396+0000","last_active":"2024-09-10T10:44:22.083361+0000","last_peered":"2024-09-10T10:44:22.083361+0000","last_clean":"2024-09-10T10:44:22.083361+0000","last_became_active":"2024-09-10T10:44:09.022876+0000","last_became_peered":"2024-09-10T10:44:09.022876+0000","last_unstale":"2024-09-10T10:44:22.083361+0000","last_undegraded":"2024-09-10T10:44:22.083361+0000","last_fullsized":"2024-09-10T10:44:22.083361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:01.901011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.079435+0000","last_change":"2024-09-10T10:44:11.033338+0000","last_active":"2024-09-10T10:44:22.079435+0000","last_peered":"2024-09-10T10:44:22.079435+0000","last_clean":"2024-09-10T10:44:22.079435+0000","last_became_active":"2024-09-10T10:44:11.032941+0000","last_became_peered":"2024-09-10T10:44:11.032941+0000","last_unstale":"2024-09-10T10:44:22.079435+0000","last_undegraded":"2024-09-10T10:44:22.079435+0000","last_fullsized":"2024-09-10T10:44:22.079435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:13.787799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.154016+0000","last_change":"2024-09-10T10:44:09.143814+0000","last_active":"2024-09-10T10:44:22.154016+0000","last_peered":"2024-09-10T10:44:22.154016+0000","last_clean":"2024-09-10T10:44:22.154016+0000","last_became_active":"2024-09-10T10:44:09.141942+0000","last_became_peered":"2024-09-10T10:44:09.141942+0000","last_unstale":"2024-09-10T10:44:22.154016+0000","last_undegraded":"2024-09-10T10:44:22.154016+0000","last_fullsized":"2024-09-10T10:44:22.154016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:31.839457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.153764+0000","last_change":"2024-09-10T10:44:11.041386+0000","last_active":"2024-09-10T10:44:22.153764+0000","last_peered":"2024-09-10T10:44:22.153764+0000","last_clean":"2024-09-10T10:44:22.153764+0000","last_became_active":"2024-09-10T10:44:11.041060+0000","last_became_peered":"2024-09-10T10:44:11.041060+0000","last_unstale":"2024-09-10T10:44:22.153764+0000","last_undegraded":"2024-09-10T10:44:22.153764+0000","last_fullsized":"2024-09-10T10:44:22.153764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:10.008215+0000","last_clean_scrub_stamp":"2024-09-10T10:44:10.008215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:38.528977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-10T10:44:22.080586+0000","last_change":"2024-09-10T10:44:09.023375+0000","last_active":"2024-09-10T10:44:22.080586+0000","last_peered":"2024-09-10T10:44:22.080586+0000","last_clean":"2024-09-10T10:44:22.080586+0000","last_became_active":"2024-09-10T10:44:09.023082+0000","last_became_peered":"2024-09-10T10:44:09.023082+0000","last_unstale":"2024-09-10T10:44:22.080586+0000","last_undegraded":"2024-09-10T10:44:22.080586+0000","last_fullsized":"2024-09-10T10:44:22.080586+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:07.993389+0000","last_clean_scrub_stamp":"2024-09-10T10:44:07.993389+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:51.685194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64777,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":344,"num_read_kb":240,"num_write":164,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612634,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":880640,"data_stored":430529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841566,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1576960,"data_stored":1083177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005092,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":430765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":430685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495662,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28480,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716672,"statfs":{"total":95995035648,"available":95965872128,"internally_reserved":0,"allocated":1490944,"data_stored":1022224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626483,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40368,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704784,"statfs":{"total":95995035648,"available":95953698816,"internally_reserved":0,"allocated":884736,"data_stored":430154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855416,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1021908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":888832,"data_stored":430591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":430465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":492464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":491114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":905216,"data_stored":431425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:15.569 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T10:45:15.569 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T10:45:15.569 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T10:45:15.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T10:45:15.827 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.310+0000 7f4e00f99640 1 -- 172.21.15.7:0/549464281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dfc102790 msgr2=0x7f4dfc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.310+0000 7f4e00f99640 1 --2- 172.21.15.7:0/549464281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dfc102790 0x7f4dfc1067e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009900 tx=0x7f4de402a410 comp rx=0 tx=0).stop 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.310+0000 7f4e00f99640 1 -- 172.21.15.7:0/549464281 shutdown_connections 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.310+0000 7f4e00f99640 1 --2- 172.21.15.7:0/549464281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dfc106f10 0x7f4dfc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.310+0000 7f4e00f99640 1 --2- 172.21.15.7:0/549464281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dfc102790 0x7f4dfc1067e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.310+0000 7f4e00f99640 1 --2- 172.21.15.7:0/549464281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc101de0 0x7f4dfc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.310+0000 7f4e00f99640 1 -- 172.21.15.7:0/549464281 >> 172.21.15.7:0/549464281 conn(0x7f4dfc0fd8c0 msgr2=0x7f4dfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.310+0000 7f4e00f99640 1 -- 172.21.15.7:0/549464281 shutdown_connections 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.310+0000 7f4e00f99640 1 -- 172.21.15.7:0/549464281 wait complete. 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4e00f99640 1 Processor -- start 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4e00f99640 1 -- start start 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4e00f99640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dfc101de0 0x7f4dfc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4e00f99640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc102790 0x7f4dfc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4e00f99640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dfc106f10 0x7f4dfc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4e00f99640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc111f70 con 0x7f4dfc101de0 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4e00f99640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc111df0 con 0x7f4dfc102790 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4e00f99640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1120f0 con 0x7f4dfc106f10 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4dfa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dfc101de0 0x7f4dfc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4dfa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dfc101de0 0x7f4dfc198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35930/0 (socket says 172.21.15.7:35930) 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.311+0000 7f4dfa59c640 1 -- 172.21.15.7:0/599619054 learned_addr learned my addr 172.21.15.7:0/599619054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4dfad9d640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dfc106f10 0x7f4dfc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4df1d9b640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc102790 0x7f4dfc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4dfa59c640 1 -- 172.21.15.7:0/599619054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dfc106f10 msgr2=0x7f4dfc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4dfa59c640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dfc106f10 0x7f4dfc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4dfa59c640 1 -- 172.21.15.7:0/599619054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc102790 msgr2=0x7f4dfc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4dfa59c640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc102790 0x7f4dfc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4dfa59c640 1 -- 172.21.15.7:0/599619054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc19d3e0 con 0x7f4dfc101de0 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4df1d9b640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc102790 0x7f4dfc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4dfad9d640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dfc106f10 0x7f4dfc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4dfa59c640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dfc101de0 0x7f4dfc198430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00d950 tx=0x7f4dec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4df3fff640 1 -- 172.21.15.7:0/599619054 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec017030 con 0x7f4dfc101de0 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4e00f99640 1 -- 172.21.15.7:0/599619054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc19d6d0 con 0x7f4dfc101de0 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4df3fff640 1 -- 172.21.15.7:0/599619054 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dec004020 con 0x7f4dfc101de0 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4df3fff640 1 -- 172.21.15.7:0/599619054 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec0047a0 con 0x7f4dfc101de0 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.312+0000 7f4e00f99640 1 -- 172.21.15.7:0/599619054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc1a4fb0 con 0x7f4dfc101de0 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.313+0000 7f4e00f99640 1 -- 172.21.15.7:0/599619054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dfc1021c0 con 0x7f4dfc101de0 2024-09-10T10:45:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.314+0000 7f4df3fff640 1 -- 172.21.15.7:0/599619054 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4dec0041d0 con 0x7f4dfc101de0 2024-09-10T10:45:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.315+0000 7f4df3fff640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4de80a2c10 0x7f4de80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.315+0000 7f4df1d9b640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4de80a2c10 0x7f4de80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.315+0000 7f4df3fff640 1 -- 172.21.15.7:0/599619054 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f4dec0cf330 con 0x7f4dfc101de0 2024-09-10T10:45:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.316+0000 7f4df1d9b640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4de80a2c10 0x7f4de80a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4de4000c00 tx=0x7f4de4032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.318+0000 7f4df3fff640 1 -- 172.21.15.7:0/599619054 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4dec09c0e0 con 0x7f4dfc101de0 2024-09-10T10:45:16.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.533+0000 7f4e00f99640 1 -- 172.21.15.7:0/599619054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4dfc199790 con 0x7f4dfc101de0 2024-09-10T10:45:16.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.533+0000 7f4df3fff640 1 -- 172.21.15.7:0/599619054 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4dfc199790 con 0x7f4dfc101de0 2024-09-10T10:45:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:16.536 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T10:45:16.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.535+0000 7f4e00f99640 1 -- 172.21.15.7:0/599619054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4de80a2c10 msgr2=0x7f4de80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.535+0000 7f4e00f99640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4de80a2c10 0x7f4de80a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4de4000c00 tx=0x7f4de4032c70 comp rx=0 tx=0).stop 2024-09-10T10:45:16.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.535+0000 7f4e00f99640 1 -- 172.21.15.7:0/599619054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dfc101de0 msgr2=0x7f4dfc198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.535+0000 7f4e00f99640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dfc101de0 0x7f4dfc198430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00d950 tx=0x7f4dec00de10 comp rx=0 tx=0).stop 2024-09-10T10:45:16.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.536+0000 7f4e00f99640 1 -- 172.21.15.7:0/599619054 shutdown_connections 2024-09-10T10:45:16.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.536+0000 7f4e00f99640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4de80a2c10 0x7f4de80a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.536+0000 7f4e00f99640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4dfc106f10 0x7f4dfc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.536+0000 7f4e00f99640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dfc102790 0x7f4dfc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.536+0000 7f4e00f99640 1 --2- 172.21.15.7:0/599619054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dfc101de0 0x7f4dfc198430 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.536+0000 7f4e00f99640 1 -- 172.21.15.7:0/599619054 >> 172.21.15.7:0/599619054 conn(0x7f4dfc0fd8c0 msgr2=0x7f4dfc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.536+0000 7f4e00f99640 1 -- 172.21.15.7:0/599619054 shutdown_connections 2024-09-10T10:45:16.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:16.536+0000 7f4e00f99640 1 -- 172.21.15.7:0/599619054 wait complete. 2024-09-10T10:45:16.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:16 smithi103 ceph-mon[35160]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:16.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:16 smithi103 ceph-mon[35160]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:16.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:16 smithi007 ceph-mon[30873]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:16.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:16 smithi007 ceph-mon[30873]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:16.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:16 smithi133 ceph-mon[35282]: pgmap v308: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:16.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:16 smithi133 ceph-mon[35282]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:45:17.156 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T10:45:17.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:17.788 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:17 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/599619054' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T10:45:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.868+0000 7fb444b3d640 1 -- 172.21.15.7:0/3273906321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44010c170 msgr2=0x7fb44010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.868+0000 7fb444b3d640 1 --2- 172.21.15.7:0/3273906321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44010c170 0x7fb44010e5f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb438009ca0 tx=0x7fb43802a950 comp rx=0 tx=0).stop 2024-09-10T10:45:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.868+0000 7fb444b3d640 1 -- 172.21.15.7:0/3273906321 shutdown_connections 2024-09-10T10:45:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.868+0000 7fb444b3d640 1 --2- 172.21.15.7:0/3273906321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44010c170 0x7fb44010e5f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.868+0000 7fb444b3d640 1 --2- 172.21.15.7:0/3273906321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4401036b0 0x7fb44010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.868+0000 7fb444b3d640 1 --2- 172.21.15.7:0/3273906321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb440102d90 0x7fb440103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.868+0000 7fb444b3d640 1 -- 172.21.15.7:0/3273906321 >> 172.21.15.7:0/3273906321 conn(0x7fb4400fc790 msgr2=0x7fb4400febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.868+0000 7fb444b3d640 1 -- 172.21.15.7:0/3273906321 shutdown_connections 2024-09-10T10:45:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.868+0000 7fb444b3d640 1 -- 172.21.15.7:0/3273906321 wait complete. 2024-09-10T10:45:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb444b3d640 1 Processor -- start 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb444b3d640 1 -- start start 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb444b3d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440102d90 0x7fb4401009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb444b3d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4401036b0 0x7fb440100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb444b3d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb44010c170 0x7fb440101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb444b3d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb440111f50 con 0x7fb440102d90 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb444b3d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb440111dd0 con 0x7fb4401036b0 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb444b3d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb4401120d0 con 0x7fb44010c170 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb43e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440102d90 0x7fb4401009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb43e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440102d90 0x7fb4401009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35948/0 (socket says 172.21.15.7:35948) 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb43e59c640 1 -- 172.21.15.7:0/2633509931 learned_addr learned my addr 172.21.15.7:0/2633509931 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb43dd9b640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4401036b0 0x7fb440100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.869+0000 7fb43ed9d640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb44010c170 0x7fb440101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb43e59c640 1 -- 172.21.15.7:0/2633509931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb44010c170 msgr2=0x7fb440101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb43e59c640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb44010c170 0x7fb440101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb43e59c640 1 -- 172.21.15.7:0/2633509931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4401036b0 msgr2=0x7fb440100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb43e59c640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4401036b0 0x7fb440100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb43e59c640 1 -- 172.21.15.7:0/2633509931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4401a6930 con 0x7fb440102d90 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb43ed9d640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb44010c170 0x7fb440101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb43e59c640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440102d90 0x7fb4401009f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb43000d770 tx=0x7fb43000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb42f7fe640 1 -- 172.21.15.7:0/2633509931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430017070 con 0x7fb440102d90 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb444b3d640 1 -- 172.21.15.7:0/2633509931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4401a6c20 con 0x7fb440102d90 2024-09-10T10:45:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb42f7fe640 1 -- 172.21.15.7:0/2633509931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb430005500 con 0x7fb440102d90 2024-09-10T10:45:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb42f7fe640 1 -- 172.21.15.7:0/2633509931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430004e00 con 0x7fb440102d90 2024-09-10T10:45:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.870+0000 7fb444b3d640 1 -- 172.21.15.7:0/2633509931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4401a7110 con 0x7fb440102d90 2024-09-10T10:45:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.871+0000 7fb444b3d640 1 -- 172.21.15.7:0/2633509931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb440103170 con 0x7fb440102d90 2024-09-10T10:45:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.872+0000 7fb42f7fe640 1 -- 172.21.15.7:0/2633509931 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb430004fa0 con 0x7fb440102d90 2024-09-10T10:45:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.873+0000 7fb42f7fe640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb4280a2c10 0x7fb4280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.873+0000 7fb43dd9b640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb4280a2c10 0x7fb4280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.873+0000 7fb42f7fe640 1 -- 172.21.15.7:0/2633509931 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fb4300ceca0 con 0x7fb440102d90 2024-09-10T10:45:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.874+0000 7fb43dd9b640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb4280a2c10 0x7fb4280a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb440102480 tx=0x7fb43400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:17.877+0000 7fb42f7fe640 1 -- 172.21.15.7:0/2633509931 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb43009ba50 con 0x7fb440102d90 2024-09-10T10:45:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:17 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/599619054' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T10:45:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:17 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/599619054' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T10:45:18.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.089+0000 7fb444b3d640 1 -- 172.21.15.7:0/2633509931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb440107210 con 0x7fb440102d90 2024-09-10T10:45:18.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.089+0000 7fb42f7fe640 1 -- 172.21.15.7:0/2633509931 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7fb43009b350 con 0x7fb440102d90 2024-09-10T10:45:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:18.092 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"btime":"2024-09-10T10:44:23:076442+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1984917528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1984917528},{"type":"v1","addr":"172.21.15.7:6839","nonce":1984917528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1887416712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1887416712},{"type":"v1","addr":"172.21.15.7:6841","nonce":1887416712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:44:11.549269+0000","modified":"2024-09-10T10:44:22.697096+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/2537001809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2537001809},{"type":"v1","addr":"172.21.15.103:6839","nonce":2537001809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24481,24482,24499]},"id":1}]} 2024-09-10T10:45:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-09-10T10:45:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 -- 172.21.15.7:0/2633509931 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb4280a2c10 msgr2=0x7fb4280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb4280a2c10 0x7fb4280a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb440102480 tx=0x7fb43400b390 comp rx=0 tx=0).stop 2024-09-10T10:45:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 -- 172.21.15.7:0/2633509931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440102d90 msgr2=0x7fb4401009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440102d90 0x7fb4401009f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb43000d770 tx=0x7fb43000dc30 comp rx=0 tx=0).stop 2024-09-10T10:45:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 -- 172.21.15.7:0/2633509931 shutdown_connections 2024-09-10T10:45:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb4280a2c10 0x7fb4280a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb44010c170 0x7fb440101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb4401036b0 0x7fb440100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 --2- 172.21.15.7:0/2633509931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb440102d90 0x7fb4401009f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 -- 172.21.15.7:0/2633509931 >> 172.21.15.7:0/2633509931 conn(0x7fb4400fc790 msgr2=0x7fb4400fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 -- 172.21.15.7:0/2633509931 shutdown_connections 2024-09-10T10:45:18.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:18.092+0000 7fb444b3d640 1 -- 172.21.15.7:0/2633509931 wait complete. 2024-09-10T10:45:18.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:18 smithi007 ceph-mon[30873]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:18.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:18 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2633509931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:18.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:18.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:18 smithi103 ceph-mon[35160]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:18.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:18 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2633509931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:18.944 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:18.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:18 smithi133 ceph-mon[35282]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:18.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:18 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2633509931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:19.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.382+0000 7f1d23083640 1 -- 172.21.15.7:0/495922357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c10c170 msgr2=0x7f1d1c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.382+0000 7f1d23083640 1 --2- 172.21.15.7:0/495922357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c10c170 0x7f1d1c10e610 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009ca0 tx=0x7f1d1402a950 comp rx=0 tx=0).stop 2024-09-10T10:45:19.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.382+0000 7f1d23083640 1 -- 172.21.15.7:0/495922357 shutdown_connections 2024-09-10T10:45:19.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.382+0000 7f1d23083640 1 --2- 172.21.15.7:0/495922357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c10c170 0x7f1d1c10e610 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.382+0000 7f1d23083640 1 --2- 172.21.15.7:0/495922357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d1c103660 0x7f1d1c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.382+0000 7f1d23083640 1 --2- 172.21.15.7:0/495922357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d1c102d40 0x7f1d1c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.382+0000 7f1d23083640 1 -- 172.21.15.7:0/495922357 >> 172.21.15.7:0/495922357 conn(0x7f1d1c0fc7a0 msgr2=0x7f1d1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:19.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.382+0000 7f1d23083640 1 -- 172.21.15.7:0/495922357 shutdown_connections 2024-09-10T10:45:19.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.382+0000 7f1d23083640 1 -- 172.21.15.7:0/495922357 wait complete. 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.382+0000 7f1d23083640 1 Processor -- start 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d23083640 1 -- start start 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d23083640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d1c102d40 0x7f1d1c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d23083640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c103660 0x7f1d1c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d23083640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d1c10c170 0x7f1d1c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d23083640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c111f50 con 0x7f1d1c103660 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d23083640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c111dd0 con 0x7f1d1c102d40 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d23083640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1120d0 con 0x7f1d1c10c170 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c103660 0x7f1d1c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c103660 0x7f1d1c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35966/0 (socket says 172.21.15.7:35966) 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d1bfff640 1 -- 172.21.15.7:0/3792951726 learned_addr learned my addr 172.21.15.7:0/3792951726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d21620640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d1c10c170 0x7f1d1c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d20e1f640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d1c102d40 0x7f1d1c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d1bfff640 1 -- 172.21.15.7:0/3792951726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d1c10c170 msgr2=0x7f1d1c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d1bfff640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d1c10c170 0x7f1d1c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d1bfff640 1 -- 172.21.15.7:0/3792951726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d1c102d40 msgr2=0x7f1d1c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d1bfff640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d1c102d40 0x7f1d1c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d1bfff640 1 -- 172.21.15.7:0/3792951726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c1a6830 con 0x7f1d1c103660 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.383+0000 7f1d1bfff640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c103660 0x7f1d1c100ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400d750 tx=0x7f1d0400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.384+0000 7f1d19ffb640 1 -- 172.21.15.7:0/3792951726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04017070 con 0x7f1d1c103660 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.384+0000 7f1d19ffb640 1 -- 172.21.15.7:0/3792951726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d04005020 con 0x7f1d1c103660 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.384+0000 7f1d23083640 1 -- 172.21.15.7:0/3792951726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c1a6b20 con 0x7f1d1c103660 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.384+0000 7f1d19ffb640 1 -- 172.21.15.7:0/3792951726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04004d00 con 0x7f1d1c103660 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.384+0000 7f1d23083640 1 -- 172.21.15.7:0/3792951726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c1a7010 con 0x7f1d1c103660 2024-09-10T10:45:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.385+0000 7f1d23083640 1 -- 172.21.15.7:0/3792951726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1c103120 con 0x7f1d1c103660 2024-09-10T10:45:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.386+0000 7f1d19ffb640 1 -- 172.21.15.7:0/3792951726 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1d04009a30 con 0x7f1d1c103660 2024-09-10T10:45:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.387+0000 7f1d19ffb640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f1d080a2c10 0x7f1d080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.387+0000 7f1d20e1f640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f1d080a2c10 0x7f1d080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.387+0000 7f1d19ffb640 1 -- 172.21.15.7:0/3792951726 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f1d040ceb90 con 0x7f1d1c103660 2024-09-10T10:45:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.387+0000 7f1d20e1f640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f1d080a2c10 0x7f1d080a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00ba40 tx=0x7f1d0c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.391+0000 7f1d19ffb640 1 -- 172.21.15.7:0/3792951726 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d0409b940 con 0x7f1d1c103660 2024-09-10T10:45:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.603+0000 7f1d23083640 1 -- 172.21.15.7:0/3792951726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d1c1a72b0 con 0x7f1d1c103660 2024-09-10T10:45:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.604+0000 7f1d19ffb640 1 -- 172.21.15.7:0/3792951726 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7f1d0409b240 con 0x7f1d1c103660 2024-09-10T10:45:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:19.607 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"btime":"2024-09-10T10:44:23:076442+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1984917528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1984917528},{"type":"v1","addr":"172.21.15.7:6839","nonce":1984917528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1887416712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1887416712},{"type":"v1","addr":"172.21.15.7:6841","nonce":1887416712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:44:11.549269+0000","modified":"2024-09-10T10:44:22.697096+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/2537001809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2537001809},{"type":"v1","addr":"172.21.15.103:6839","nonce":2537001809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24481,24482,24499]},"id":1}]} 2024-09-10T10:45:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 -- 172.21.15.7:0/3792951726 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f1d080a2c10 msgr2=0x7f1d080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f1d080a2c10 0x7f1d080a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00ba40 tx=0x7f1d0c005cc0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 -- 172.21.15.7:0/3792951726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c103660 msgr2=0x7f1d1c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c103660 0x7f1d1c100ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400d750 tx=0x7f1d0400dc10 comp rx=0 tx=0).stop 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 -- 172.21.15.7:0/3792951726 shutdown_connections 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f1d080a2c10 0x7f1d080a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d1c10c170 0x7f1d1c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d1c103660 0x7f1d1c100ee0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 --2- 172.21.15.7:0/3792951726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d1c102d40 0x7f1d1c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 -- 172.21.15.7:0/3792951726 >> 172.21.15.7:0/3792951726 conn(0x7f1d1c0fc7a0 msgr2=0x7f1d1c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 -- 172.21.15.7:0/3792951726 shutdown_connections 2024-09-10T10:45:19.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:19.606+0000 7f1d23083640 1 -- 172.21.15.7:0/3792951726 wait complete. 2024-09-10T10:45:20.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:45:20.451 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:20.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:20 smithi007 ceph-mon[30873]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:20.844 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:20 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3792951726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:20 smithi103 ceph-mon[35160]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:20.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:20 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3792951726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.924+0000 7f0f84042640 1 -- 172.21.15.7:0/1640562870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7c102ef0 msgr2=0x7f0f7c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.924+0000 7f0f84042640 1 --2- 172.21.15.7:0/1640562870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7c102ef0 0x7f0f7c10da40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009ca0 tx=0x7f0f7402a950 comp rx=0 tx=0).stop 2024-09-10T10:45:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.924+0000 7f0f84042640 1 -- 172.21.15.7:0/1640562870 shutdown_connections 2024-09-10T10:45:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.924+0000 7f0f84042640 1 --2- 172.21.15.7:0/1640562870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7c102ef0 0x7f0f7c10da40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.924+0000 7f0f84042640 1 --2- 172.21.15.7:0/1640562870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7c102570 0x7f0f7c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.924+0000 7f0f84042640 1 --2- 172.21.15.7:0/1640562870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f7c100e20 0x7f0f7c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.924+0000 7f0f84042640 1 -- 172.21.15.7:0/1640562870 >> 172.21.15.7:0/1640562870 conn(0x7f0f7c0fc7d0 msgr2=0x7f0f7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.924+0000 7f0f84042640 1 -- 172.21.15.7:0/1640562870 shutdown_connections 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.924+0000 7f0f84042640 1 -- 172.21.15.7:0/1640562870 wait complete. 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f84042640 1 Processor -- start 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f84042640 1 -- start start 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f84042640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7c100e20 0x7f0f7c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f84042640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f7c102570 0x7f0f7c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f84042640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7c102ef0 0x7f0f7c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f84042640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c110f80 con 0x7f0f7c100e20 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f84042640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c110e00 con 0x7f0f7c102570 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f84042640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c111100 con 0x7f0f7c102ef0 2024-09-10T10:45:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f81dde640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7c100e20 0x7f0f7c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f81dde640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7c100e20 0x7f0f7c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47836/0 (socket says 172.21.15.7:47836) 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f81dde640 1 -- 172.21.15.7:0/3031184313 learned_addr learned my addr 172.21.15.7:0/3031184313 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f815dd640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f7c102570 0x7f0f7c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f825df640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7c102ef0 0x7f0f7c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f81dde640 1 -- 172.21.15.7:0/3031184313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7c102ef0 msgr2=0x7f0f7c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f81dde640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7c102ef0 0x7f0f7c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f81dde640 1 -- 172.21.15.7:0/3031184313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f7c102570 msgr2=0x7f0f7c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f81dde640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f7c102570 0x7f0f7c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f81dde640 1 -- 172.21.15.7:0/3031184313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c1a69f0 con 0x7f0f7c100e20 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.925+0000 7f0f825df640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7c102ef0 0x7f0f7c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.926+0000 7f0f81dde640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7c100e20 0x7f0f7c0fcc80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800d950 tx=0x7f0f7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.926+0000 7f0f72ffd640 1 -- 172.21.15.7:0/3031184313 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78017030 con 0x7f0f7c100e20 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.926+0000 7f0f84042640 1 -- 172.21.15.7:0/3031184313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7c1a6ce0 con 0x7f0f7c100e20 2024-09-10T10:45:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.926+0000 7f0f72ffd640 1 -- 172.21.15.7:0/3031184313 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f78004020 con 0x7f0f7c100e20 2024-09-10T10:45:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.926+0000 7f0f72ffd640 1 -- 172.21.15.7:0/3031184313 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780047a0 con 0x7f0f7c100e20 2024-09-10T10:45:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.926+0000 7f0f84042640 1 -- 172.21.15.7:0/3031184313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f7c1a7210 con 0x7f0f7c100e20 2024-09-10T10:45:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.927+0000 7f0f84042640 1 -- 172.21.15.7:0/3031184313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7c101200 con 0x7f0f7c100e20 2024-09-10T10:45:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.928+0000 7f0f72ffd640 1 -- 172.21.15.7:0/3031184313 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0f780041d0 con 0x7f0f7c100e20 2024-09-10T10:45:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.929+0000 7f0f72ffd640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0f680a2c10 0x7f0f680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.929+0000 7f0f815dd640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0f680a2c10 0x7f0f680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.929+0000 7f0f72ffd640 1 -- 172.21.15.7:0/3031184313 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f0f780cecb0 con 0x7f0f7c100e20 2024-09-10T10:45:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.929+0000 7f0f815dd640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0f680a2c10 0x7f0f680a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005d70 tx=0x7f0f6c005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:20.933+0000 7f0f72ffd640 1 -- 172.21.15.7:0/3031184313 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f7809ba60 con 0x7f0f7c100e20 2024-09-10T10:45:20.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:20 smithi133 ceph-mon[35282]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:20.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:20 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3792951726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:21.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.111+0000 7f0f84042640 1 -- 172.21.15.7:0/3031184313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f7c10a0a0 con 0x7f0f7c100e20 2024-09-10T10:45:21.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.112+0000 7f0f72ffd640 1 -- 172.21.15.7:0/3031184313 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22646 (secure 0 0 0) 0x7f0f7809b360 con 0x7f0f7c100e20 2024-09-10T10:45:21.115 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:21.115 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:36:33.117443+0000","modified":"2024-09-10T10:44:22.070795+0000","last_up_change":"2024-09-10T10:43:23.913978+0000","last_in_change":"2024-09-10T10:42:59.733844+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:39:37.104114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:07.441548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:09.970273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9239e99-3a62-4883-b2f1-8964ebb54492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6803","nonce":704452390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6805","nonce":704452390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6809","nonce":704452390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6807","nonce":704452390}]},"public_addr":"172.21.15.7:6803/704452390","cluster_addr":"172.21.15.7:6805/704452390","heartbeat_back_addr":"172.21.15.7:6809/704452390","heartbeat_front_addr":"172.21.15.7:6807/704452390","state":["exists","up"]},{"osd":1,"uuid":"d750ff16-48c8-421d-b46b-49f0d2b7f4e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6801","nonce":2409158189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6803","nonce":2409158189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6807","nonce":2409158189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6805","nonce":2409158189}]},"public_addr":"172.21.15.103:6801/2409158189","cluster_addr":"172.21.15.103:6803/2409158189","heartbeat_back_addr":"172.21.15.103:6807/2409158189","heartbeat_front_addr":"172.21.15.103:6805/2409158189","state":["exists","up"]},{"osd":2,"uuid":"1ecb070a-a2f1-499a-99e7-f1967ebb52ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6801","nonce":1657349858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6803","nonce":1657349858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6807","nonce":1657349858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6805","nonce":1657349858}]},"public_addr":"172.21.15.133:6801/1657349858","cluster_addr":"172.21.15.133:6803/1657349858","heartbeat_back_addr":"172.21.15.133:6807/1657349858","heartbeat_front_addr":"172.21.15.133:6805/1657349858","state":["exists","up"]},{"osd":3,"uuid":"8e53d79e-038f-4a7f-9ac9-16b60f47bb4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6811","nonce":871781144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6813","nonce":871781144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6817","nonce":871781144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6815","nonce":871781144}]},"public_addr":"172.21.15.7:6811/871781144","cluster_addr":"172.21.15.7:6813/871781144","heartbeat_back_addr":"172.21.15.7:6817/871781144","heartbeat_front_addr":"172.21.15.7:6815/871781144","state":["exists","up"]},{"osd":4,"uuid":"e4f04c13-594c-43dd-9a98-d2138061b1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6809","nonce":1937243043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6811","nonce":1937243043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6815","nonce":1937243043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6813","nonce":1937243043}]},"public_addr":"172.21.15.103:6809/1937243043","cluster_addr":"172.21.15.103:6811/1937243043","heartbeat_back_addr":"172.21.15.103:6815/1937243043","heartbeat_front_addr":"172.21.15.103:6813/1937243043","state":["exists","up"]},{"osd":5,"uuid":"73143170-5816-407d-ae03-dcf896057681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6809","nonce":577882774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6811","nonce":577882774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6815","nonce":577882774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6813","nonce":577882774}]},"public_addr":"172.21.15.133:6809/577882774","cluster_addr":"172.21.15.133:6811/577882774","heartbeat_back_addr":"172.21.15.133:6815/577882774","heartbeat_front_addr":"172.21.15.133:6813/577882774","state":["exists","up"]},{"osd":6,"uuid":"3a5904d5-36de-47b0-97fa-5a10b24d0c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6819","nonce":2646192734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6821","nonce":2646192734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6825","nonce":2646192734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6823","nonce":2646192734}]},"public_addr":"172.21.15.7:6819/2646192734","cluster_addr":"172.21.15.7:6821/2646192734","heartbeat_back_addr":"172.21.15.7:6825/2646192734","heartbeat_front_addr":"172.21.15.7:6823/2646192734","state":["exists","up"]},{"osd":7,"uuid":"91e79d42-decd-4405-b9f2-9480f7edfca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6817","nonce":3858867236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6819","nonce":3858867236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6823","nonce":3858867236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6821","nonce":3858867236}]},"public_addr":"172.21.15.103:6817/3858867236","cluster_addr":"172.21.15.103:6819/3858867236","heartbeat_back_addr":"172.21.15.103:6823/3858867236","heartbeat_front_addr":"172.21.15.103:6821/3858867236","state":["exists","up"]},{"osd":8,"uuid":"64373754-e81a-430c-967d-4b4f289f3473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6817","nonce":131428321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6819","nonce":131428321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6823","nonce":131428321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6821","nonce":131428321}]},"public_addr":"172.21.15.133:6817/131428321","cluster_addr":"172.21.15.133:6819/131428321","heartbeat_back_addr":"172.21.15.133:6823/131428321","heartbeat_front_addr":"172.21.15.133:6821/131428321","state":["exists","up"]},{"osd":9,"uuid":"91ebe2c9-bda3-4596-ac76-5678c71f850b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6827","nonce":945633240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6829","nonce":945633240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6833","nonce":945633240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6831","nonce":945633240}]},"public_addr":"172.21.15.7:6827/945633240","cluster_addr":"172.21.15.7:6829/945633240","heartbeat_back_addr":"172.21.15.7:6833/945633240","heartbeat_front_addr":"172.21.15.7:6831/945633240","state":["exists","up"]},{"osd":10,"uuid":"2f65568f-89d1-4a67-a222-8e263d34c710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6825","nonce":907835659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6827","nonce":907835659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6831","nonce":907835659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6829","nonce":907835659}]},"public_addr":"172.21.15.103:6825/907835659","cluster_addr":"172.21.15.103:6827/907835659","heartbeat_back_addr":"172.21.15.103:6831/907835659","heartbeat_front_addr":"172.21.15.103:6829/907835659","state":["exists","up"]},{"osd":11,"uuid":"4b49839b-32e9-4105-abee-a253f1aa3f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6825","nonce":2538184038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6827","nonce":2538184038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6831","nonce":2538184038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6829","nonce":2538184038}]},"public_addr":"172.21.15.133:6825/2538184038","cluster_addr":"172.21.15.133:6827/2538184038","heartbeat_back_addr":"172.21.15.133:6831/2538184038","heartbeat_front_addr":"172.21.15.133:6829/2538184038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:44.541279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:10.103733+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:35.290268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:01.070763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:25.666941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:50.925733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:16.486537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:41.464155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:06.610089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:31.994080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:57.000696+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:21.398669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3394895925":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6800/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/262340568":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/3768226338":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/258851660":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3411487770":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/11850255":"2024-09-11T10:36:55.407956+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:45:21.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 -- 172.21.15.7:0/3031184313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0f680a2c10 msgr2=0x7f0f680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0f680a2c10 0x7f0f680a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005d70 tx=0x7f0f6c005d00 comp rx=0 tx=0).stop 2024-09-10T10:45:21.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 -- 172.21.15.7:0/3031184313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7c100e20 msgr2=0x7f0f7c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:21.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7c100e20 0x7f0f7c0fcc80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800d950 tx=0x7f0f7800de10 comp rx=0 tx=0).stop 2024-09-10T10:45:21.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 -- 172.21.15.7:0/3031184313 shutdown_connections 2024-09-10T10:45:21.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0f680a2c10 0x7f0f680a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7c102ef0 0x7f0f7c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f7c102570 0x7f0f7c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 --2- 172.21.15.7:0/3031184313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7c100e20 0x7f0f7c0fcc80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 -- 172.21.15.7:0/3031184313 >> 172.21.15.7:0/3031184313 conn(0x7f0f7c0fc7d0 msgr2=0x7f0f7c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 -- 172.21.15.7:0/3031184313 shutdown_connections 2024-09-10T10:45:21.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:21.115+0000 7f0f84042640 1 -- 172.21.15.7:0/3031184313 wait complete. 2024-09-10T10:45:21.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:21.804 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:21 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3031184313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:21 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3031184313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:21 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3031184313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/2103522658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f241082a0 msgr2=0x7f8f2410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 --2- 172.21.15.7:0/2103522658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f241082a0 0x7f8f2410ec40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009cd0 tx=0x7f8f1c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/2103522658 shutdown_connections 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 --2- 172.21.15.7:0/2103522658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f241082a0 0x7f8f2410ec40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 --2- 172.21.15.7:0/2103522658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f241036e0 0x7f8f24107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 --2- 172.21.15.7:0/2103522658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f24102d30 0x7f8f24103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/2103522658 >> 172.21.15.7:0/2103522658 conn(0x7f8f240fc7d0 msgr2=0x7f8f240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/2103522658 shutdown_connections 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/2103522658 wait complete. 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 Processor -- start 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.440+0000 7f8f2ac2c640 1 -- start start 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f2ac2c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f24102d30 0x7f8f2419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f2ac2c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f241036e0 0x7f8f2419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f2ac2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f241082a0 0x7f8f2419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f2ac2c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241120a0 con 0x7f8f241082a0 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f2ac2c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24111f20 con 0x7f8f24102d30 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f2ac2c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24112220 con 0x7f8f241036e0 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f291c9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f241082a0 0x7f8f2419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f291c9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f241082a0 0x7f8f2419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47858/0 (socket says 172.21.15.7:47858) 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f291c9640 1 -- 172.21.15.7:0/400697843 learned_addr learned my addr 172.21.15.7:0/400697843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f23fff640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f241036e0 0x7f8f2419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f289c8640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f24102d30 0x7f8f2419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f291c9640 1 -- 172.21.15.7:0/400697843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f241036e0 msgr2=0x7f8f2419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f291c9640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f241036e0 0x7f8f2419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f291c9640 1 -- 172.21.15.7:0/400697843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f24102d30 msgr2=0x7f8f2419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f291c9640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f24102d30 0x7f8f2419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f291c9640 1 -- 172.21.15.7:0/400697843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2419f6e0 con 0x7f8f241082a0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f291c9640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f241082a0 0x7f8f2419f000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c005cc0 tx=0x7f8f1c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f21ffb640 1 -- 172.21.15.7:0/400697843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c004040 con 0x7f8f241082a0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f21ffb640 1 -- 172.21.15.7:0/400697843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1c0046c0 con 0x7f8f241082a0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/400697843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2419f970 con 0x7f8f241082a0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.441+0000 7f8f21ffb640 1 -- 172.21.15.7:0/400697843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c046480 con 0x7f8f241082a0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.442+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/400697843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241a72b0 con 0x7f8f241082a0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.443+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/400697843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f24103110 con 0x7f8f241082a0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.444+0000 7f8f21ffb640 1 -- 172.21.15.7:0/400697843 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8f1c0305a0 con 0x7f8f241082a0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.445+0000 7f8f21ffb640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8f0c0a2c10 0x7f8f0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.445+0000 7f8f289c8640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8f0c0a2c10 0x7f8f0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.445+0000 7f8f21ffb640 1 -- 172.21.15.7:0/400697843 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f8f1c02e030 con 0x7f8f241082a0 2024-09-10T10:45:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.445+0000 7f8f289c8640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8f0c0a2c10 0x7f8f0c0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f14005fd0 tx=0x7f8f14005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:22.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.448+0000 7f8f21ffb640 1 -- 172.21.15.7:0/400697843 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f1c04b060 con 0x7f8f241082a0 2024-09-10T10:45:22.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.659+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/400697843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f2410c2c0 con 0x7f8f241082a0 2024-09-10T10:45:22.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.660+0000 7f8f21ffb640 1 -- 172.21.15.7:0/400697843 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7f8f1c0bd970 con 0x7f8f241082a0 2024-09-10T10:45:22.662 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:22.663 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"btime":"2024-09-10T10:44:23:076442+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1984917528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1984917528},{"type":"v1","addr":"172.21.15.7:6839","nonce":1984917528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1887416712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1887416712},{"type":"v1","addr":"172.21.15.7:6841","nonce":1887416712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:44:11.549269+0000","modified":"2024-09-10T10:44:22.697096+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/2537001809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2537001809},{"type":"v1","addr":"172.21.15.103:6839","nonce":2537001809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24481,24482,24499]},"id":1}]} 2024-09-10T10:45:22.663 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-09-10T10:45:22.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.662+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/400697843 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8f0c0a2c10 msgr2=0x7f8f0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:22.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.662+0000 7f8f2ac2c640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8f0c0a2c10 0x7f8f0c0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f14005fd0 tx=0x7f8f14005850 comp rx=0 tx=0).stop 2024-09-10T10:45:22.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.662+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/400697843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f241082a0 msgr2=0x7f8f2419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:22.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.662+0000 7f8f2ac2c640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f241082a0 0x7f8f2419f000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c005cc0 tx=0x7f8f1c003970 comp rx=0 tx=0).stop 2024-09-10T10:45:22.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.662+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/400697843 shutdown_connections 2024-09-10T10:45:22.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.662+0000 7f8f2ac2c640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8f0c0a2c10 0x7f8f0c0a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.662+0000 7f8f2ac2c640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f241082a0 0x7f8f2419f000 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.662+0000 7f8f2ac2c640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f241036e0 0x7f8f2419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.662+0000 7f8f2ac2c640 1 --2- 172.21.15.7:0/400697843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f24102d30 0x7f8f2419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.663+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/400697843 >> 172.21.15.7:0/400697843 conn(0x7f8f240fc7d0 msgr2=0x7f8f24101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.663+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/400697843 shutdown_connections 2024-09-10T10:45:22.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:22.663+0000 7f8f2ac2c640 1 -- 172.21.15.7:0/400697843 wait complete. 2024-09-10T10:45:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:22 smithi103 ceph-mon[35160]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:22.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:22 smithi007 ceph-mon[30873]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:22 smithi133 ceph-mon[35282]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:23.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T10:44:11.549269+0000', 'modified': '2024-09-10T10:44:22.697096+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24499, 'mds_1': 24482, 'mds_2': 24481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6833/691908645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6832', 'nonce': 691908645}, {'type': 'v1', 'addr': '172.21.15.133:6833', 'nonce': 691908645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6837/2960409846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 2960409846}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 2960409846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/4118505620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 4118505620}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 4118505620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.133:6837/3030361099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 3030361099}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3030361099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.103:6839/2537001809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2537001809}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2537001809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6839/1182528850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 1182528850}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 1182528850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24499, 'qdb_cluster': [24481, 24482, 24499]} 2024-09-10T10:45:23.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T10:45:23.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T10:45:23.587 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:23.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:23 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/400697843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:23 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/400697843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:23 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/400697843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.008+0000 7fb7af38c640 1 -- 172.21.15.7:0/1956992538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7a81082a0 msgr2=0x7fb7a810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.008+0000 7fb7af38c640 1 --2- 172.21.15.7:0/1956992538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7a81082a0 0x7fb7a810ec40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0009930 tx=0x7fb7a002a420 comp rx=0 tx=0).stop 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.008+0000 7fb7af38c640 1 -- 172.21.15.7:0/1956992538 shutdown_connections 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.008+0000 7fb7af38c640 1 --2- 172.21.15.7:0/1956992538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7a81082a0 0x7fb7a810ec40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.008+0000 7fb7af38c640 1 --2- 172.21.15.7:0/1956992538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a81036e0 0x7fb7a8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.008+0000 7fb7af38c640 1 --2- 172.21.15.7:0/1956992538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7a8102d30 0x7fb7a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.008+0000 7fb7af38c640 1 -- 172.21.15.7:0/1956992538 >> 172.21.15.7:0/1956992538 conn(0x7fb7a80fc7d0 msgr2=0x7fb7a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.008+0000 7fb7af38c640 1 -- 172.21.15.7:0/1956992538 shutdown_connections 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.008+0000 7fb7af38c640 1 -- 172.21.15.7:0/1956992538 wait complete. 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7af38c640 1 Processor -- start 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7af38c640 1 -- start start 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7af38c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7a8102d30 0x7fb7a81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7af38c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7a81036e0 0x7fb7a8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7af38c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a81082a0 0x7fb7a819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7af38c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a81120a0 con 0x7fb7a81036e0 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7af38c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8111f20 con 0x7fb7a8102d30 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7af38c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8112220 con 0x7fb7a81082a0 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7ac927640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7a81036e0 0x7fb7a8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7ac927640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7a81036e0 0x7fb7a8198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47878/0 (socket says 172.21.15.7:47878) 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.009+0000 7fb7ac927640 1 -- 172.21.15.7:0/4074006828 learned_addr learned my addr 172.21.15.7:0/4074006828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7ad128640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7a8102d30 0x7fb7a81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7ad929640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a81082a0 0x7fb7a819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7ac927640 1 -- 172.21.15.7:0/4074006828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a81082a0 msgr2=0x7fb7a819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7ac927640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a81082a0 0x7fb7a819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7ac927640 1 -- 172.21.15.7:0/4074006828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7a8102d30 msgr2=0x7fb7a81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7ac927640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7a8102d30 0x7fb7a81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7ac927640 1 -- 172.21.15.7:0/4074006828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a819d480 con 0x7fb7a81036e0 2024-09-10T10:45:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7ad929640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a81082a0 0x7fb7a819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7ac927640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7a81036e0 0x7fb7a8198a10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400db20 tx=0x7fb7a400df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb79a7fc640 1 -- 172.21.15.7:0/4074006828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4017070 con 0x7fb7a81036e0 2024-09-10T10:45:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb79a7fc640 1 -- 172.21.15.7:0/4074006828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a4003e50 con 0x7fb7a81036e0 2024-09-10T10:45:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7af38c640 1 -- 172.21.15.7:0/4074006828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a819d770 con 0x7fb7a81036e0 2024-09-10T10:45:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb79a7fc640 1 -- 172.21.15.7:0/4074006828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a40106c0 con 0x7fb7a81036e0 2024-09-10T10:45:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.010+0000 7fb7af38c640 1 -- 172.21.15.7:0/4074006828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a81a5050 con 0x7fb7a81036e0 2024-09-10T10:45:24.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.013+0000 7fb79a7fc640 1 -- 172.21.15.7:0/4074006828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb7a4003970 con 0x7fb7a81036e0 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.013+0000 7fb7b1765740 1 Processor -- start 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7b1765740 1 -- start start 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 0x5604950a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 0x5604950b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7b1765740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56049500f090 con 0x5604950a92f0 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7b1765740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560494fb6a70 con 0x5604950a89d0 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7b1765740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56049500b050 con 0x5604950b3bb0 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7ad128640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb79a7fc640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7940a2c90 0x7fb7940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7ad128640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47880/0 (socket says 172.21.15.7:47880) 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7ad128640 1 -- 172.21.15.7:0/3967047758 learned_addr learned my addr 172.21.15.7:0/3967047758 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:24.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7ac927640 1 --2- 172.21.15.7:0/3967047758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 0x5604950b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7ad929640 1 --2- 172.21.15.7:0/3967047758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 0x5604950a8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb79a7fc640 1 -- 172.21.15.7:0/4074006828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fb7a40cf040 con 0x7fb7a81036e0 2024-09-10T10:45:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7ad929640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7940a2c90 0x7fb7940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7ad128640 1 -- 172.21.15.7:0/3967047758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 msgr2=0x5604950b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.014+0000 7fb7ad128640 1 --2- 172.21.15.7:0/3967047758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 0x5604950b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.015+0000 7fb7ad128640 1 -- 172.21.15.7:0/3967047758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 msgr2=0x5604950a8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.015+0000 7fb7ad128640 1 --2- 172.21.15.7:0/3967047758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 0x5604950a8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.015+0000 7fb7ad128640 1 -- 172.21.15.7:0/3967047758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560494fc3130 con 0x5604950a92f0 2024-09-10T10:45:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.015+0000 7fb7ac927640 1 --2- 172.21.15.7:0/3967047758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 0x5604950b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:24.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.015+0000 7fb7ad128640 1 --2- 172.21.15.7:0/3967047758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b3670 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00ad00 tx=0x7fb79c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64656820e9d45eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:24.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.015+0000 7fb7ad929640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7940a2c90 0x7fb7940a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb7a002a940 tx=0x7fb7a0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:24.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.015+0000 7fb7937fe640 1 -- 172.21.15.7:0/3967047758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c01d070 con 0x5604950a92f0 2024-09-10T10:45:24.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.015+0000 7fb7937fe640 1 -- 172.21.15.7:0/3967047758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb79c018b20 con 0x5604950a92f0 2024-09-10T10:45:24.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.015+0000 7fb7937fe640 1 -- 172.21.15.7:0/3967047758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c010c20 con 0x5604950a92f0 2024-09-10T10:45:24.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 -- 172.21.15.7:0/3967047758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 msgr2=0x5604950b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 --2- 172.21.15.7:0/3967047758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b3670 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00ad00 tx=0x7fb79c00f790 comp rx=0 tx=0).stop 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 -- 172.21.15.7:0/3967047758 shutdown_connections 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 --2- 172.21.15.7:0/3967047758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 0x5604950b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 --2- 172.21.15.7:0/3967047758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b3670 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 --2- 172.21.15.7:0/3967047758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 0x5604950a8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 -- 172.21.15.7:0/3967047758 >> 172.21.15.7:0/3967047758 conn(0x5604950a4f60 msgr2=0x5604950a5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 -- 172.21.15.7:0/3967047758 shutdown_connections 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 -- 172.21.15.7:0/3967047758 wait complete. 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 -- start start 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.016+0000 7fb7b1765740 1 Processor -- start 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 0x5604950b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950b3bb0 0x5604950b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7b1765740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5604950bab90 con 0x5604950a92f0 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7b1765740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560494fc9f20 con 0x5604950b3bb0 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7b1765740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5604950bad10 con 0x5604950a89d0 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7b1765740 10 client.0 _ll_register_callbacks cb 0x560494f93030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ac927640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ad128640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 0x5604950b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ac927640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47882/0 (socket says 172.21.15.7:47882) 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ac927640 1 -- 172.21.15.7:0/551879787 learned_addr learned my addr 172.21.15.7:0/551879787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ad929640 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950b3bb0 0x5604950b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ac927640 1 -- 172.21.15.7:0/551879787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 msgr2=0x5604950b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ac927640 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 0x5604950b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ac927640 1 -- 172.21.15.7:0/551879787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950b3bb0 msgr2=0x5604950b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ac927640 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950b3bb0 0x5604950b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ac927640 1 -- 172.21.15.7:0/551879787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560494d802b0 con 0x5604950a92f0 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ad929640 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950b3bb0 0x5604950b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.017+0000 7fb7ac927640 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b07a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4000f80 tx=0x7fb7a409b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26fb5117df5b3e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb790ff9640 1 -- 172.21.15.7:0/551879787 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a409b750 con 0x5604950a92f0 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb790ff9640 1 -- 172.21.15.7:0/551879787 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fb7a40860c0 con 0x5604950a92f0 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb7b1765740 1 -- 172.21.15.7:0/551879787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb79c018b20 con 0x5604950a92f0 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb790ff9640 1 -- 172.21.15.7:0/551879787 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb7a409fb00 con 0x5604950a92f0 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb7b1765740 10 client.14952 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb7b1765740 1 -- 172.21.15.7:0/551879787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5604950c6480 con 0x5604950a92f0 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb790ff9640 1 -- 172.21.15.7:0/551879787 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb79c018b20 con 0x5604950a92f0 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb790ff9640 1 -- 172.21.15.7:0/551879787 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb7a4084ce0 con 0x5604950a92f0 2024-09-10T10:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb7b1765740 10 client.14952 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb7b1765740 10 client.14952 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb7b1765740 20 client.14952 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb7b1765740 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x5604950c8a50 0x5604950c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb7b1765740 4 client.14952 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.018+0000 7fb7b1765740 1 -- 172.21.15.7:0/551879787 --> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56049501c600 con 0x5604950c8a50 2024-09-10T10:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.019+0000 7fb7ad128640 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x5604950c8a50 0x5604950c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.019+0000 7fb790ff9640 10 client.14952 ms_handle_connect on v2:172.21.15.133:6836/3030361099 2024-09-10T10:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.019+0000 7fb7ad128640 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x5604950c8a50 0x5604950c9ec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=183bb4ad0eb71b84 server_cookie=42b6e5f1e08a5756 in_seq=0 out_seq=0 2024-09-10T10:45:24.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb790ff9640 1 -- 172.21.15.7:0/551879787 <== mds.2 v2:172.21.15.133:6836/3030361099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56049501c600 con 0x5604950c8a50 2024-09-10T10:45:24.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb790ff9640 10 client.14952 handle_command_reply: tid=0 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 client.14952 shutdown 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 -- 172.21.15.7:0/551879787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 msgr2=0x5604950b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b07a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4000f80 tx=0x7fb7a409b380 comp rx=0 tx=0).stop 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 -- 172.21.15.7:0/551879787 shutdown_connections 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950b3bb0 0x5604950b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x5604950c8a50 0x5604950c9ec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950b07a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 --2- 172.21.15.7:0/551879787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 0x5604950b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 -- 172.21.15.7:0/551879787 >> 172.21.15.7:0/551879787 conn(0x5604950a4f60 msgr2=0x5604950aac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb790ff9640 0 client.14952 ms_handle_reset on v2:172.21.15.133:6836/3030361099 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 -- 172.21.15.7:0/551879787 shutdown_connections 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 1 -- 172.21.15.7:0/551879787 wait complete. 2024-09-10T10:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.020+0000 7fb7b1765740 20 client.14952 trim_cache size 0 max 16384 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.038+0000 7fb7b1765740 1 Processor -- start 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7b1765740 1 -- start start 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 0x5604950c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a92f0 0x5604950c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950b3bb0 0x5604950b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7b1765740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5604950c8200 con 0x5604950b3bb0 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7b1765740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604950c88d0 con 0x5604950a92f0 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7b1765740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5604950c8fa0 con 0x5604950a89d0 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ad929640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950b3bb0 0x5604950b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ad929640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950b3bb0 0x5604950b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47896/0 (socket says 172.21.15.7:47896) 2024-09-10T10:45:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ad929640 1 -- 172.21.15.7:0/1531173211 learned_addr learned my addr 172.21.15.7:0/1531173211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ac927640 1 --2- 172.21.15.7:0/1531173211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a92f0 0x5604950c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ad128640 1 --2- 172.21.15.7:0/1531173211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 0x5604950c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ad929640 1 -- 172.21.15.7:0/1531173211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 msgr2=0x5604950c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ad929640 1 --2- 172.21.15.7:0/1531173211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 0x5604950c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ad929640 1 -- 172.21.15.7:0/1531173211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a92f0 msgr2=0x5604950c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ad929640 1 --2- 172.21.15.7:0/1531173211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a92f0 0x5604950c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ad929640 1 -- 172.21.15.7:0/1531173211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604950a4d90 con 0x5604950b3bb0 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.039+0000 7fb7ad128640 1 --2- 172.21.15.7:0/1531173211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 0x5604950c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.040+0000 7fb7ad929640 1 --2- 172.21.15.7:0/1531173211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950b3bb0 0x5604950b0060 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0033000 tx=0x7fb7a0005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6a4819059f0591 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.040+0000 7fb792ffd640 1 -- 172.21.15.7:0/1531173211 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0042040 con 0x5604950b3bb0 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.040+0000 7fb792ffd640 1 -- 172.21.15.7:0/1531173211 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a004a410 con 0x5604950b3bb0 2024-09-10T10:45:24.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.040+0000 7fb792ffd640 1 -- 172.21.15.7:0/1531173211 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0035040 con 0x5604950b3bb0 2024-09-10T10:45:24.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 -- 172.21.15.7:0/1531173211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950b3bb0 msgr2=0x5604950b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 --2- 172.21.15.7:0/1531173211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950b3bb0 0x5604950b0060 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0033000 tx=0x7fb7a0005b90 comp rx=0 tx=0).stop 2024-09-10T10:45:24.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 -- 172.21.15.7:0/1531173211 shutdown_connections 2024-09-10T10:45:24.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 --2- 172.21.15.7:0/1531173211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950b3bb0 0x5604950b0060 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 --2- 172.21.15.7:0/1531173211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a92f0 0x5604950c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 --2- 172.21.15.7:0/1531173211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950a89d0 0x5604950c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 -- 172.21.15.7:0/1531173211 >> 172.21.15.7:0/1531173211 conn(0x5604950a4f60 msgr2=0x5604950b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 -- 172.21.15.7:0/1531173211 shutdown_connections 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 -- 172.21.15.7:0/1531173211 wait complete. 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 -- start start 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.041+0000 7fb7b1765740 1 Processor -- start 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 0x5604950cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7b1765740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 0x560495131150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7b1765740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5604950c9d40 con 0x5604950a92f0 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7b1765740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604950c9670 con 0x5604950a89d0 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7b1765740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5604950ca410 con 0x5604950b3bb0 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7b1765740 10 client.0 _ll_register_callbacks cb 0x7fb7a40092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ac927640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ac927640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47906/0 (socket says 172.21.15.7:47906) 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ac927640 1 -- 172.21.15.7:0/1981800897 learned_addr learned my addr 172.21.15.7:0/1981800897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ad929640 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 0x560495131150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ad128640 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 0x5604950cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ac927640 1 -- 172.21.15.7:0/1981800897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 msgr2=0x560495131150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ac927640 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 0x560495131150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ac927640 1 -- 172.21.15.7:0/1981800897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 msgr2=0x5604950cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ac927640 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 0x5604950cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ac927640 1 -- 172.21.15.7:0/1981800897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560495131690 con 0x5604950a92f0 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.042+0000 7fb7ad128640 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 0x5604950cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb7ac927640 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950cb6f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009fd0 tx=0x7fb7a409b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c9e1d1aeeb1d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:24.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb790ff9640 1 -- 172.21.15.7:0/1981800897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a409b750 con 0x5604950a92f0 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb790ff9640 1 -- 172.21.15.7:0/1981800897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fb7a409aba0 con 0x5604950a92f0 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb7b1765740 1 -- 172.21.15.7:0/1981800897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a004a410 con 0x5604950a92f0 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb790ff9640 1 -- 172.21.15.7:0/1981800897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb7a409be50 con 0x5604950a92f0 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb7b1765740 10 client.14958 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb7b1765740 1 -- 172.21.15.7:0/1981800897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560495132780 con 0x5604950a92f0 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb790ff9640 1 -- 172.21.15.7:0/1981800897 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a004a410 con 0x5604950a92f0 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb790ff9640 1 -- 172.21.15.7:0/1981800897 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb7a409a610 con 0x5604950a92f0 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb7b1765740 10 client.14958 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb7b1765740 10 client.14958 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb7b1765740 20 client.14958 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb7b1765740 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x560495137170 0x560495135e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb7b1765740 4 client.14958 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:24.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.043+0000 7fb7b1765740 1 -- 172.21.15.7:0/1981800897 --> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] -- command(tid 0: {"prefix": "status"}) -- 0x560495016b10 con 0x560495137170 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.044+0000 7fb7ad128640 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x560495137170 0x560495135e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.044+0000 7fb7ad128640 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x560495137170 0x560495135e80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b498682ab15ee3de server_cookie=b46869f49469846b in_seq=0 out_seq=0 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.044+0000 7fb790ff9640 10 client.14958 ms_handle_connect on v2:172.21.15.133:6836/3030361099 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.044+0000 7fb790ff9640 1 -- 172.21.15.7:0/1981800897 <== mds.2 v2:172.21.15.133:6836/3030361099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560495016b10 con 0x560495137170 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.044+0000 7fb790ff9640 10 client.14958 handle_command_reply: tid=0 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.044+0000 7fb7b1765740 1 client.14958 shutdown 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.044+0000 7fb7b1765740 1 -- 172.21.15.7:0/1981800897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 msgr2=0x5604950cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.044+0000 7fb7b1765740 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950cb6f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009fd0 tx=0x7fb7a409b290 comp rx=0 tx=0).stop 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7b1765740 1 -- 172.21.15.7:0/1981800897 shutdown_connections 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7b1765740 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604950b3bb0 0x560495131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7b1765740 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x560495137170 0x560495135e80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7b1765740 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604950a92f0 0x5604950cb6f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7b1765740 1 --2- 172.21.15.7:0/1981800897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604950a89d0 0x5604950cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb790ff9640 0 client.14958 ms_handle_reset on v2:172.21.15.133:6836/3030361099 2024-09-10T10:45:24.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7b1765740 1 -- 172.21.15.7:0/1981800897 >> 172.21.15.7:0/1981800897 conn(0x5604950a4f60 msgr2=0x5604950af070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:24.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7b1765740 1 -- 172.21.15.7:0/1981800897 shutdown_connections 2024-09-10T10:45:24.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7b1765740 1 -- 172.21.15.7:0/1981800897 wait complete. 2024-09-10T10:45:24.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7b1765740 20 client.14958 trim_cache size 0 max 16384 2024-09-10T10:45:24.052 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T10:45:24.052 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "65af4d96-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:24.052 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-09-10T10:45:24.052 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24481, 2024-09-10T10:45:24.052 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T10:45:24.053 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T10:45:24.053 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T10:45:24.053 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 63.980381737999998, 2024-09-10T10:45:24.053 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-09-10T10:45:24.053 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 81, 2024-09-10T10:45:24.053 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 81, 2024-09-10T10:45:24.053 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 96.059837219000002 2024-09-10T10:45:24.053 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7af38c640 1 -- 172.21.15.7:0/4074006828 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7940a2c90 msgr2=0x7fb7940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7af38c640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7940a2c90 0x7fb7940a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb7a002a940 tx=0x7fb7a0005fd0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7af38c640 1 -- 172.21.15.7:0/4074006828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7a81036e0 msgr2=0x7fb7a8198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.045+0000 7fb7af38c640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7a81036e0 0x7fb7a8198a10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400db20 tx=0x7fb7a400df20 comp rx=0 tx=0).stop 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.046+0000 7fb7af38c640 1 -- 172.21.15.7:0/4074006828 shutdown_connections 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.046+0000 7fb7af38c640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fb7940a2c90 0x7fb7940a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.046+0000 7fb7af38c640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a81082a0 0x7fb7a819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.046+0000 7fb7af38c640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7a81036e0 0x7fb7a8198a10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.046+0000 7fb7af38c640 1 --2- 172.21.15.7:0/4074006828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7a8102d30 0x7fb7a81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.046+0000 7fb7af38c640 1 -- 172.21.15.7:0/4074006828 >> 172.21.15.7:0/4074006828 conn(0x7fb7a80fc7d0 msgr2=0x7fb7a8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.046+0000 7fb7af38c640 1 -- 172.21.15.7:0/4074006828 shutdown_connections 2024-09-10T10:45:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:24.046+0000 7fb7af38c640 1 -- 172.21.15.7:0/4074006828 wait complete. 2024-09-10T10:45:24.837 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:24 smithi007 ceph-mon[30873]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:24.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T10:45:24.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:24 smithi103 ceph-mon[35160]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:24.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:24 smithi133 ceph-mon[35282]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.582+0000 7f26e2c10640 1 -- 172.21.15.7:0/3886697788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc102d40 msgr2=0x7f26dc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.582+0000 7f26e2c10640 1 --2- 172.21.15.7:0/3886697788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc102d40 0x7f26dc103120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009900 tx=0x7f26d002a480 comp rx=0 tx=0).stop 2024-09-10T10:45:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.582+0000 7f26e2c10640 1 -- 172.21.15.7:0/3886697788 shutdown_connections 2024-09-10T10:45:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.582+0000 7f26e2c10640 1 --2- 172.21.15.7:0/3886697788 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc10c170 0x7f26dc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.582+0000 7f26e2c10640 1 --2- 172.21.15.7:0/3886697788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26dc103660 0x7f26dc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.582+0000 7f26e2c10640 1 --2- 172.21.15.7:0/3886697788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc102d40 0x7f26dc103120 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.582+0000 7f26e2c10640 1 -- 172.21.15.7:0/3886697788 >> 172.21.15.7:0/3886697788 conn(0x7f26dc0fc7a0 msgr2=0x7f26dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.582+0000 7f26e2c10640 1 -- 172.21.15.7:0/3886697788 shutdown_connections 2024-09-10T10:45:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.582+0000 7f26e2c10640 1 -- 172.21.15.7:0/3886697788 wait complete. 2024-09-10T10:45:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.583+0000 7f26e2c10640 1 Processor -- start 2024-09-10T10:45:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.583+0000 7f26e2c10640 1 -- start start 2024-09-10T10:45:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26e2c10640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26dc102d40 0x7f26dc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26e2c10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc103660 0x7f26dc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26e2c10640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc10c170 0x7f26dc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26e2c10640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc111f70 con 0x7f26dc103660 2024-09-10T10:45:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26e2c10640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc111df0 con 0x7f26dc10c170 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26e2c10640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1120f0 con 0x7f26dc102d40 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26dbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc103660 0x7f26dc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26dbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc103660 0x7f26dc100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47920/0 (socket says 172.21.15.7:47920) 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26dbfff640 1 -- 172.21.15.7:0/3091041356 learned_addr learned my addr 172.21.15.7:0/3091041356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26e09ac640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26dc102d40 0x7f26dc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26e11ad640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc10c170 0x7f26dc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26dbfff640 1 -- 172.21.15.7:0/3091041356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26dc102d40 msgr2=0x7f26dc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26dbfff640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26dc102d40 0x7f26dc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26dbfff640 1 -- 172.21.15.7:0/3091041356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc10c170 msgr2=0x7f26dc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26dbfff640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc10c170 0x7f26dc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26dbfff640 1 -- 172.21.15.7:0/3091041356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc1a6810 con 0x7f26dc103660 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26e09ac640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26dc102d40 0x7f26dc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26dbfff640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc103660 0x7f26dc100ee0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c970 tx=0x7f26c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26d9ffb640 1 -- 172.21.15.7:0/3091041356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8007bb0 con 0x7f26dc103660 2024-09-10T10:45:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26d9ffb640 1 -- 172.21.15.7:0/3091041356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8007d50 con 0x7f26dc103660 2024-09-10T10:45:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.584+0000 7f26e2c10640 1 -- 172.21.15.7:0/3091041356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc1a6b00 con 0x7f26dc103660 2024-09-10T10:45:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.585+0000 7f26d9ffb640 1 -- 172.21.15.7:0/3091041356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8018820 con 0x7f26dc103660 2024-09-10T10:45:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.585+0000 7f26e2c10640 1 -- 172.21.15.7:0/3091041356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc1a7090 con 0x7f26dc103660 2024-09-10T10:45:25.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.587+0000 7f26d9ffb640 1 -- 172.21.15.7:0/3091041356 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f26c8018a80 con 0x7f26dc103660 2024-09-10T10:45:25.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.587+0000 7f26e4fe9740 1 Processor -- start 2024-09-10T10:45:25.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.587+0000 7f26e4fe9740 1 -- start start 2024-09-10T10:45:25.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.587+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a0fc9d0 0x56350a0fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.587+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a107670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.587+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a107bb0 0x56350a109fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.587+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56350a063090 con 0x56350a0fd2f0 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.587+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56350a00aa70 con 0x56350a0fc9d0 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.587+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56350a05f050 con 0x56350a107bb0 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e09ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a107670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26d9ffb640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f26cc0a2c90 0x7f26cc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e09ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a107670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47930/0 (socket says 172.21.15.7:47930) 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e09ac640 1 -- 172.21.15.7:0/663863640 learned_addr learned my addr 172.21.15.7:0/663863640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e11ad640 1 --2- 172.21.15.7:0/663863640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a0fc9d0 0x56350a0fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e11ad640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f26cc0a2c90 0x7f26cc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26dbfff640 1 --2- 172.21.15.7:0/663863640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a107bb0 0x56350a109fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26d9ffb640 1 -- 172.21.15.7:0/3091041356 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f26c80cf2d0 con 0x7f26dc103660 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e09ac640 1 -- 172.21.15.7:0/663863640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a107bb0 msgr2=0x56350a109fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e09ac640 1 --2- 172.21.15.7:0/663863640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a107bb0 0x56350a109fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e09ac640 1 -- 172.21.15.7:0/663863640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a0fc9d0 msgr2=0x56350a0fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e09ac640 1 --2- 172.21.15.7:0/663863640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a0fc9d0 0x56350a0fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e09ac640 1 -- 172.21.15.7:0/663863640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350a017130 con 0x56350a0fd2f0 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e09ac640 1 --2- 172.21.15.7:0/663863640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a107670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009900 tx=0x7f26d00316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81edb30a3188b48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.588+0000 7f26e11ad640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f26cc0a2c90 0x7f26cc0a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f26d400bfd0 tx=0x7f26d4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26c6ffd640 1 -- 172.21.15.7:0/663863640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d003f070 con 0x56350a0fd2f0 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26c6ffd640 1 -- 172.21.15.7:0/663863640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d00035f0 con 0x56350a0fd2f0 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26c6ffd640 1 -- 172.21.15.7:0/663863640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0035070 con 0x56350a0fd2f0 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26e4fe9740 1 -- 172.21.15.7:0/663863640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 msgr2=0x56350a107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/663863640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a107670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009900 tx=0x7f26d00316e0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26e4fe9740 1 -- 172.21.15.7:0/663863640 shutdown_connections 2024-09-10T10:45:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/663863640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a107bb0 0x56350a109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/663863640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a107670 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/663863640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a0fc9d0 0x56350a0fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26e4fe9740 1 -- 172.21.15.7:0/663863640 >> 172.21.15.7:0/663863640 conn(0x56350a0f8f60 msgr2=0x56350a0f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26e4fe9740 1 -- 172.21.15.7:0/663863640 shutdown_connections 2024-09-10T10:45:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.589+0000 7f26e4fe9740 1 -- 172.21.15.7:0/663863640 wait complete. 2024-09-10T10:45:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e4fe9740 1 -- start start 2024-09-10T10:45:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e4fe9740 1 Processor -- start 2024-09-10T10:45:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a107460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fd2f0 0x56350a1047b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 0x56350a104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56350a10a780 con 0x56350a0fc9d0 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56350a01df20 con 0x56350a107bb0 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56350a10a900 con 0x56350a0fd2f0 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e4fe9740 10 client.0 _ll_register_callbacks cb 0x563509fe7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e09ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e09ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47938/0 (socket says 172.21.15.7:47938) 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.590+0000 7f26e09ac640 1 -- 172.21.15.7:0/133435227 learned_addr learned my addr 172.21.15.7:0/133435227 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26e11ad640 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 0x56350a104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26dbfff640 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fd2f0 0x56350a1047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26e09ac640 1 -- 172.21.15.7:0/133435227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fd2f0 msgr2=0x56350a1047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26e09ac640 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fd2f0 0x56350a1047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26e09ac640 1 -- 172.21.15.7:0/133435227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 msgr2=0x56350a104cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26e09ac640 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 0x56350a104cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26e09ac640 1 -- 172.21.15.7:0/133435227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563509dd42b0 con 0x56350a0fc9d0 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26dbfff640 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fd2f0 0x56350a1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26e09ac640 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a107460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f26d0005940 tx=0x7f26d0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9483ef4c768ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26bbfff640 1 -- 172.21.15.7:0/133435227 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d004a020 con 0x56350a0fc9d0 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26bbfff640 1 -- 172.21.15.7:0/133435227 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f26d003b430 con 0x56350a0fc9d0 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26e4fe9740 1 -- 172.21.15.7:0/133435227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d00035f0 con 0x56350a0fc9d0 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26bbfff640 1 -- 172.21.15.7:0/133435227 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26d003f070 con 0x56350a0fc9d0 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26e4fe9740 10 client.14970 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.591+0000 7f26e4fe9740 1 -- 172.21.15.7:0/133435227 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56350a1160d0 con 0x56350a0fc9d0 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.592+0000 7f26bbfff640 1 -- 172.21.15.7:0/133435227 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d00035f0 con 0x56350a0fc9d0 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.592+0000 7f26bbfff640 1 -- 172.21.15.7:0/133435227 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f26d0043760 con 0x56350a0fc9d0 2024-09-10T10:45:25.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.592+0000 7f26e4fe9740 10 client.14970 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.592+0000 7f26e4fe9740 10 client.14970 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.592+0000 7f26e4fe9740 20 client.14970 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.592+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x56350a1181d0 0x56350a119640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.592+0000 7f26e4fe9740 4 client.14970 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.592+0000 7f26e4fe9740 1 -- 172.21.15.7:0/133435227 --> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56350a070600 con 0x56350a1181d0 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.592+0000 7f26dbfff640 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x56350a1181d0 0x56350a119640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.593+0000 7f26dbfff640 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x56350a1181d0 0x56350a119640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f1ffdb39e133615 server_cookie=f83c4a541b7a490b in_seq=0 out_seq=0 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.593+0000 7f26bbfff640 10 client.14970 ms_handle_connect on v2:172.21.15.103:6838/2537001809 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.593+0000 7f26bbfff640 1 -- 172.21.15.7:0/133435227 <== mds.1 v2:172.21.15.103:6838/2537001809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56350a070600 con 0x56350a1181d0 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.593+0000 7f26bbfff640 10 client.14970 handle_command_reply: tid=0 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.593+0000 7f26e4fe9740 1 client.14970 shutdown 2024-09-10T10:45:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 1 -- 172.21.15.7:0/133435227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 msgr2=0x56350a107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a107460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f26d0005940 tx=0x7f26d0003c80 comp rx=0 tx=0).stop 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 1 -- 172.21.15.7:0/133435227 shutdown_connections 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 0x56350a104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x56350a1181d0 0x56350a119640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fd2f0 0x56350a1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/133435227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a107460 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 1 -- 172.21.15.7:0/133435227 >> 172.21.15.7:0/133435227 conn(0x56350a0f8f60 msgr2=0x56350a0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26bbfff640 0 client.14970 ms_handle_reset on v2:172.21.15.103:6838/2537001809 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 1 -- 172.21.15.7:0/133435227 shutdown_connections 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 1 -- 172.21.15.7:0/133435227 wait complete. 2024-09-10T10:45:25.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.594+0000 7f26e4fe9740 20 client.14970 trim_cache size 0 max 16384 2024-09-10T10:45:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e4fe9740 1 Processor -- start 2024-09-10T10:45:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e4fe9740 1 -- start start 2024-09-10T10:45:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fc9d0 0x56350a1188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a0ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 0x56350a101a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56350a117980 con 0x56350a0fd2f0 2024-09-10T10:45:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56350a118050 con 0x56350a107bb0 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56350a118720 con 0x56350a0fc9d0 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26dbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a0ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26dbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a0ffc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47942/0 (socket says 172.21.15.7:47942) 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26dbfff640 1 -- 172.21.15.7:0/286579923 learned_addr learned my addr 172.21.15.7:0/286579923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e11ad640 1 --2- 172.21.15.7:0/286579923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 0x56350a101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e09ac640 1 --2- 172.21.15.7:0/286579923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fc9d0 0x56350a1188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26dbfff640 1 -- 172.21.15.7:0/286579923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fc9d0 msgr2=0x56350a1188a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26dbfff640 1 --2- 172.21.15.7:0/286579923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fc9d0 0x56350a1188a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26dbfff640 1 -- 172.21.15.7:0/286579923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 msgr2=0x56350a101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26dbfff640 1 --2- 172.21.15.7:0/286579923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 0x56350a101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26dbfff640 1 -- 172.21.15.7:0/286579923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350a0f8d90 con 0x56350a0fd2f0 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.612+0000 7f26e11ad640 1 --2- 172.21.15.7:0/286579923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 0x56350a101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26dbfff640 1 --2- 172.21.15.7:0/286579923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a0ffc70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f26c8097de0 tx=0x7f26c8013b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1ab84602088778 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26c67fc640 1 -- 172.21.15.7:0/286579923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c80a45f0 con 0x56350a0fd2f0 2024-09-10T10:45:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26c67fc640 1 -- 172.21.15.7:0/286579923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8099a20 con 0x56350a0fd2f0 2024-09-10T10:45:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26c67fc640 1 -- 172.21.15.7:0/286579923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8097ff0 con 0x56350a0fd2f0 2024-09-10T10:45:25.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26e4fe9740 1 -- 172.21.15.7:0/286579923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 msgr2=0x56350a0ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:25.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/286579923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a0ffc70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f26c8097de0 tx=0x7f26c8013b80 comp rx=0 tx=0).stop 2024-09-10T10:45:25.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26e4fe9740 1 -- 172.21.15.7:0/286579923 shutdown_connections 2024-09-10T10:45:25.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/286579923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a107bb0 0x56350a101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/286579923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fd2f0 0x56350a0ffc70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/286579923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a0fc9d0 0x56350a1188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.613+0000 7f26e4fe9740 1 -- 172.21.15.7:0/286579923 >> 172.21.15.7:0/286579923 conn(0x56350a0f8f60 msgr2=0x56350a0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 1 -- 172.21.15.7:0/286579923 shutdown_connections 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 1 -- 172.21.15.7:0/286579923 wait complete. 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 1 -- start start 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 1 Processor -- start 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a11aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a0fd2f0 0x56350a11b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a107bb0 0x56350a1808d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56350a1194c0 con 0x56350a0fc9d0 2024-09-10T10:45:25.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56350a118df0 con 0x56350a0fd2f0 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56350a119b90 con 0x56350a107bb0 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e4fe9740 10 client.0 _ll_register_callbacks cb 0x7f26d400bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.614+0000 7f26e09ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a11aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e09ac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a11aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47944/0 (socket says 172.21.15.7:47944) 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e09ac640 1 -- 172.21.15.7:0/3884728564 learned_addr learned my addr 172.21.15.7:0/3884728564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e11ad640 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a107bb0 0x56350a1808d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26dbfff640 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a0fd2f0 0x56350a11b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e09ac640 1 -- 172.21.15.7:0/3884728564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a107bb0 msgr2=0x56350a1808d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e09ac640 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a107bb0 0x56350a1808d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e09ac640 1 -- 172.21.15.7:0/3884728564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a0fd2f0 msgr2=0x56350a11b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e09ac640 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a0fd2f0 0x56350a11b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e09ac640 1 -- 172.21.15.7:0/3884728564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350a180e10 con 0x56350a0fc9d0 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e09ac640 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a11aeb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26d0053040 tx=0x7f26d002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ffc8e7bfc13078 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26bbfff640 1 -- 172.21.15.7:0/3884728564 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d004a040 con 0x56350a0fc9d0 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e4fe9740 1 -- 172.21.15.7:0/3884728564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c8099a20 con 0x56350a0fc9d0 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26bbfff640 1 -- 172.21.15.7:0/3884728564 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f26d0005230 con 0x56350a0fc9d0 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26bbfff640 1 -- 172.21.15.7:0/3884728564 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26d0035040 con 0x56350a0fc9d0 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e4fe9740 10 client.14976 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.615+0000 7f26e4fe9740 1 -- 172.21.15.7:0/3884728564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56350a182110 con 0x56350a0fc9d0 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.616+0000 7f26bbfff640 1 -- 172.21.15.7:0/3884728564 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8099a20 con 0x56350a0fc9d0 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.616+0000 7f26bbfff640 1 -- 172.21.15.7:0/3884728564 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f26d0058020 con 0x56350a0fc9d0 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.616+0000 7f26e4fe9740 10 client.14976 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.616+0000 7f26e4fe9740 10 client.14976 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.616+0000 7f26e4fe9740 20 client.14976 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.616+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x56350a186c40 0x56350a185950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:25.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.616+0000 7f26e4fe9740 4 client.14976 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.616+0000 7f26e4fe9740 1 -- 172.21.15.7:0/3884728564 --> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] -- command(tid 0: {"prefix": "status"}) -- 0x56350a06ab10 con 0x56350a186c40 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.616+0000 7f26dbfff640 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x56350a186c40 0x56350a185950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26dbfff640 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x56350a186c40 0x56350a185950 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0a40dd802855169 server_cookie=6d25ae66d524d6fa in_seq=0 out_seq=0 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26bbfff640 10 client.14976 ms_handle_connect on v2:172.21.15.103:6838/2537001809 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26bbfff640 1 -- 172.21.15.7:0/3884728564 <== mds.1 v2:172.21.15.103:6838/2537001809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56350a06ab10 con 0x56350a186c40 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26bbfff640 10 client.14976 handle_command_reply: tid=0 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 client.14976 shutdown 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 -- 172.21.15.7:0/3884728564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 msgr2=0x56350a11aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a11aeb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26d0053040 tx=0x7f26d002a480 comp rx=0 tx=0).stop 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 -- 172.21.15.7:0/3884728564 shutdown_connections 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x56350a186c40 0x56350a185950 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56350a107bb0 0x56350a1808d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56350a0fd2f0 0x56350a11b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 --2- 172.21.15.7:0/3884728564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56350a0fc9d0 0x56350a11aeb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26bbfff640 0 client.14976 ms_handle_reset on v2:172.21.15.103:6838/2537001809 2024-09-10T10:45:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 -- 172.21.15.7:0/3884728564 >> 172.21.15.7:0/3884728564 conn(0x56350a0f8f60 msgr2=0x56350a0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 -- 172.21.15.7:0/3884728564 shutdown_connections 2024-09-10T10:45:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 1 -- 172.21.15.7:0/3884728564 wait complete. 2024-09-10T10:45:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.617+0000 7f26e4fe9740 20 client.14976 trim_cache size 0 max 16384 2024-09-10T10:45:25.625 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T10:45:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "65af4d96-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-09-10T10:45:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24482, 2024-09-10T10:45:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T10:45:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T10:45:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T10:45:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 66.559036947999999, 2024-09-10T10:45:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-09-10T10:45:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 81, 2024-09-10T10:45:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 81, 2024-09-10T10:45:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 84.393121821999998 2024-09-10T10:45:25.626 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T10:45:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 -- 172.21.15.7:0/3091041356 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f26cc0a2c90 msgr2=0x7f26cc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f26cc0a2c90 0x7f26cc0a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f26d400bfd0 tx=0x7f26d4004a40 comp rx=0 tx=0).stop 2024-09-10T10:45:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 -- 172.21.15.7:0/3091041356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc103660 msgr2=0x7f26dc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc103660 0x7f26dc100ee0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c970 tx=0x7f26c800ce30 comp rx=0 tx=0).stop 2024-09-10T10:45:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 -- 172.21.15.7:0/3091041356 shutdown_connections 2024-09-10T10:45:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f26cc0a2c90 0x7f26cc0a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26dc10c170 0x7f26dc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc103660 0x7f26dc100ee0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 --2- 172.21.15.7:0/3091041356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26dc102d40 0x7f26dc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 -- 172.21.15.7:0/3091041356 >> 172.21.15.7:0/3091041356 conn(0x7f26dc0fc7a0 msgr2=0x7f26dc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 -- 172.21.15.7:0/3091041356 shutdown_connections 2024-09-10T10:45:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:25.618+0000 7f26e2c10640 1 -- 172.21.15.7:0/3091041356 wait complete. 2024-09-10T10:45:26.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T10:45:26.577 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:26.813 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:26 smithi007 ceph-mon[30873]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:26.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:26 smithi103 ceph-mon[35160]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:26.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:26 smithi133 ceph-mon[35282]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.024+0000 7f82dc99b640 1 -- 172.21.15.7:0/3206103015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d8106f50 msgr2=0x7f82d810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.024+0000 7f82dc99b640 1 --2- 172.21.15.7:0/3206103015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d8106f50 0x7f82d810eaf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009930 tx=0x7f82d002a420 comp rx=0 tx=0).stop 2024-09-10T10:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.024+0000 7f82dc99b640 1 -- 172.21.15.7:0/3206103015 shutdown_connections 2024-09-10T10:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.024+0000 7f82dc99b640 1 --2- 172.21.15.7:0/3206103015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d8106f50 0x7f82d810eaf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.024+0000 7f82dc99b640 1 --2- 172.21.15.7:0/3206103015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8100d00 0x7f82d8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.024+0000 7f82dc99b640 1 --2- 172.21.15.7:0/3206103015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82d8100350 0x7f82d8100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.024+0000 7f82dc99b640 1 -- 172.21.15.7:0/3206103015 >> 172.21.15.7:0/3206103015 conn(0x7f82d8077210 msgr2=0x7f82d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.024+0000 7f82dc99b640 1 -- 172.21.15.7:0/3206103015 shutdown_connections 2024-09-10T10:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.024+0000 7f82dc99b640 1 -- 172.21.15.7:0/3206103015 wait complete. 2024-09-10T10:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82dc99b640 1 Processor -- start 2024-09-10T10:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82dc99b640 1 -- start start 2024-09-10T10:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82dc99b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d8100350 0x7f82d8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82dc99b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8100d00 0x7f82d8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82dc99b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82d8106f50 0x7f82d819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82dc99b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8112000 con 0x7f82d8100350 2024-09-10T10:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82dc99b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8111e80 con 0x7f82d8106f50 2024-09-10T10:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82dc99b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8112180 con 0x7f82d8100d00 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d8100350 0x7f82d8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d8100350 0x7f82d8198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47966/0 (socket says 172.21.15.7:47966) 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82d659c640 1 -- 172.21.15.7:0/2851608971 learned_addr learned my addr 172.21.15.7:0/2851608971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.025+0000 7f82d6d9d640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82d8106f50 0x7f82d819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82d5d9b640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8100d00 0x7f82d8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82d659c640 1 -- 172.21.15.7:0/2851608971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8100d00 msgr2=0x7f82d8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82d659c640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8100d00 0x7f82d8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82d659c640 1 -- 172.21.15.7:0/2851608971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82d8106f50 msgr2=0x7f82d819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82d659c640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82d8106f50 0x7f82d819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82d659c640 1 -- 172.21.15.7:0/2851608971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d819d400 con 0x7f82d8100350 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82d659c640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d8100350 0x7f82d8198450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f82c800c920 tx=0x7f82c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82c77fe640 1 -- 172.21.15.7:0/2851608971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8007c70 con 0x7f82d8100350 2024-09-10T10:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82c77fe640 1 -- 172.21.15.7:0/2851608971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c8007e10 con 0x7f82d8100350 2024-09-10T10:45:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82dc99b640 1 -- 172.21.15.7:0/2851608971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d819d6f0 con 0x7f82d8100350 2024-09-10T10:45:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82c77fe640 1 -- 172.21.15.7:0/2851608971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8018890 con 0x7f82d8100350 2024-09-10T10:45:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.026+0000 7f82dc99b640 1 -- 172.21.15.7:0/2851608971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d81a4fd0 con 0x7f82d8100350 2024-09-10T10:45:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.029+0000 7f82c77fe640 1 -- 172.21.15.7:0/2851608971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f82c8018a70 con 0x7f82d8100350 2024-09-10T10:45:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.029+0000 7f82ded74740 1 Processor -- start 2024-09-10T10:45:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.029+0000 7f82ded74740 1 -- start start 2024-09-10T10:45:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.029+0000 7f82ded74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x56518433adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.029+0000 7f82ded74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184345670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.029+0000 7f82ded74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565184345bb0 0x565184347fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.029+0000 7f82ded74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5651842a1090 con 0x56518433b2f0 2024-09-10T10:45:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.029+0000 7f82ded74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565184248a70 con 0x56518433a9d0 2024-09-10T10:45:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.029+0000 7f82ded74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56518429d050 con 0x565184345bb0 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.029+0000 7f82d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184345670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184345670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47974/0 (socket says 172.21.15.7:47974) 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d659c640 1 -- 172.21.15.7:0/3727054364 learned_addr learned my addr 172.21.15.7:0/3727054364 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d6d9d640 1 --2- 172.21.15.7:0/3727054364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x56518433adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d5d9b640 1 --2- 172.21.15.7:0/3727054364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565184345bb0 0x565184347fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82c77fe640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82c00a2c10 0x7f82c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d659c640 1 -- 172.21.15.7:0/3727054364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565184345bb0 msgr2=0x565184347fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d659c640 1 --2- 172.21.15.7:0/3727054364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565184345bb0 0x565184347fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d659c640 1 -- 172.21.15.7:0/3727054364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 msgr2=0x56518433adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d659c640 1 --2- 172.21.15.7:0/3727054364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x56518433adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d659c640 1 -- 172.21.15.7:0/3727054364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565184255130 con 0x56518433b2f0 2024-09-10T10:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d6d9d640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82c00a2c10 0x7f82c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82c77fe640 1 -- 172.21.15.7:0/2851608971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f82c80cf310 con 0x7f82d8100350 2024-09-10T10:45:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d659c640 1 --2- 172.21.15.7:0/3727054364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184345670 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f82c80ccc40 tx=0x7f82c8097af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28a0fadc1f25d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82c4ff9640 1 -- 172.21.15.7:0/3727054364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c809b7f0 con 0x56518433b2f0 2024-09-10T10:45:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.030+0000 7f82d6d9d640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82c00a2c10 0x7f82c00a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f82d0031550 tx=0x7f82d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82c4ff9640 1 -- 172.21.15.7:0/3727054364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c809c120 con 0x56518433b2f0 2024-09-10T10:45:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82c4ff9640 1 -- 172.21.15.7:0/3727054364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c809b380 con 0x56518433b2f0 2024-09-10T10:45:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82ded74740 1 -- 172.21.15.7:0/3727054364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 msgr2=0x565184345670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82ded74740 1 --2- 172.21.15.7:0/3727054364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184345670 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f82c80ccc40 tx=0x7f82c8097af0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82ded74740 1 -- 172.21.15.7:0/3727054364 shutdown_connections 2024-09-10T10:45:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82ded74740 1 --2- 172.21.15.7:0/3727054364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565184345bb0 0x565184347fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82ded74740 1 --2- 172.21.15.7:0/3727054364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184345670 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82ded74740 1 --2- 172.21.15.7:0/3727054364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x56518433adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82ded74740 1 -- 172.21.15.7:0/3727054364 >> 172.21.15.7:0/3727054364 conn(0x565184336f60 msgr2=0x565184337340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82ded74740 1 -- 172.21.15.7:0/3727054364 shutdown_connections 2024-09-10T10:45:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.031+0000 7f82ded74740 1 -- 172.21.15.7:0/3727054364 wait complete. 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82ded74740 1 -- start start 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82ded74740 1 Processor -- start 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82ded74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x565184345450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82ded74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433b2f0 0x5651843427a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82ded74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565184345bb0 0x565184342ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82ded74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565184348780 con 0x565184345bb0 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82ded74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56518425bf20 con 0x56518433a9d0 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82ded74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x565184348900 con 0x56518433b2f0 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82ded74740 10 client.0 _ll_register_callbacks cb 0x565184225030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82d6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565184345bb0 0x565184342ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82d6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565184345bb0 0x565184342ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47984/0 (socket says 172.21.15.7:47984) 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82d6d9d640 1 -- 172.21.15.7:0/3133048707 learned_addr learned my addr 172.21.15.7:0/3133048707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:27.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.032+0000 7f82d659c640 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x565184345450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82d5d9b640 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433b2f0 0x5651843427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82d6d9d640 1 -- 172.21.15.7:0/3133048707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433b2f0 msgr2=0x5651843427a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82d6d9d640 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433b2f0 0x5651843427a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82d6d9d640 1 -- 172.21.15.7:0/3133048707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 msgr2=0x565184345450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82d6d9d640 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x565184345450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82d6d9d640 1 -- 172.21.15.7:0/3133048707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651840122b0 con 0x565184345bb0 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82d6d9d640 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565184345bb0 0x565184342ce0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f82d0033000 tx=0x7f82d002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb23cdaeaac67aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82bdffb640 1 -- 172.21.15.7:0/3133048707 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d004d020 con 0x565184345bb0 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82bdffb640 1 -- 172.21.15.7:0/3133048707 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f82d003e480 con 0x565184345bb0 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82ded74740 1 -- 172.21.15.7:0/3133048707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c809c120 con 0x565184345bb0 2024-09-10T10:45:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82bdffb640 1 -- 172.21.15.7:0/3133048707 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f82d0035040 con 0x565184345bb0 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82ded74740 10 client.14988 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82ded74740 1 -- 172.21.15.7:0/3133048707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651843540a0 con 0x565184345bb0 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.033+0000 7f82bdffb640 1 -- 172.21.15.7:0/3133048707 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c809c120 con 0x565184345bb0 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.034+0000 7f82bdffb640 1 -- 172.21.15.7:0/3133048707 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f82d003c570 con 0x565184345bb0 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.034+0000 7f82ded74740 10 client.14988 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.034+0000 7f82ded74740 10 client.14988 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.034+0000 7f82ded74740 20 client.14988 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.034+0000 7f82ded74740 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5651843567a0 0x565184357c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.034+0000 7f82ded74740 4 client.14988 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.034+0000 7f82ded74740 1 -- 172.21.15.7:0/3133048707 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651842ae600 con 0x5651843567a0 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.034+0000 7f82d659c640 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5651843567a0 0x565184357c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.034+0000 7f82d659c640 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5651843567a0 0x565184357c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b4fb2060d63b7 server_cookie=1b7c050de86ffde4 in_seq=0 out_seq=0 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.034+0000 7f82bdffb640 10 client.14988 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82bdffb640 1 -- 172.21.15.7:0/3133048707 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651842ae600 con 0x5651843567a0 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82bdffb640 10 client.14988 handle_command_reply: tid=0 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82ded74740 1 client.14988 shutdown 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82ded74740 1 -- 172.21.15.7:0/3133048707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565184345bb0 msgr2=0x565184342ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82ded74740 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565184345bb0 0x565184342ce0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f82d0033000 tx=0x7f82d002a420 comp rx=0 tx=0).stop 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82ded74740 1 -- 172.21.15.7:0/3133048707 shutdown_connections 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82ded74740 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5651843567a0 0x565184357c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82ded74740 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565184345bb0 0x565184342ce0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82ded74740 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433b2f0 0x5651843427a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82ded74740 1 --2- 172.21.15.7:0/3133048707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x565184345450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82ded74740 1 -- 172.21.15.7:0/3133048707 >> 172.21.15.7:0/3133048707 conn(0x565184336f60 msgr2=0x56518433cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:27.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.035+0000 7f82bdffb640 0 client.14988 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:45:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.036+0000 7f82ded74740 1 -- 172.21.15.7:0/3133048707 shutdown_connections 2024-09-10T10:45:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.036+0000 7f82ded74740 1 -- 172.21.15.7:0/3133048707 wait complete. 2024-09-10T10:45:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.036+0000 7f82ded74740 20 client.14988 trim_cache size 0 max 16384 2024-09-10T10:45:27.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.053+0000 7f82ded74740 1 Processor -- start 2024-09-10T10:45:27.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.053+0000 7f82ded74740 1 -- start start 2024-09-10T10:45:27.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.053+0000 7f82ded74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433a9d0 0x565184356e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.053+0000 7f82ded74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184342760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.053+0000 7f82ded74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565184345bb0 0x565184342ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.053+0000 7f82ded74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565184355f50 con 0x56518433b2f0 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.053+0000 7f82ded74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565184356620 con 0x565184345bb0 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.053+0000 7f82ded74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x565184356cf0 con 0x56518433a9d0 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184342760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184342760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47994/0 (socket says 172.21.15.7:47994) 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d5d9b640 1 -- 172.21.15.7:0/3089480118 learned_addr learned my addr 172.21.15.7:0/3089480118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d6d9d640 1 --2- 172.21.15.7:0/3089480118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565184345bb0 0x565184342ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d659c640 1 --2- 172.21.15.7:0/3089480118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433a9d0 0x565184356e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d5d9b640 1 -- 172.21.15.7:0/3089480118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433a9d0 msgr2=0x565184356e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d5d9b640 1 --2- 172.21.15.7:0/3089480118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433a9d0 0x565184356e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d5d9b640 1 -- 172.21.15.7:0/3089480118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565184345bb0 msgr2=0x565184342ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d5d9b640 1 --2- 172.21.15.7:0/3089480118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565184345bb0 0x565184342ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d5d9b640 1 -- 172.21.15.7:0/3089480118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651843380f0 con 0x56518433b2f0 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d659c640 1 --2- 172.21.15.7:0/3089480118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433a9d0 0x565184356e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82d5d9b640 1 --2- 172.21.15.7:0/3089480118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184342760 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f82d806a0a0 tx=0x7f82cc00e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42815fe76da4ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82bffff640 1 -- 172.21.15.7:0/3089480118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc01c070 con 0x56518433b2f0 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82bffff640 1 -- 172.21.15.7:0/3089480118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc016cb0 con 0x56518433b2f0 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.054+0000 7f82bffff640 1 -- 172.21.15.7:0/3089480118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc015320 con 0x56518433b2f0 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 -- 172.21.15.7:0/3089480118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 msgr2=0x565184342760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 --2- 172.21.15.7:0/3089480118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184342760 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f82d806a0a0 tx=0x7f82cc00e040 comp rx=0 tx=0).stop 2024-09-10T10:45:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 -- 172.21.15.7:0/3089480118 shutdown_connections 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 --2- 172.21.15.7:0/3089480118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565184345bb0 0x565184342ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 --2- 172.21.15.7:0/3089480118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184342760 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 --2- 172.21.15.7:0/3089480118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56518433a9d0 0x565184356e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 -- 172.21.15.7:0/3089480118 >> 172.21.15.7:0/3089480118 conn(0x565184336f60 msgr2=0x565184331d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 -- 172.21.15.7:0/3089480118 shutdown_connections 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 -- 172.21.15.7:0/3089480118 wait complete. 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 -- start start 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 Processor -- start 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x565184358e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184359350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565184345bb0 0x565184359890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565184357a90 con 0x56518433b2f0 2024-09-10T10:45:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651843573c0 con 0x56518433a9d0 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.055+0000 7f82ded74740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x565184358160 con 0x565184345bb0 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82ded74740 10 client.0 _ll_register_callbacks cb 0x7f82d0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184359350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184359350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48006/0 (socket says 172.21.15.7:48006) 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d5d9b640 1 -- 172.21.15.7:0/4196436424 learned_addr learned my addr 172.21.15.7:0/4196436424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d659c640 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x565184358e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d6d9d640 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565184345bb0 0x565184359890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d5d9b640 1 -- 172.21.15.7:0/4196436424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565184345bb0 msgr2=0x565184359890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d5d9b640 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565184345bb0 0x565184359890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d5d9b640 1 -- 172.21.15.7:0/4196436424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 msgr2=0x565184358e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d5d9b640 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x565184358e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d5d9b640 1 -- 172.21.15.7:0/4196436424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651843beed0 con 0x56518433b2f0 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d659c640 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x565184358e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82d5d9b640 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184359350 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f82cc004a80 tx=0x7f82cc017990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f39184bfdf0ae1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82bdffb640 1 -- 172.21.15.7:0/4196436424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc01c050 con 0x56518433b2f0 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82bdffb640 1 -- 172.21.15.7:0/4196436424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f82cc0165d0 con 0x56518433b2f0 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.056+0000 7f82ded74740 1 -- 172.21.15.7:0/4196436424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82cc016cb0 con 0x56518433b2f0 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82bdffb640 1 -- 172.21.15.7:0/4196436424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f82cc0167d0 con 0x56518433b2f0 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82ded74740 10 client.14994 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82ded74740 1 -- 172.21.15.7:0/4196436424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651843c0140 con 0x56518433b2f0 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82bdffb640 1 -- 172.21.15.7:0/4196436424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc016cb0 con 0x56518433b2f0 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82bdffb640 1 -- 172.21.15.7:0/4196436424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f82cc02b3b0 con 0x56518433b2f0 2024-09-10T10:45:27.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82ded74740 10 client.14994 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:27.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82ded74740 10 client.14994 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:45:27.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82ded74740 20 client.14994 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:27.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82ded74740 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5651843c4970 0x5651843c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:27.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82ded74740 4 client.14994 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:27.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82ded74740 1 -- 172.21.15.7:0/4196436424 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "status"}) -- 0x5651842a8b10 con 0x5651843c4970 2024-09-10T10:45:27.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.057+0000 7f82d659c640 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5651843c4970 0x5651843c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:27.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82d659c640 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5651843c4970 0x5651843c3680 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295b659f396c6aae server_cookie=81a7fb7df0423b50 in_seq=0 out_seq=0 2024-09-10T10:45:27.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82bdffb640 10 client.14994 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:45:27.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82bdffb640 1 -- 172.21.15.7:0/4196436424 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5651842a8b10 con 0x5651843c4970 2024-09-10T10:45:27.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82bdffb640 10 client.14994 handle_command_reply: tid=0 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 client.14994 shutdown 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 -- 172.21.15.7:0/4196436424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 msgr2=0x565184359350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184359350 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f82cc004a80 tx=0x7f82cc017990 comp rx=0 tx=0).stop 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 -- 172.21.15.7:0/4196436424 shutdown_connections 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5651843c4970 0x5651843c3680 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565184345bb0 0x565184359890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56518433b2f0 0x565184359350 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82bdffb640 0 client.14994 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 --2- 172.21.15.7:0/4196436424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56518433a9d0 0x565184358e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 -- 172.21.15.7:0/4196436424 >> 172.21.15.7:0/4196436424 conn(0x565184336f60 msgr2=0x565184342760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 -- 172.21.15.7:0/4196436424 shutdown_connections 2024-09-10T10:45:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.058+0000 7f82ded74740 1 -- 172.21.15.7:0/4196436424 wait complete. 2024-09-10T10:45:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82ded74740 20 client.14994 trim_cache size 0 max 16384 2024-09-10T10:45:27.068 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T10:45:27.068 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "65af4d96-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:27.068 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T10:45:27.068 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24499, 2024-09-10T10:45:27.068 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T10:45:27.068 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T10:45:27.068 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T10:45:27.068 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 75.037953305000002, 2024-09-10T10:45:27.069 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-09-10T10:45:27.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 81, 2024-09-10T10:45:27.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 81, 2024-09-10T10:45:27.069 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 94.263359686000001 2024-09-10T10:45:27.069 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T10:45:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 -- 172.21.15.7:0/2851608971 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82c00a2c10 msgr2=0x7f82c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82c00a2c10 0x7f82c00a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f82d0031550 tx=0x7f82d0009260 comp rx=0 tx=0).stop 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 -- 172.21.15.7:0/2851608971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d8100350 msgr2=0x7f82d8198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d8100350 0x7f82d8198450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f82c800c920 tx=0x7f82c800cde0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 -- 172.21.15.7:0/2851608971 shutdown_connections 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f82c00a2c10 0x7f82c00a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82d8106f50 0x7f82d819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d8100d00 0x7f82d8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 --2- 172.21.15.7:0/2851608971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d8100350 0x7f82d8198450 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 -- 172.21.15.7:0/2851608971 >> 172.21.15.7:0/2851608971 conn(0x7f82d8077210 msgr2=0x7f82d8105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 -- 172.21.15.7:0/2851608971 shutdown_connections 2024-09-10T10:45:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:27.059+0000 7f82dc99b640 1 -- 172.21.15.7:0/2851608971 wait complete. 2024-09-10T10:45:27.811 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T10:45:27.812 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T10:45:27.822 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T10:45:27.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T10:45:28.081 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:28.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 -- 172.21.15.7:0/3526012495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1037d0 msgr2=0x7fdf3c10a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 --2- 172.21.15.7:0/3526012495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1037d0 0x7fdf3c10a060 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009ca0 tx=0x7fdf3402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 -- 172.21.15.7:0/3526012495 shutdown_connections 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 --2- 172.21.15.7:0/3526012495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c1037d0 0x7fdf3c10a060 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 --2- 172.21.15.7:0/3526012495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf3c102e10 0x7fdf3c103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 --2- 172.21.15.7:0/3526012495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf3c101c10 0x7fdf3c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 -- 172.21.15.7:0/3526012495 >> 172.21.15.7:0/3526012495 conn(0x7fdf3c0fd3c0 msgr2=0x7fdf3c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 -- 172.21.15.7:0/3526012495 shutdown_connections 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 -- 172.21.15.7:0/3526012495 wait complete. 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 Processor -- start 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.501+0000 7fdf43e5a640 1 -- start start 2024-09-10T10:45:28.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf43e5a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c101c10 0x7fdf3c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf43e5a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf3c102e10 0x7fdf3c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf43e5a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf3c1037d0 0x7fdf3c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf43e5a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c10d870 con 0x7fdf3c101c10 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf43e5a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c10d6f0 con 0x7fdf3c1037d0 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf43e5a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c10d9f0 con 0x7fdf3c102e10 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf41bf6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c101c10 0x7fdf3c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf41bf6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c101c10 0x7fdf3c197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48026/0 (socket says 172.21.15.7:48026) 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf41bf6640 1 -- 172.21.15.7:0/2847062309 learned_addr learned my addr 172.21.15.7:0/2847062309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf413f5640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf3c102e10 0x7fdf3c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf423f7640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf3c1037d0 0x7fdf3c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf41bf6640 1 -- 172.21.15.7:0/2847062309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf3c102e10 msgr2=0x7fdf3c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf41bf6640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf3c102e10 0x7fdf3c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf41bf6640 1 -- 172.21.15.7:0/2847062309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf3c1037d0 msgr2=0x7fdf3c19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf41bf6640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf3c1037d0 0x7fdf3c19f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf41bf6640 1 -- 172.21.15.7:0/2847062309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3c19fa10 con 0x7fdf3c101c10 2024-09-10T10:45:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf41bf6640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c101c10 0x7fdf3c197de0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800c970 tx=0x7fdf3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf32ffd640 1 -- 172.21.15.7:0/2847062309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38007c10 con 0x7fdf3c101c10 2024-09-10T10:45:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf43e5a640 1 -- 172.21.15.7:0/2847062309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3c19fd00 con 0x7fdf3c101c10 2024-09-10T10:45:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf32ffd640 1 -- 172.21.15.7:0/2847062309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf38007db0 con 0x7fdf3c101c10 2024-09-10T10:45:28.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.502+0000 7fdf32ffd640 1 -- 172.21.15.7:0/2847062309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800f8a0 con 0x7fdf3c101c10 2024-09-10T10:45:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.503+0000 7fdf43e5a640 1 -- 172.21.15.7:0/2847062309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3c1a0190 con 0x7fdf3c101c10 2024-09-10T10:45:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.504+0000 7fdf43e5a640 1 -- 172.21.15.7:0/2847062309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3c108f40 con 0x7fdf3c101c10 2024-09-10T10:45:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.505+0000 7fdf32ffd640 1 -- 172.21.15.7:0/2847062309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdf3800faa0 con 0x7fdf3c101c10 2024-09-10T10:45:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.506+0000 7fdf32ffd640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fdf280a2c40 0x7fdf280a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.506+0000 7fdf413f5640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fdf280a2c40 0x7fdf280a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:28.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.506+0000 7fdf32ffd640 1 -- 172.21.15.7:0/2847062309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fdf380cf460 con 0x7fdf3c101c10 2024-09-10T10:45:28.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.506+0000 7fdf413f5640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fdf280a2c40 0x7fdf280a5100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c199300 tx=0x7fdf2c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:28.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.510+0000 7fdf32ffd640 1 -- 172.21.15.7:0/2847062309 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf3809c300 con 0x7fdf3c101c10 2024-09-10T10:45:28.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.702+0000 7fdf43e5a640 1 -- 172.21.15.7:0/2847062309 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fdf3c062630 con 0x7fdf280a2c40 2024-09-10T10:45:28.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.703+0000 7fdf32ffd640 1 -- 172.21.15.7:0/2847062309 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fdf3c062630 con 0x7fdf280a2c40 2024-09-10T10:45:28.704 INFO:teuthology.orchestra.run.smithi007.stdout:Backend: cephadm 2024-09-10T10:45:28.705 INFO:teuthology.orchestra.run.smithi007.stdout:Available: Yes 2024-09-10T10:45:28.705 INFO:teuthology.orchestra.run.smithi007.stdout:Paused: No 2024-09-10T10:45:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 -- 172.21.15.7:0/2847062309 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fdf280a2c40 msgr2=0x7fdf280a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fdf280a2c40 0x7fdf280a5100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c199300 tx=0x7fdf2c00b390 comp rx=0 tx=0).stop 2024-09-10T10:45:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 -- 172.21.15.7:0/2847062309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c101c10 msgr2=0x7fdf3c197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c101c10 0x7fdf3c197de0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800c970 tx=0x7fdf3800ce30 comp rx=0 tx=0).stop 2024-09-10T10:45:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 -- 172.21.15.7:0/2847062309 shutdown_connections 2024-09-10T10:45:28.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fdf280a2c40 0x7fdf280a5100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf3c1037d0 0x7fdf3c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf3c102e10 0x7fdf3c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 --2- 172.21.15.7:0/2847062309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf3c101c10 0x7fdf3c197de0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 -- 172.21.15.7:0/2847062309 >> 172.21.15.7:0/2847062309 conn(0x7fdf3c0fd3c0 msgr2=0x7fdf3c106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 -- 172.21.15.7:0/2847062309 shutdown_connections 2024-09-10T10:45:28.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:28.706+0000 7fdf43e5a640 1 -- 172.21.15.7:0/2847062309 wait complete. 2024-09-10T10:45:28.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:28 smithi103 ceph-mon[35160]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:28.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:28 smithi007 ceph-mon[30873]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:28.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:28 smithi133 ceph-mon[35282]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:29.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T10:45:29.650 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:30.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.099+0000 7fe504d30640 1 -- 172.21.15.7:0/3831876676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe500101790 msgr2=0x7fe500101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.099+0000 7fe504d30640 1 --2- 172.21.15.7:0/3831876676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe500101790 0x7fe500101c40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009900 tx=0x7fe4f802a410 comp rx=0 tx=0).stop 2024-09-10T10:45:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.099+0000 7fe504d30640 1 -- 172.21.15.7:0/3831876676 shutdown_connections 2024-09-10T10:45:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.099+0000 7fe504d30640 1 --2- 172.21.15.7:0/3831876676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe500101790 0x7fe500101c40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.099+0000 7fe504d30640 1 --2- 172.21.15.7:0/3831876676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe500100df0 0x7fe500101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.099+0000 7fe504d30640 1 --2- 172.21.15.7:0/3831876676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5001004b0 0x7fe5001008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.099+0000 7fe504d30640 1 -- 172.21.15.7:0/3831876676 >> 172.21.15.7:0/3831876676 conn(0x7fe5000fc280 msgr2=0x7fe5000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.099+0000 7fe504d30640 1 -- 172.21.15.7:0/3831876676 shutdown_connections 2024-09-10T10:45:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.099+0000 7fe504d30640 1 -- 172.21.15.7:0/3831876676 wait complete. 2024-09-10T10:45:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe504d30640 1 Processor -- start 2024-09-10T10:45:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe504d30640 1 -- start start 2024-09-10T10:45:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe504d30640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5001004b0 0x7fe50019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe504d30640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe500100df0 0x7fe50019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe504d30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe500101790 0x7fe5001a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe504d30640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe50010fa20 con 0x7fe500100df0 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe504d30640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe50010f8a0 con 0x7fe5001004b0 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe504d30640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe50010fba0 con 0x7fe500101790 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe4fdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe500100df0 0x7fe50019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe4fdd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe500100df0 0x7fe50019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48044/0 (socket says 172.21.15.7:48044) 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe4fdd9b640 1 -- 172.21.15.7:0/196227804 learned_addr learned my addr 172.21.15.7:0/196227804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe4fe59c640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5001004b0 0x7fe50019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.100+0000 7fe4fed9d640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe500101790 0x7fe5001a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe4fdd9b640 1 -- 172.21.15.7:0/196227804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe500101790 msgr2=0x7fe5001a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe4fdd9b640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe500101790 0x7fe5001a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe4fdd9b640 1 -- 172.21.15.7:0/196227804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5001004b0 msgr2=0x7fe50019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe4fdd9b640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5001004b0 0x7fe50019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe4fdd9b640 1 -- 172.21.15.7:0/196227804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5001a1d60 con 0x7fe500100df0 2024-09-10T10:45:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe4fed9d640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe500101790 0x7fe5001a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe4fdd9b640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe500100df0 0x7fe50019a670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800d950 tx=0x7fe4e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe4f77fe640 1 -- 172.21.15.7:0/196227804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8017070 con 0x7fe500100df0 2024-09-10T10:45:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe504d30640 1 -- 172.21.15.7:0/196227804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5001a1ff0 con 0x7fe500100df0 2024-09-10T10:45:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe4f77fe640 1 -- 172.21.15.7:0/196227804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8009c30 con 0x7fe500100df0 2024-09-10T10:45:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe4f77fe640 1 -- 172.21.15.7:0/196227804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800f6b0 con 0x7fe500100df0 2024-09-10T10:45:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.101+0000 7fe504d30640 1 -- 172.21.15.7:0/196227804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001a2500 con 0x7fe500100df0 2024-09-10T10:45:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.102+0000 7fe504d30640 1 -- 172.21.15.7:0/196227804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50010b200 con 0x7fe500100df0 2024-09-10T10:45:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.103+0000 7fe4f77fe640 1 -- 172.21.15.7:0/196227804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe4e80109d0 con 0x7fe500100df0 2024-09-10T10:45:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.104+0000 7fe4f77fe640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe4ec0a2c40 0x7fe4ec0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.104+0000 7fe4f77fe640 1 -- 172.21.15.7:0/196227804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fe4e80ced50 con 0x7fe500100df0 2024-09-10T10:45:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.104+0000 7fe4fe59c640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe4ec0a2c40 0x7fe4ec0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.105+0000 7fe4fe59c640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe4ec0a2c40 0x7fe4ec0a5100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0005fd0 tx=0x7fe4f0009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.108+0000 7fe4f77fe640 1 -- 172.21.15.7:0/196227804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4e8009de0 con 0x7fe500100df0 2024-09-10T10:45:30.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.293+0000 7fe504d30640 1 -- 172.21.15.7:0/196227804 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe500062630 con 0x7fe4ec0a2c40 2024-09-10T10:45:30.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.306+0000 7fe4f77fe640 1 -- 172.21.15.7:0/196227804 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fe500062630 con 0x7fe4ec0a2c40 2024-09-10T10:45:30.307 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T10:45:30.307 INFO:teuthology.orchestra.run.smithi007.stdout:mds.a smithi007 running (92s) 16s ago 91s 16.6M - 19.1.1-305-g1cf819b7 41d671128357 064f0afb982e 2024-09-10T10:45:30.307 INFO:teuthology.orchestra.run.smithi007.stdout:mds.b smithi103 running (89s) 23s ago 88s 17.1M - 19.1.1-305-g1cf819b7 41d671128357 0aeddf290aee 2024-09-10T10:45:30.307 INFO:teuthology.orchestra.run.smithi007.stdout:mds.c smithi133 running (102s) 23s ago 102s 20.3M - 19.1.1-305-g1cf819b7 41d671128357 87822b8b05ce 2024-09-10T10:45:30.307 INFO:teuthology.orchestra.run.smithi007.stdout:mds.d smithi007 running (110s) 16s ago 109s 15.9M - 19.1.1-305-g1cf819b7 41d671128357 4f30e13f3f4e 2024-09-10T10:45:30.307 INFO:teuthology.orchestra.run.smithi007.stdout:mds.e smithi103 running (107s) 23s ago 107s 16.4M - 19.1.1-305-g1cf819b7 41d671128357 26f870833e69 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mds.f smithi133 running (105s) 23s ago 104s 18.0M - 19.1.1-305-g1cf819b7 41d671128357 78ce81a2e7d0 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mds.g smithi007 running (95s) 16s ago 94s 13.6M - 19.1.1-305-g1cf819b7 41d671128357 28b0843720ee 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mds.h smithi103 running (100s) 23s ago 99s 16.9M - 19.1.1-305-g1cf819b7 41d671128357 95c1fb125d95 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mds.i smithi133 running (97s) 23s ago 97s 26.6M - 19.1.1-305-g1cf819b7 41d671128357 354565dfd373 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mds.j smithi007 running (113s) 16s ago 112s 16.8M - 19.1.1-305-g1cf819b7 41d671128357 5f3188481593 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mds.k smithi103 running (116s) 23s ago 115s 14.4M - 19.1.1-305-g1cf819b7 41d671128357 17da46448619 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mds.l smithi133 running (118s) 23s ago 118s 18.3M - 19.1.1-305-g1cf819b7 41d671128357 185a2aa8bf29 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.x smithi007 *:9283,8765 running (8m) 16s ago 8m 509M - 19.1.1-305-g1cf819b7 41d671128357 562a96b83dc9 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.y smithi103 *:8443,8765 running (7m) 23s ago 7m 441M - 19.1.1-305-g1cf819b7 41d671128357 70049c4f604a 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.z smithi133 *:8443,8765 running (7m) 23s ago 7m 443M - 19.1.1-305-g1cf819b7 41d671128357 fda7baca4a5d 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a smithi007 running (8m) 16s ago 8m 51.3M 2048M 19.1.1-305-g1cf819b7 41d671128357 bd02b0b05566 2024-09-10T10:45:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b smithi103 running (7m) 23s ago 7m 43.6M 2048M 19.1.1-305-g1cf819b7 41d671128357 85482a8afe41 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c smithi133 running (7m) 23s ago 7m 41.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 973b4a04a53c 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (6m) 16s ago 6m 54.5M 4273M 19.1.1-305-g1cf819b7 41d671128357 c6dd876c6dcb 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi103 running (6m) 23s ago 6m 55.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 6fe1f10dba82 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi133 running (6m) 23s ago 6m 49.3M 4272M 19.1.1-305-g1cf819b7 41d671128357 0ac99fa985e7 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi007 running (5m) 16s ago 5m 52.5M 4273M 19.1.1-305-g1cf819b7 41d671128357 7ef99490c4b1 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi103 running (5m) 23s ago 5m 52.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 fdfec9708c9f 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi133 running (4m) 23s ago 4m 46.1M 4272M 19.1.1-305-g1cf819b7 41d671128357 1ccc6f626995 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 smithi007 running (4m) 16s ago 4m 50.7M 4273M 19.1.1-305-g1cf819b7 41d671128357 0313a75fd994 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 smithi103 running (3m) 23s ago 3m 46.8M 4272M 19.1.1-305-g1cf819b7 41d671128357 bbd596177d29 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 smithi133 running (3m) 23s ago 3m 53.9M 4272M 19.1.1-305-g1cf819b7 41d671128357 eedbb2bab8d4 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 smithi007 running (3m) 16s ago 3m 50.9M 4273M 19.1.1-305-g1cf819b7 41d671128357 59e6e1583b4d 2024-09-10T10:45:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 smithi103 running (2m) 23s ago 2m 53.9M 4272M 19.1.1-305-g1cf819b7 41d671128357 5e60b3da221d 2024-09-10T10:45:30.310 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 smithi133 running (2m) 23s ago 2m 51.7M 4272M 19.1.1-305-g1cf819b7 41d671128357 4cc791ae6f00 2024-09-10T10:45:30.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 -- 172.21.15.7:0/196227804 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe4ec0a2c40 msgr2=0x7fe4ec0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe4ec0a2c40 0x7fe4ec0a5100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0005fd0 tx=0x7fe4f0009680 comp rx=0 tx=0).stop 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 -- 172.21.15.7:0/196227804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe500100df0 msgr2=0x7fe50019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe500100df0 0x7fe50019a670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800d950 tx=0x7fe4e800de10 comp rx=0 tx=0).stop 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 -- 172.21.15.7:0/196227804 shutdown_connections 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fe4ec0a2c40 0x7fe4ec0a5100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe500101790 0x7fe5001a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe500100df0 0x7fe50019a670 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 --2- 172.21.15.7:0/196227804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5001004b0 0x7fe50019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 -- 172.21.15.7:0/196227804 >> 172.21.15.7:0/196227804 conn(0x7fe5000fc280 msgr2=0x7fe5000fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 -- 172.21.15.7:0/196227804 shutdown_connections 2024-09-10T10:45:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:30.308+0000 7fe504d30640 1 -- 172.21.15.7:0/196227804 wait complete. 2024-09-10T10:45:30.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:30 smithi103 ceph-mon[35160]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:30.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:30 smithi103 ceph-mon[35160]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:30.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:30 smithi007 ceph-mon[30873]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:30.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:30 smithi007 ceph-mon[30873]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:30.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:30 smithi133 ceph-mon[35282]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:30.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:30 smithi133 ceph-mon[35282]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:30.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T10:45:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:31.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:31 smithi007 ceph-mon[30873]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.688+0000 7f4abc482640 1 -- 172.21.15.7:0/4174140509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4101d10 msgr2=0x7f4ab4102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.688+0000 7f4abc482640 1 --2- 172.21.15.7:0/4174140509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4101d10 0x7f4ab4102190 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009900 tx=0x7f4aa802a410 comp rx=0 tx=0).stop 2024-09-10T10:45:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.688+0000 7f4abc482640 1 -- 172.21.15.7:0/4174140509 shutdown_connections 2024-09-10T10:45:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.688+0000 7f4abc482640 1 --2- 172.21.15.7:0/4174140509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ab41026d0 0x7f4ab4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.688+0000 7f4abc482640 1 --2- 172.21.15.7:0/4174140509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4101d10 0x7f4ab4102190 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.688+0000 7f4abc482640 1 --2- 172.21.15.7:0/4174140509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab4100b10 0x7f4ab4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.688+0000 7f4abc482640 1 -- 172.21.15.7:0/4174140509 >> 172.21.15.7:0/4174140509 conn(0x7f4ab40fc2a0 msgr2=0x7f4ab40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.688+0000 7f4abc482640 1 -- 172.21.15.7:0/4174140509 shutdown_connections 2024-09-10T10:45:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.688+0000 7f4abc482640 1 -- 172.21.15.7:0/4174140509 wait complete. 2024-09-10T10:45:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4abc482640 1 Processor -- start 2024-09-10T10:45:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4abc482640 1 -- start start 2024-09-10T10:45:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4abc482640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ab4100b10 0x7f4ab410b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4abc482640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4101d10 0x7f4ab4109960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4abc482640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab41026d0 0x7f4ab4109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4abc482640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab410d900 con 0x7f4ab4101d10 2024-09-10T10:45:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4abc482640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab410d780 con 0x7f4ab41026d0 2024-09-10T10:45:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4abc482640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab410da80 con 0x7f4ab4100b10 2024-09-10T10:45:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4ab9a1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4101d10 0x7f4ab4109960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4ab9a1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4101d10 0x7f4ab4109960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55028/0 (socket says 172.21.15.7:55028) 2024-09-10T10:45:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4ab9a1d640 1 -- 172.21.15.7:0/2363520163 learned_addr learned my addr 172.21.15.7:0/2363520163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4abaa1f640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab41026d0 0x7f4ab4109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.689+0000 7f4aba21e640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ab4100b10 0x7f4ab410b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4ab9a1d640 1 -- 172.21.15.7:0/2363520163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ab4100b10 msgr2=0x7f4ab410b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4ab9a1d640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ab4100b10 0x7f4ab410b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4ab9a1d640 1 -- 172.21.15.7:0/2363520163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab41026d0 msgr2=0x7f4ab4109ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4ab9a1d640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab41026d0 0x7f4ab4109ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4ab9a1d640 1 -- 172.21.15.7:0/2363520163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab410a760 con 0x7f4ab4101d10 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4aba21e640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ab4100b10 0x7f4ab410b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4abaa1f640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab41026d0 0x7f4ab4109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4ab9a1d640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4101d10 0x7f4ab4109960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009230 tx=0x7f4aa8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4aa77fe640 1 -- 172.21.15.7:0/2363520163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa803f030 con 0x7f4ab4101d10 2024-09-10T10:45:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4aa77fe640 1 -- 172.21.15.7:0/2363520163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa8003ce0 con 0x7f4ab4101d10 2024-09-10T10:45:31.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4abc482640 1 -- 172.21.15.7:0/2363520163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab41a20b0 con 0x7f4ab4101d10 2024-09-10T10:45:31.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4aa77fe640 1 -- 172.21.15.7:0/2363520163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8035070 con 0x7f4ab4101d10 2024-09-10T10:45:31.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.690+0000 7f4abc482640 1 -- 172.21.15.7:0/2363520163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab41a2630 con 0x7f4ab4101d10 2024-09-10T10:45:31.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.691+0000 7f4abc482640 1 -- 172.21.15.7:0/2363520163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab4107ee0 con 0x7f4ab4101d10 2024-09-10T10:45:31.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.693+0000 7f4aa77fe640 1 -- 172.21.15.7:0/2363520163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4aa8042030 con 0x7f4ab4101d10 2024-09-10T10:45:31.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.693+0000 7f4aa77fe640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4aa00a2c90 0x7f4aa00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:31.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.693+0000 7f4aba21e640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4aa00a2c90 0x7f4aa00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:31.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.693+0000 7f4aa77fe640 1 -- 172.21.15.7:0/2363520163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f4aa80ef950 con 0x7f4ab4101d10 2024-09-10T10:45:31.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.694+0000 7f4aba21e640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4aa00a2c90 0x7f4aa00a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4101b70 tx=0x7f4ab0005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:31.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.696+0000 7f4aa77fe640 1 -- 172.21.15.7:0/2363520163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4aa80bc700 con 0x7f4ab4101d10 2024-09-10T10:45:31.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.882+0000 7f4abc482640 1 -- 172.21.15.7:0/2363520163 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4ab4062630 con 0x7f4aa00a2c90 2024-09-10T10:45:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.887+0000 7f4aa77fe640 1 -- 172.21.15.7:0/2363520163 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4ab4062630 con 0x7f4aa00a2c90 2024-09-10T10:45:31.889 INFO:teuthology.orchestra.run.smithi007.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T10:45:31.889 INFO:teuthology.orchestra.run.smithi007.stdout:mds.all 12/12 24s ago 2m smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-09-10T10:45:31.889 INFO:teuthology.orchestra.run.smithi007.stdout:mgr 3/3 24s ago 7m smithi007=x;smithi103=y;smithi133=z;count:3 2024-09-10T10:45:31.889 INFO:teuthology.orchestra.run.smithi007.stdout:mon 3/3 24s ago 7m smithi007:172.21.15.7=a;smithi103:172.21.15.103=b;smithi133:172.21.15.133=c;count:3 2024-09-10T10:45:31.889 INFO:teuthology.orchestra.run.smithi007.stdout:osd 12 24s ago - 2024-09-10T10:45:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.889+0000 7f4abc482640 1 -- 172.21.15.7:0/2363520163 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4aa00a2c90 msgr2=0x7f4aa00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.889+0000 7f4abc482640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4aa00a2c90 0x7f4aa00a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4101b70 tx=0x7f4ab0005d30 comp rx=0 tx=0).stop 2024-09-10T10:45:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.889+0000 7f4abc482640 1 -- 172.21.15.7:0/2363520163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4101d10 msgr2=0x7f4ab4109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.889+0000 7f4abc482640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4101d10 0x7f4ab4109960 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8009230 tx=0x7f4aa8004430 comp rx=0 tx=0).stop 2024-09-10T10:45:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.889+0000 7f4abc482640 1 -- 172.21.15.7:0/2363520163 shutdown_connections 2024-09-10T10:45:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.889+0000 7f4abc482640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f4aa00a2c90 0x7f4aa00a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.889+0000 7f4abc482640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ab41026d0 0x7f4ab4109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.889+0000 7f4abc482640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4101d10 0x7f4ab4109960 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.889+0000 7f4abc482640 1 --2- 172.21.15.7:0/2363520163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ab4100b10 0x7f4ab410b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:31.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.889+0000 7f4abc482640 1 -- 172.21.15.7:0/2363520163 >> 172.21.15.7:0/2363520163 conn(0x7f4ab40fc2a0 msgr2=0x7f4ab41050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:31.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.890+0000 7f4abc482640 1 -- 172.21.15.7:0/2363520163 shutdown_connections 2024-09-10T10:45:31.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:31.890+0000 7f4abc482640 1 -- 172.21.15.7:0/2363520163 wait complete. 2024-09-10T10:45:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:31 smithi103 ceph-mon[35160]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:31 smithi133 ceph-mon[35282]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:32.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T10:45:32.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:32 smithi007 ceph-mon[30873]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:32.823 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:32 smithi103 ceph-mon[35160]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:32.877 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:32 smithi133 ceph-mon[35282]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:33.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 -- 172.21.15.7:0/780421648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ec41026d0 msgr2=0x7f2ec4108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:33.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 --2- 172.21.15.7:0/780421648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ec41026d0 0x7f2ec4108f60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009ca0 tx=0x7f2ebc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 -- 172.21.15.7:0/780421648 shutdown_connections 2024-09-10T10:45:33.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 --2- 172.21.15.7:0/780421648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ec41026d0 0x7f2ec4108f60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 --2- 172.21.15.7:0/780421648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec4101d10 0x7f2ec4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 --2- 172.21.15.7:0/780421648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ec4100b10 0x7f2ec4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 -- 172.21.15.7:0/780421648 >> 172.21.15.7:0/780421648 conn(0x7f2ec40fc2a0 msgr2=0x7f2ec40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:33.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 -- 172.21.15.7:0/780421648 shutdown_connections 2024-09-10T10:45:33.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 -- 172.21.15.7:0/780421648 wait complete. 2024-09-10T10:45:33.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 Processor -- start 2024-09-10T10:45:33.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.248+0000 7f2eca911640 1 -- start start 2024-09-10T10:45:33.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2eca911640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec4100b10 0x7f2ec4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2eca911640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ec4101d10 0x7f2ec41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2eca911640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ec41026d0 0x7f2ec419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2eca911640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec410c890 con 0x7f2ec41026d0 2024-09-10T10:45:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2eca911640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec410c710 con 0x7f2ec4100b10 2024-09-10T10:45:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2eca911640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec410ca10 con 0x7f2ec4101d10 2024-09-10T10:45:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec8eae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ec41026d0 0x7f2ec419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec8eae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ec41026d0 0x7f2ec419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55034/0 (socket says 172.21.15.7:55034) 2024-09-10T10:45:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec8eae640 1 -- 172.21.15.7:0/201734594 learned_addr learned my addr 172.21.15.7:0/201734594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:33.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec37fe640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ec4101d10 0x7f2ec41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:33.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec3fff640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec4100b10 0x7f2ec4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:33.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec8eae640 1 -- 172.21.15.7:0/201734594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ec4101d10 msgr2=0x7f2ec41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:33.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec8eae640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ec4101d10 0x7f2ec41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec8eae640 1 -- 172.21.15.7:0/201734594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec4100b10 msgr2=0x7f2ec4198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:33.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec8eae640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec4100b10 0x7f2ec4198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec8eae640 1 -- 172.21.15.7:0/201734594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec419fca0 con 0x7f2ec41026d0 2024-09-10T10:45:33.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.249+0000 7f2ec3fff640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec4100b10 0x7f2ec4198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:33.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.250+0000 7f2ec8eae640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ec41026d0 0x7f2ec419f630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0098d0 tx=0x7f2ebc02f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:33.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.250+0000 7f2ec17fa640 1 -- 172.21.15.7:0/201734594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc02fc30 con 0x7f2ec41026d0 2024-09-10T10:45:33.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.250+0000 7f2ec17fa640 1 -- 172.21.15.7:0/201734594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ebc003e50 con 0x7f2ec41026d0 2024-09-10T10:45:33.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.250+0000 7f2eca911640 1 -- 172.21.15.7:0/201734594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec419ff30 con 0x7f2ec41026d0 2024-09-10T10:45:33.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.250+0000 7f2ec17fa640 1 -- 172.21.15.7:0/201734594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc004810 con 0x7f2ec41026d0 2024-09-10T10:45:33.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.250+0000 7f2eca911640 1 -- 172.21.15.7:0/201734594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec41a0460 con 0x7f2ec41026d0 2024-09-10T10:45:33.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.251+0000 7f2eca911640 1 -- 172.21.15.7:0/201734594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec41a06c0 con 0x7f2ec41026d0 2024-09-10T10:45:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.252+0000 7f2ec17fa640 1 -- 172.21.15.7:0/201734594 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2ebc003970 con 0x7f2ec41026d0 2024-09-10T10:45:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.253+0000 7f2ec17fa640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2eac0a2c40 0x7f2eac0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.253+0000 7f2ec3fff640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2eac0a2c40 0x7f2eac0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.253+0000 7f2ec17fa640 1 -- 172.21.15.7:0/201734594 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f2ebc0f0430 con 0x7f2ec41026d0 2024-09-10T10:45:33.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.253+0000 7f2ec3fff640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2eac0a2c40 0x7f2eac0a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4101b70 tx=0x7f2eb4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:33.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.257+0000 7f2ec17fa640 1 -- 172.21.15.7:0/201734594 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ebc0bd470 con 0x7f2ec41026d0 2024-09-10T10:45:33.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.436+0000 7f2eca911640 1 -- 172.21.15.7:0/201734594 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2ec41a0890 con 0x7f2eac0a2c40 2024-09-10T10:45:33.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.438+0000 7f2ec17fa640 1 -- 172.21.15.7:0/201734594 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f2ec41a0890 con 0x7f2eac0a2c40 2024-09-10T10:45:33.439 INFO:teuthology.orchestra.run.smithi007.stdout:HOST ADDR LABELS STATUS 2024-09-10T10:45:33.439 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 172.21.15.7 2024-09-10T10:45:33.439 INFO:teuthology.orchestra.run.smithi007.stdout:smithi103 172.21.15.103 2024-09-10T10:45:33.439 INFO:teuthology.orchestra.run.smithi007.stdout:smithi133 172.21.15.133 2024-09-10T10:45:33.439 INFO:teuthology.orchestra.run.smithi007.stdout:3 hosts in cluster 2024-09-10T10:45:33.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 -- 172.21.15.7:0/201734594 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2eac0a2c40 msgr2=0x7f2eac0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:33.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2eac0a2c40 0x7f2eac0a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4101b70 tx=0x7f2eb4008040 comp rx=0 tx=0).stop 2024-09-10T10:45:33.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 -- 172.21.15.7:0/201734594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ec41026d0 msgr2=0x7f2ec419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:33.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ec41026d0 0x7f2ec419f630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0098d0 tx=0x7f2ebc02f840 comp rx=0 tx=0).stop 2024-09-10T10:45:33.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 -- 172.21.15.7:0/201734594 shutdown_connections 2024-09-10T10:45:33.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2eac0a2c40 0x7f2eac0a5100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ec41026d0 0x7f2ec419f630 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ec4101d10 0x7f2ec41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 --2- 172.21.15.7:0/201734594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ec4100b10 0x7f2ec4198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:33.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 -- 172.21.15.7:0/201734594 >> 172.21.15.7:0/201734594 conn(0x7f2ec40fc2a0 msgr2=0x7f2ec406a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:33.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 -- 172.21.15.7:0/201734594 shutdown_connections 2024-09-10T10:45:33.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:33.440+0000 7f2eca911640 1 -- 172.21.15.7:0/201734594 wait complete. 2024-09-10T10:45:33.828 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:33 smithi007 ceph-mon[30873]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:33.888 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:33 smithi133 ceph-mon[35282]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:33 smithi103 ceph-mon[35160]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:34.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T10:45:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:34.781 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:34 smithi007 ceph-mon[30873]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:34.781 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:34 smithi007 ceph-mon[30873]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.864+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3900584840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc098107900 msgr2=0x7fc098109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.864+0000 7fc09d8f6640 1 --2- 172.21.15.7:0/3900584840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc098107900 0x7fc098109d50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc090009ca0 tx=0x7fc09002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.864+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3900584840 shutdown_connections 2024-09-10T10:45:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.864+0000 7fc09d8f6640 1 --2- 172.21.15.7:0/3900584840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc098107900 0x7fc098109d50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.864+0000 7fc09d8f6640 1 --2- 172.21.15.7:0/3900584840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc098104fd0 0x7fc0981073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.864+0000 7fc09d8f6640 1 --2- 172.21.15.7:0/3900584840 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0981026a0 0x7fc098104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.864+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3900584840 >> 172.21.15.7:0/3900584840 conn(0x7fc0980fc2a0 msgr2=0x7fc0980fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.865+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3900584840 shutdown_connections 2024-09-10T10:45:34.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.865+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3900584840 wait complete. 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.865+0000 7fc09d8f6640 1 Processor -- start 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.865+0000 7fc09d8f6640 1 -- start start 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.865+0000 7fc09d8f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0981026a0 0x7fc098197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.865+0000 7fc09d8f6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc098104fd0 0x7fc098198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.865+0000 7fc09d8f6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc098107900 0x7fc09819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc09d8f6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc09810d8c0 con 0x7fc0981026a0 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc09d8f6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc09810d740 con 0x7fc098107900 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc09d8f6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc09810da40 con 0x7fc098104fd0 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc096ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0981026a0 0x7fc098197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc096ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0981026a0 0x7fc098197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55044/0 (socket says 172.21.15.7:55044) 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc096ffd640 1 -- 172.21.15.7:0/3672193313 learned_addr learned my addr 172.21.15.7:0/3672193313 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc0967fc640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc098104fd0 0x7fc098198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:34.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc0977fe640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc098107900 0x7fc09819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc096ffd640 1 -- 172.21.15.7:0/3672193313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc098104fd0 msgr2=0x7fc098198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc096ffd640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc098104fd0 0x7fc098198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc096ffd640 1 -- 172.21.15.7:0/3672193313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc098107900 msgr2=0x7fc09819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc096ffd640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc098107900 0x7fc09819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc096ffd640 1 -- 172.21.15.7:0/3672193313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09819fb10 con 0x7fc0981026a0 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc0967fc640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc098104fd0 0x7fc098198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc096ffd640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0981026a0 0x7fc098197ee0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc08800d950 tx=0x7fc08800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc09c8f4640 1 -- 172.21.15.7:0/3672193313 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088017030 con 0x7fc0981026a0 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc09c8f4640 1 -- 172.21.15.7:0/3672193313 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc088004020 con 0x7fc0981026a0 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3672193313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09819fe00 con 0x7fc0981026a0 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.866+0000 7fc09c8f4640 1 -- 172.21.15.7:0/3672193313 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0880047a0 con 0x7fc0981026a0 2024-09-10T10:45:34.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.867+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3672193313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0981a02b0 con 0x7fc0981026a0 2024-09-10T10:45:34.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.868+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3672193313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0980fdd90 con 0x7fc0981026a0 2024-09-10T10:45:34.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.869+0000 7fc09c8f4640 1 -- 172.21.15.7:0/3672193313 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc0880041d0 con 0x7fc0981026a0 2024-09-10T10:45:34.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.869+0000 7fc09c8f4640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc0800a2cc0 0x7fc0800a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:34.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.870+0000 7fc09c8f4640 1 -- 172.21.15.7:0/3672193313 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fc0880ceb90 con 0x7fc0981026a0 2024-09-10T10:45:34.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.870+0000 7fc0967fc640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc0800a2cc0 0x7fc0800a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:34.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.870+0000 7fc0967fc640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc0800a2cc0 0x7fc0800a5180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc098199400 tx=0x7fc08c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:34.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:34.874+0000 7fc09c8f4640 1 -- 172.21.15.7:0/3672193313 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc08809b9a0 con 0x7fc0981026a0 2024-09-10T10:45:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:34 smithi103 ceph-mon[35160]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:34 smithi103 ceph-mon[35160]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:34.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:34 smithi133 ceph-mon[35282]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:34.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:34 smithi133 ceph-mon[35282]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:35.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.061+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3672193313 --> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc098062630 con 0x7fc0800a2cc0 2024-09-10T10:45:35.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.063+0000 7fc09c8f4640 1 -- 172.21.15.7:0/3672193313 <== mgr.14150 v2:172.21.15.7:6800/3525096194 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc098062630 con 0x7fc0800a2cc0 2024-09-10T10:45:35.064 INFO:teuthology.orchestra.run.smithi007.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T10:45:35.064 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GE480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-10T10:45:35.065 INFO:teuthology.orchestra.run.smithi007.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-10T10:45:35.065 INFO:teuthology.orchestra.run.smithi007.stdout:smithi133 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401AQ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-10T10:45:35.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.065+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3672193313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc0800a2cc0 msgr2=0x7fc0800a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:35.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.065+0000 7fc09d8f6640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc0800a2cc0 0x7fc0800a5180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc098199400 tx=0x7fc08c005f90 comp rx=0 tx=0).stop 2024-09-10T10:45:35.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.065+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3672193313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0981026a0 msgr2=0x7fc098197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:35.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.065+0000 7fc09d8f6640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0981026a0 0x7fc098197ee0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc08800d950 tx=0x7fc08800de10 comp rx=0 tx=0).stop 2024-09-10T10:45:35.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.066+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3672193313 shutdown_connections 2024-09-10T10:45:35.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.066+0000 7fc09d8f6640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fc0800a2cc0 0x7fc0800a5180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.066+0000 7fc09d8f6640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc098107900 0x7fc09819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.066+0000 7fc09d8f6640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc098104fd0 0x7fc098198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.066+0000 7fc09d8f6640 1 --2- 172.21.15.7:0/3672193313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0981026a0 0x7fc098197ee0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:35.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.066+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3672193313 >> 172.21.15.7:0/3672193313 conn(0x7fc0980fc2a0 msgr2=0x7fc098103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:35.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.066+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3672193313 shutdown_connections 2024-09-10T10:45:35.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:35.066+0000 7fc09d8f6640 1 -- 172.21.15.7:0/3672193313 wait complete. 2024-09-10T10:45:35.714 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T10:45:35.723 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T10:45:35.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T10:45:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 -- 172.21.15.7:0/630989437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc1026d0 msgr2=0x7f00cc108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 --2- 172.21.15.7:0/630989437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc1026d0 0x7f00cc108f60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009ca0 tx=0x7f00c402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 -- 172.21.15.7:0/630989437 shutdown_connections 2024-09-10T10:45:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 --2- 172.21.15.7:0/630989437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc1026d0 0x7f00cc108f60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 --2- 172.21.15.7:0/630989437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00cc101d10 0x7f00cc102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 --2- 172.21.15.7:0/630989437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00cc100b10 0x7f00cc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 -- 172.21.15.7:0/630989437 >> 172.21.15.7:0/630989437 conn(0x7f00cc0fc2a0 msgr2=0x7f00cc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 -- 172.21.15.7:0/630989437 shutdown_connections 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 -- 172.21.15.7:0/630989437 wait complete. 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 Processor -- start 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.439+0000 7f00d0ce7640 1 -- start start 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00d0ce7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00cc100b10 0x7f00cc06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00d0ce7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00cc101d10 0x7f00cc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00d0ce7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc1026d0 0x7f00cc1a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00d0ce7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc10c770 con 0x7f00cc1026d0 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00d0ce7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc10c5f0 con 0x7f00cc100b10 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00d0ce7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc10c8f0 con 0x7f00cc101d10 2024-09-10T10:45:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00cad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc1026d0 0x7f00cc1a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00cad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc1026d0 0x7f00cc1a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55054/0 (socket says 172.21.15.7:55054) 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00cad9d640 1 -- 172.21.15.7:0/670907593 learned_addr learned my addr 172.21.15.7:0/670907593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00c3fff640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00cc101d10 0x7f00cc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00ca59c640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00cc100b10 0x7f00cc06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00cad9d640 1 -- 172.21.15.7:0/670907593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00cc101d10 msgr2=0x7f00cc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00cad9d640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00cc101d10 0x7f00cc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00cad9d640 1 -- 172.21.15.7:0/670907593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00cc100b10 msgr2=0x7f00cc06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00cad9d640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00cc100b10 0x7f00cc06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00cad9d640 1 -- 172.21.15.7:0/670907593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc1a1d30 con 0x7f00cc1026d0 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00ca59c640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00cc100b10 0x7f00cc06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00cad9d640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc1026d0 0x7f00cc1a1540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00c4005e30 tx=0x7f00c4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00c37fe640 1 -- 172.21.15.7:0/670907593 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4003c30 con 0x7f00cc1026d0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.440+0000 7f00c37fe640 1 -- 172.21.15.7:0/670907593 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00c4003dd0 con 0x7f00cc1026d0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.441+0000 7f00d0ce7640 1 -- 172.21.15.7:0/670907593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc1a1f60 con 0x7f00cc1026d0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.441+0000 7f00c37fe640 1 -- 172.21.15.7:0/670907593 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c402fde0 con 0x7f00cc1026d0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.441+0000 7f00d0ce7640 1 -- 172.21.15.7:0/670907593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc1a2470 con 0x7f00cc1026d0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.442+0000 7f00d0ce7640 1 -- 172.21.15.7:0/670907593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00cc107f60 con 0x7f00cc1026d0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.443+0000 7f00c37fe640 1 -- 172.21.15.7:0/670907593 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f00c4004050 con 0x7f00cc1026d0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.444+0000 7f00c37fe640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00b80a2c40 0x7f00b80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.444+0000 7f00ca59c640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00b80a2c40 0x7f00b80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.444+0000 7f00c37fe640 1 -- 172.21.15.7:0/670907593 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f00c40f0570 con 0x7f00cc1026d0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.445+0000 7f00ca59c640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00b80a2c40 0x7f00b80a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f00bc005d00 tx=0x7f00bc005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:36.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.447+0000 7f00c37fe640 1 -- 172.21.15.7:0/670907593 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00c40bd430 con 0x7f00cc1026d0 2024-09-10T10:45:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.660+0000 7f00d0ce7640 1 -- 172.21.15.7:0/670907593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f00cc02d7c0 con 0x7f00cc1026d0 2024-09-10T10:45:36.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:36 smithi007 ceph-mon[30873]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:36.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:36 smithi007 ceph-mon[30873]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.661+0000 7f00c37fe640 1 -- 172.21.15.7:0/670907593 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3087 (secure 0 0 0) 0x7f00c40bcd30 con 0x7f00cc1026d0 2024-09-10T10:45:36.663 INFO:teuthology.orchestra.run.smithi007.stdout:e24 2024-09-10T10:45:36.663 INFO:teuthology.orchestra.run.smithi007.stdout:btime 2024-09-10T10:44:23:076442+0000 2024-09-10T10:45:36.663 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T10:45:36.663 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T10:45:36.663 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-09-10T10:45:36.663 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:36.663 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-09-10T10:45:36.663 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 24 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-10T10:44:11.549269+0000 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-10T10:44:22.697096+0000 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 300 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-09-10T10:45:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-09-10T10:45:36.665 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-09-10T10:45:36.665 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-09-10T10:45:36.665 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T10:45:36.665 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 3 2024-09-10T10:45:36.665 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1,2 2024-09-10T10:45:36.665 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=24499,1=24482,2=24481} 2024-09-10T10:45:36.665 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-09-10T10:45:36.665 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-09-10T10:45:36.665 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 0 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 24499 members: 24481,24482,24499 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.i{0:24499} state up:active seq 6 addr [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f{0:24472} state up:standby-replay seq 1 addr [v2:172.21.15.133:6834/4118505620,v1:172.21.15.133:6835/4118505620] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b{1:24482} state up:active seq 6 addr [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.h{1:24452} state up:standby-replay seq 1 addr [v2:172.21.15.103:6836/2960409846,v1:172.21.15.103:6837/2960409846] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c{2:24481} state up:active seq 10 addr [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.666 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.l{2:24439} state up:standby-replay seq 1 addr [v2:172.21.15.133:6832/691908645,v1:172.21.15.133:6833/691908645] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.667 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:36.667 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:36.667 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-09-10T10:45:36.667 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:36.667 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.7:6834/3661855188,v1:172.21.15.7:6835/3661855188] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.667 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.7:6836/2814690837,v1:172.21.15.7:6837/2814690837] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.667 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.g{-1:14592} state up:standby seq 1 addr [v2:172.21.15.7:6838/1984917528,v1:172.21.15.7:6839/1984917528] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.667 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a{-1:14598} state up:standby seq 1 addr [v2:172.21.15.7:6840/1887416712,v1:172.21.15.7:6841/1887416712] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.667 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.k{-1:24416} state up:standby seq 1 addr [v2:172.21.15.103:6832/2889573301,v1:172.21.15.103:6833/2889573301] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.667 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e{-1:24437} state up:standby seq 1 addr [v2:172.21.15.103:6834/3115622627,v1:172.21.15.103:6835/3115622627] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:45:36.668 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-09-10T10:45:36.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 -- 172.21.15.7:0/670907593 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00b80a2c40 msgr2=0x7f00b80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:36.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00b80a2c40 0x7f00b80a5100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f00bc005d00 tx=0x7f00bc005c90 comp rx=0 tx=0).stop 2024-09-10T10:45:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 -- 172.21.15.7:0/670907593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc1026d0 msgr2=0x7f00cc1a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc1026d0 0x7f00cc1a1540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00c4005e30 tx=0x7f00c4031ee0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 -- 172.21.15.7:0/670907593 shutdown_connections 2024-09-10T10:45:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f00b80a2c40 0x7f00b80a5100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00cc1026d0 0x7f00cc1a1540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00cc101d10 0x7f00cc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 --2- 172.21.15.7:0/670907593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00cc100b10 0x7f00cc06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 -- 172.21.15.7:0/670907593 >> 172.21.15.7:0/670907593 conn(0x7f00cc0fc2a0 msgr2=0x7f00cc105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 -- 172.21.15.7:0/670907593 shutdown_connections 2024-09-10T10:45:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:36.664+0000 7f00d0ce7640 1 -- 172.21.15.7:0/670907593 wait complete. 2024-09-10T10:45:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:36 smithi103 ceph-mon[35160]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:36.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:36 smithi103 ceph-mon[35160]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:36.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:36 smithi133 ceph-mon[35282]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:36.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:36 smithi133 ceph-mon[35282]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:37.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T10:45:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:37.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:37 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/670907593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T10:45:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:37 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/670907593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T10:45:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.912+0000 7f287eae3640 1 -- 172.21.15.7:0/892247484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28781026d0 msgr2=0x7f2878108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.912+0000 7f287eae3640 1 --2- 172.21.15.7:0/892247484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28781026d0 0x7f2878108f60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2870009ca0 tx=0x7f287002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.912+0000 7f287eae3640 1 -- 172.21.15.7:0/892247484 shutdown_connections 2024-09-10T10:45:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.912+0000 7f287eae3640 1 --2- 172.21.15.7:0/892247484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28781026d0 0x7f2878108f60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.912+0000 7f287eae3640 1 --2- 172.21.15.7:0/892247484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2878101d10 0x7f2878102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.912+0000 7f287eae3640 1 --2- 172.21.15.7:0/892247484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2878100b10 0x7f2878100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.912+0000 7f287eae3640 1 -- 172.21.15.7:0/892247484 >> 172.21.15.7:0/892247484 conn(0x7f28780fc2a0 msgr2=0x7f28780fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.912+0000 7f287eae3640 1 -- 172.21.15.7:0/892247484 shutdown_connections 2024-09-10T10:45:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.912+0000 7f287eae3640 1 -- 172.21.15.7:0/892247484 wait complete. 2024-09-10T10:45:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287eae3640 1 Processor -- start 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287eae3640 1 -- start start 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287eae3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2878100b10 0x7f287810b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287eae3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2878101d10 0x7f2878109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287eae3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28781026d0 0x7f287810a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287eae3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f287810d940 con 0x7f2878101d10 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287eae3640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f287810d7c0 con 0x7f2878100b10 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287eae3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f287810dac0 con 0x7f28781026d0 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f2877fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2878101d10 0x7f2878109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f2877fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2878101d10 0x7f2878109ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55076/0 (socket says 172.21.15.7:55076) 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f2877fff640 1 -- 172.21.15.7:0/456693692 learned_addr learned my addr 172.21.15.7:0/456693692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287d080640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28781026d0 0x7f287810a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287c87f640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2878100b10 0x7f287810b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f2877fff640 1 -- 172.21.15.7:0/456693692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28781026d0 msgr2=0x7f287810a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f2877fff640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28781026d0 0x7f287810a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f2877fff640 1 -- 172.21.15.7:0/456693692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2878100b10 msgr2=0x7f287810b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f2877fff640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2878100b10 0x7f287810b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f2877fff640 1 -- 172.21.15.7:0/456693692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287810a8e0 con 0x7f2878101d10 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.913+0000 7f287c87f640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2878100b10 0x7f287810b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.914+0000 7f2877fff640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2878101d10 0x7f2878109ae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f286400d770 tx=0x7f286400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.914+0000 7f2875ffb640 1 -- 172.21.15.7:0/456693692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864017070 con 0x7f2878101d10 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.914+0000 7f2875ffb640 1 -- 172.21.15.7:0/456693692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2864005500 con 0x7f2878101d10 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.914+0000 7f287eae3640 1 -- 172.21.15.7:0/456693692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28781a20f0 con 0x7f2878101d10 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.914+0000 7f2875ffb640 1 -- 172.21.15.7:0/456693692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864004e00 con 0x7f2878101d10 2024-09-10T10:45:37.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.914+0000 7f287eae3640 1 -- 172.21.15.7:0/456693692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28781a2630 con 0x7f2878101d10 2024-09-10T10:45:37.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.915+0000 7f287eae3640 1 -- 172.21.15.7:0/456693692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2878107f60 con 0x7f2878101d10 2024-09-10T10:45:37.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.916+0000 7f2875ffb640 1 -- 172.21.15.7:0/456693692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2864004fa0 con 0x7f2878101d10 2024-09-10T10:45:37.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.917+0000 7f2875ffb640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f28680a2c10 0x7f28680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:37.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.917+0000 7f287c87f640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f28680a2c10 0x7f28680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:37.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.917+0000 7f2875ffb640 1 -- 172.21.15.7:0/456693692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f28640cec80 con 0x7f2878101d10 2024-09-10T10:45:37.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.917+0000 7f287c87f640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f28680a2c10 0x7f28680a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f286c006fd0 tx=0x7f286c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:37.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:37.922+0000 7f2875ffb640 1 -- 172.21.15.7:0/456693692 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f286409ba30 con 0x7f2878101d10 2024-09-10T10:45:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:37 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/670907593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T10:45:38.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.098+0000 7f287eae3640 1 -- 172.21.15.7:0/456693692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f287802d7c0 con 0x7f2878101d10 2024-09-10T10:45:38.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.099+0000 7f2875ffb640 1 -- 172.21.15.7:0/456693692 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) ==== 56+0+5247 (secure 0 0 0) 0x7f286409b330 con 0x7f2878101d10 2024-09-10T10:45:38.101 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 81 2024-09-10T10:45:38.101 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T10:45:38.101 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-10T10:36:33.117443+0000 2024-09-10T10:45:38.101 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-10T10:44:22.070795+0000 2024-09-10T10:45:38.101 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T10:45:38.101 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 26 2024-09-10T10:45:38.101 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-09-10T10:45:38.101 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-09-10T10:45:38.101 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client luminous 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 12 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6802/704452390,v1:172.21.15.7:6803/704452390] [v2:172.21.15.7:6804/704452390,v1:172.21.15.7:6805/704452390] exists,up a9239e99-3a62-4883-b2f1-8964ebb54492 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6800/2409158189,v1:172.21.15.103:6801/2409158189] [v2:172.21.15.103:6802/2409158189,v1:172.21.15.103:6803/2409158189] exists,up d750ff16-48c8-421d-b46b-49f0d2b7f4e7 2024-09-10T10:45:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 18 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6800/1657349858,v1:172.21.15.133:6801/1657349858] [v2:172.21.15.133:6802/1657349858,v1:172.21.15.133:6803/1657349858] exists,up 1ecb070a-a2f1-499a-99e7-f1967ebb52ff 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 24 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6810/871781144,v1:172.21.15.7:6811/871781144] [v2:172.21.15.7:6812/871781144,v1:172.21.15.7:6813/871781144] exists,up 8e53d79e-038f-4a7f-9ac9-16b60f47bb4e 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 31 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6808/1937243043,v1:172.21.15.103:6809/1937243043] [v2:172.21.15.103:6810/1937243043,v1:172.21.15.103:6811/1937243043] exists,up e4f04c13-594c-43dd-9a98-d2138061b1a0 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 35 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6808/577882774,v1:172.21.15.133:6809/577882774] [v2:172.21.15.133:6810/577882774,v1:172.21.15.133:6811/577882774] exists,up 73143170-5816-407d-ae03-dcf896057681 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 42 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6818/2646192734,v1:172.21.15.7:6819/2646192734] [v2:172.21.15.7:6820/2646192734,v1:172.21.15.7:6821/2646192734] exists,up 3a5904d5-36de-47b0-97fa-5a10b24d0c17 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 46 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6816/3858867236,v1:172.21.15.103:6817/3858867236] [v2:172.21.15.103:6818/3858867236,v1:172.21.15.103:6819/3858867236] exists,up 91e79d42-decd-4405-b9f2-9480f7edfca8 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 up in weight 1 up_from 54 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6816/131428321,v1:172.21.15.133:6817/131428321] [v2:172.21.15.133:6818/131428321,v1:172.21.15.133:6819/131428321] exists,up 64373754-e81a-430c-967d-4b4f289f3473 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 up in weight 1 up_from 61 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6826/945633240,v1:172.21.15.7:6827/945633240] [v2:172.21.15.7:6828/945633240,v1:172.21.15.7:6829/945633240] exists,up 91ebe2c9-bda3-4596-ac76-5678c71f850b 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 up in weight 1 up_from 66 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6824/907835659,v1:172.21.15.103:6825/907835659] [v2:172.21.15.103:6826/907835659,v1:172.21.15.103:6827/907835659] exists,up 2f65568f-89d1-4a67-a222-8e263d34c710 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 up in weight 1 up_from 73 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6824/2538184038,v1:172.21.15.133:6825/2538184038] [v2:172.21.15.133:6826/2538184038,v1:172.21.15.133:6827/2538184038] exists,up 4b49839b-32e9-4105-abee-a253f1aa3f7a 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.0 [7,11] 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.b [2,4,8,11] 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.16 [2,4] 2024-09-10T10:45:38.103 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.3 [4,9] 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.1a [5,6] 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.23 [4,8] 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.27 [3,9] 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.2c [4,9] 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/2266895831 expires 2024-09-11T10:37:20.588263+0000 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3394895925 expires 2024-09-11T10:37:20.588263+0000 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/4050686422 expires 2024-09-11T10:36:55.407956+0000 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/262340568 expires 2024-09-11T10:37:20.588263+0000 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/4050686422 expires 2024-09-11T10:36:55.407956+0000 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3768226338 expires 2024-09-11T10:36:55.407956+0000 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/2266895831 expires 2024-09-11T10:37:20.588263+0000 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/258851660 expires 2024-09-11T10:37:20.588263+0000 2024-09-10T10:45:38.104 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3411487770 expires 2024-09-11T10:36:55.407956+0000 2024-09-10T10:45:38.105 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/11850255 expires 2024-09-11T10:36:55.407956+0000 2024-09-10T10:45:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.101+0000 7f287eae3640 1 -- 172.21.15.7:0/456693692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f28680a2c10 msgr2=0x7f28680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.101+0000 7f287eae3640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f28680a2c10 0x7f28680a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f286c006fd0 tx=0x7f286c008040 comp rx=0 tx=0).stop 2024-09-10T10:45:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.101+0000 7f287eae3640 1 -- 172.21.15.7:0/456693692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2878101d10 msgr2=0x7f2878109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.101+0000 7f287eae3640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2878101d10 0x7f2878109ae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f286400d770 tx=0x7f286400dc30 comp rx=0 tx=0).stop 2024-09-10T10:45:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.101+0000 7f287eae3640 1 -- 172.21.15.7:0/456693692 shutdown_connections 2024-09-10T10:45:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.101+0000 7f287eae3640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f28680a2c10 0x7f28680a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.101+0000 7f287eae3640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28781026d0 0x7f287810a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.101+0000 7f287eae3640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2878101d10 0x7f2878109ae0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.101+0000 7f287eae3640 1 --2- 172.21.15.7:0/456693692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2878100b10 0x7f287810b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.101+0000 7f287eae3640 1 -- 172.21.15.7:0/456693692 >> 172.21.15.7:0/456693692 conn(0x7f28780fc2a0 msgr2=0x7f2878105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.102+0000 7f287eae3640 1 -- 172.21.15.7:0/456693692 shutdown_connections 2024-09-10T10:45:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:38.102+0000 7f287eae3640 1 -- 172.21.15.7:0/456693692 wait complete. 2024-09-10T10:45:38.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:38 smithi007 ceph-mon[30873]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:38.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:38 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/456693692' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T10:45:38.777 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T10:45:38.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:38 smithi103 ceph-mon[35160]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:38.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:38 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/456693692' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T10:45:38.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:38 smithi133 ceph-mon[35282]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:38.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:38 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/456693692' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T10:45:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.504+0000 7f409cfd0640 1 -- 172.21.15.7:0/308151612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4098101420 msgr2=0x7f409810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.504+0000 7f409cfd0640 1 --2- 172.21.15.7:0/308151612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4098101420 0x7f409810d950 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4090009cd0 tx=0x7f409002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:45:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.504+0000 7f409cfd0640 1 -- 172.21.15.7:0/308151612 shutdown_connections 2024-09-10T10:45:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.504+0000 7f409cfd0640 1 --2- 172.21.15.7:0/308151612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4098101420 0x7f409810d950 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.504+0000 7f409cfd0640 1 --2- 172.21.15.7:0/308151612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4098100a80 0x7f4098100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.504+0000 7f409cfd0640 1 --2- 172.21.15.7:0/308151612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4098106a80 0x7f4098106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.504+0000 7f409cfd0640 1 -- 172.21.15.7:0/308151612 >> 172.21.15.7:0/308151612 conn(0x7f40980fc7a0 msgr2=0x7f40980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.504+0000 7f409cfd0640 1 -- 172.21.15.7:0/308151612 shutdown_connections 2024-09-10T10:45:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.504+0000 7f409cfd0640 1 -- 172.21.15.7:0/308151612 wait complete. 2024-09-10T10:45:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.505+0000 7f409cfd0640 1 Processor -- start 2024-09-10T10:45:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.505+0000 7f409cfd0640 1 -- start start 2024-09-10T10:45:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.505+0000 7f409cfd0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4098100a80 0x7f40981992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.505+0000 7f409cfd0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4098101420 0x7f4098199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.505+0000 7f409cfd0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4098106a80 0x7f40981a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.505+0000 7f409cfd0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4098110ea0 con 0x7f4098106a80 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.505+0000 7f409cfd0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4098110d20 con 0x7f4098100a80 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.505+0000 7f409cfd0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4098111020 con 0x7f4098101420 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f4096d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4098106a80 0x7f40981a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f4096d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4098106a80 0x7f40981a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55090/0 (socket says 172.21.15.7:55090) 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f4096d9d640 1 -- 172.21.15.7:0/2369474548 learned_addr learned my addr 172.21.15.7:0/2369474548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f409659c640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4098100a80 0x7f40981992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f4095d9b640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4098101420 0x7f4098199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f4096d9d640 1 -- 172.21.15.7:0/2369474548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4098101420 msgr2=0x7f4098199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f4096d9d640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4098101420 0x7f4098199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f4096d9d640 1 -- 172.21.15.7:0/2369474548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4098100a80 msgr2=0x7f40981992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f4096d9d640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4098100a80 0x7f40981992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f4096d9d640 1 -- 172.21.15.7:0/2369474548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40981a48b0 con 0x7f4098106a80 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f409659c640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4098100a80 0x7f40981992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f4096d9d640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4098106a80 0x7f40981a4370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4090005cc0 tx=0x7f4090003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f40877fe640 1 -- 172.21.15.7:0/2369474548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090004040 con 0x7f4098106a80 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f40877fe640 1 -- 172.21.15.7:0/2369474548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40900046c0 con 0x7f4098106a80 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f409cfd0640 1 -- 172.21.15.7:0/2369474548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40981a4a80 con 0x7f4098106a80 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f40877fe640 1 -- 172.21.15.7:0/2369474548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090046480 con 0x7f4098106a80 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.506+0000 7f409cfd0640 1 -- 172.21.15.7:0/2369474548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40981a4f60 con 0x7f4098106a80 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.507+0000 7f409cfd0640 1 -- 172.21.15.7:0/2369474548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4098100ee0 con 0x7f4098106a80 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.509+0000 7f40877fe640 1 -- 172.21.15.7:0/2369474548 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f40900305a0 con 0x7f4098106a80 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.509+0000 7f40877fe640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f40800a2c90 0x7f40800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.509+0000 7f409659c640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f40800a2c90 0x7f40800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.509+0000 7f40877fe640 1 -- 172.21.15.7:0/2369474548 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f409002e030 con 0x7f4098106a80 2024-09-10T10:45:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.510+0000 7f409659c640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f40800a2c90 0x7f40800a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f40880058c0 tx=0x7f4088005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.513+0000 7f40877fe640 1 -- 172.21.15.7:0/2369474548 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f409004b060 con 0x7f4098106a80 2024-09-10T10:45:39.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.726+0000 7f409cfd0640 1 -- 172.21.15.7:0/2369474548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4098109010 con 0x7f4098106a80 2024-09-10T10:45:39.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.726+0000 7f40877fe640 1 -- 172.21.15.7:0/2369474548 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7f40900bd980 con 0x7f4098106a80 2024-09-10T10:45:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:39.728 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"btime":"2024-09-10T10:44:23:076442+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1984917528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1984917528},{"type":"v1","addr":"172.21.15.7:6839","nonce":1984917528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1887416712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1887416712},{"type":"v1","addr":"172.21.15.7:6841","nonce":1887416712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:44:11.549269+0000","modified":"2024-09-10T10:44:22.697096+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/2537001809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2537001809},{"type":"v1","addr":"172.21.15.103:6839","nonce":2537001809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24481,24482,24499]},"id":1}]} 2024-09-10T10:45:39.728 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-09-10T10:45:39.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 -- 172.21.15.7:0/2369474548 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f40800a2c90 msgr2=0x7f40800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:39.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f40800a2c90 0x7f40800a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f40880058c0 tx=0x7f4088005850 comp rx=0 tx=0).stop 2024-09-10T10:45:39.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 -- 172.21.15.7:0/2369474548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4098106a80 msgr2=0x7f40981a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:39.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4098106a80 0x7f40981a4370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4090005cc0 tx=0x7f4090003970 comp rx=0 tx=0).stop 2024-09-10T10:45:39.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 -- 172.21.15.7:0/2369474548 shutdown_connections 2024-09-10T10:45:39.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f40800a2c90 0x7f40800a5150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:39.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4098106a80 0x7f40981a4370 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:39.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4098101420 0x7f4098199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:39.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 --2- 172.21.15.7:0/2369474548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4098100a80 0x7f40981992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:39.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 -- 172.21.15.7:0/2369474548 >> 172.21.15.7:0/2369474548 conn(0x7f40980fc7a0 msgr2=0x7f40980fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:39.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 -- 172.21.15.7:0/2369474548 shutdown_connections 2024-09-10T10:45:39.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:39.729+0000 7f409cfd0640 1 -- 172.21.15.7:0/2369474548 wait complete. 2024-09-10T10:45:40.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:45:40.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:40 smithi007 ceph-mon[30873]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:40.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:40 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2369474548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:40.686 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:40 smithi103 ceph-mon[35160]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:40.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:40 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2369474548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:40.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:40 smithi133 ceph-mon[35282]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-10T10:45:40.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:40 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2369474548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.127+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/435234551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8df4106a60 msgr2=0x7f8df4106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.127+0000 7f8dfa8ce640 1 --2- 172.21.15.7:0/435234551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8df4106a60 0x7f8df4106e40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009930 tx=0x7f8de402a420 comp rx=0 tx=0).stop 2024-09-10T10:45:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.127+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/435234551 shutdown_connections 2024-09-10T10:45:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.127+0000 7f8dfa8ce640 1 --2- 172.21.15.7:0/435234551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4101400 0x7f8df410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.127+0000 7f8dfa8ce640 1 --2- 172.21.15.7:0/435234551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8df4100a60 0x7f8df4100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.127+0000 7f8dfa8ce640 1 --2- 172.21.15.7:0/435234551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8df4106a60 0x7f8df4106e40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.127+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/435234551 >> 172.21.15.7:0/435234551 conn(0x7f8df40fc780 msgr2=0x7f8df40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.127+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/435234551 shutdown_connections 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.127+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/435234551 wait complete. 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8dfa8ce640 1 Processor -- start 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8dfa8ce640 1 -- start start 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8dfa8ce640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8df4100a60 0x7f8df41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8dfa8ce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8df4101400 0x7f8df41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8dfa8ce640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4106a60 0x7f8df419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8dfa8ce640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8df4110e20 con 0x7f8df4100a60 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8dfa8ce640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8df4110ca0 con 0x7f8df4106a60 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8dfa8ce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8df4110fa0 con 0x7f8df4101400 2024-09-10T10:45:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8df3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8df4100a60 0x7f8df41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8df3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8df4100a60 0x7f8df41984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58756/0 (socket says 172.21.15.7:58756) 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8df3fff640 1 -- 172.21.15.7:0/726531775 learned_addr learned my addr 172.21.15.7:0/726531775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8debfff640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8df4101400 0x7f8df41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8df8e6b640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4106a60 0x7f8df419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8df3fff640 1 -- 172.21.15.7:0/726531775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8df4101400 msgr2=0x7f8df41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8df3fff640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8df4101400 0x7f8df41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8df3fff640 1 -- 172.21.15.7:0/726531775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4106a60 msgr2=0x7f8df419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8df3fff640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4106a60 0x7f8df419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8df3fff640 1 -- 172.21.15.7:0/726531775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df419d460 con 0x7f8df4100a60 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.128+0000 7f8df8e6b640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4106a60 0x7f8df419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.129+0000 7f8df3fff640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8df4100a60 0x7f8df41984b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009230 tx=0x7f8de4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.129+0000 7f8df1ffb640 1 -- 172.21.15.7:0/726531775 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de403f030 con 0x7f8df4100a60 2024-09-10T10:45:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.129+0000 7f8df1ffb640 1 -- 172.21.15.7:0/726531775 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8de4003ad0 con 0x7f8df4100a60 2024-09-10T10:45:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.129+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/726531775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df419d6f0 con 0x7f8df4100a60 2024-09-10T10:45:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.129+0000 7f8df1ffb640 1 -- 172.21.15.7:0/726531775 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4035070 con 0x7f8df4100a60 2024-09-10T10:45:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.129+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/726531775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df41a5030 con 0x7f8df4100a60 2024-09-10T10:45:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.130+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/726531775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df4100ec0 con 0x7f8df4100a60 2024-09-10T10:45:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.131+0000 7f8df1ffb640 1 -- 172.21.15.7:0/726531775 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8de4042030 con 0x7f8df4100a60 2024-09-10T10:45:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.132+0000 7f8df1ffb640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8de00a2c10 0x7f8de00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.132+0000 7f8debfff640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8de00a2c10 0x7f8de00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.132+0000 7f8df1ffb640 1 -- 172.21.15.7:0/726531775 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f8de40ef6f0 con 0x7f8df4100a60 2024-09-10T10:45:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.132+0000 7f8debfff640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8de00a2c10 0x7f8de00a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8df4104a30 tx=0x7f8ddc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:41.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.136+0000 7f8df1ffb640 1 -- 172.21.15.7:0/726531775 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8de40bc4a0 con 0x7f8df4100a60 2024-09-10T10:45:41.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.347+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/726531775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8df4109fe0 con 0x7f8df4100a60 2024-09-10T10:45:41.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.348+0000 7f8df1ffb640 1 -- 172.21.15.7:0/726531775 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11912 (secure 0 0 0) 0x7f8de40bbda0 con 0x7f8df4100a60 2024-09-10T10:45:41.350 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:41.350 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"btime":"2024-09-10T10:44:23:076442+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1984917528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1984917528},{"type":"v1","addr":"172.21.15.7:6839","nonce":1984917528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1887416712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1887416712},{"type":"v1","addr":"172.21.15.7:6841","nonce":1887416712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/3115622627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":3115622627},{"type":"v1","addr":"172.21.15.103:6835","nonce":3115622627}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:44:11.549269+0000","modified":"2024-09-10T10:44:22.697096+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/4118505620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":4118505620},{"type":"v1","addr":"172.21.15.133:6835","nonce":4118505620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/2537001809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2537001809},{"type":"v1","addr":"172.21.15.103:6839","nonce":2537001809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24481,24482,24499]},"id":1}]} 2024-09-10T10:45:41.350 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-09-10T10:45:41.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/726531775 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8de00a2c10 msgr2=0x7f8de00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8de00a2c10 0x7f8de00a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8df4104a30 tx=0x7f8ddc00a2d0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/726531775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8df4100a60 msgr2=0x7f8df41984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:41.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8df4100a60 0x7f8df41984b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009230 tx=0x7f8de4004270 comp rx=0 tx=0).stop 2024-09-10T10:45:41.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/726531775 shutdown_connections 2024-09-10T10:45:41.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f8de00a2c10 0x7f8de00a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8df4106a60 0x7f8df419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8df4101400 0x7f8df41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 --2- 172.21.15.7:0/726531775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8df4100a60 0x7f8df41984b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:41.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/726531775 >> 172.21.15.7:0/726531775 conn(0x7f8df40fc780 msgr2=0x7f8df406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:41.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/726531775 shutdown_connections 2024-09-10T10:45:41.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:41.350+0000 7f8dfa8ce640 1 -- 172.21.15.7:0/726531775 wait complete. 2024-09-10T10:45:41.813 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:41 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/726531775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:41 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/726531775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:41 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/726531775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:45:42.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:45:42.283 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:42.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:42 smithi007 ceph-mon[30873]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.716+0000 7f0ba2f83640 1 -- 172.21.15.7:0/1529778195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b9c101de0 msgr2=0x7f0b9c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.716+0000 7f0ba2f83640 1 --2- 172.21.15.7:0/1529778195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b9c101de0 0x7f0b9c1021c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009900 tx=0x7f0b9002a410 comp rx=0 tx=0).stop 2024-09-10T10:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.716+0000 7f0ba2f83640 1 -- 172.21.15.7:0/1529778195 shutdown_connections 2024-09-10T10:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.716+0000 7f0ba2f83640 1 --2- 172.21.15.7:0/1529778195 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9c106f10 0x7f0b9c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.716+0000 7f0ba2f83640 1 --2- 172.21.15.7:0/1529778195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c102790 0x7f0b9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.716+0000 7f0ba2f83640 1 --2- 172.21.15.7:0/1529778195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b9c101de0 0x7f0b9c1021c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.716+0000 7f0ba2f83640 1 -- 172.21.15.7:0/1529778195 >> 172.21.15.7:0/1529778195 conn(0x7f0b9c0fd8c0 msgr2=0x7f0b9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.716+0000 7f0ba2f83640 1 -- 172.21.15.7:0/1529778195 shutdown_connections 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.716+0000 7f0ba2f83640 1 -- 172.21.15.7:0/1529778195 wait complete. 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba2f83640 1 Processor -- start 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba2f83640 1 -- start start 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba2f83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b9c101de0 0x7f0b9c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba2f83640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c102790 0x7f0b9c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba2f83640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9c106f10 0x7f0b9c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba2f83640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c111f30 con 0x7f0b9c101de0 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba2f83640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c111db0 con 0x7f0b9c106f10 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba2f83640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1120b0 con 0x7f0b9c102790 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba0d1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b9c101de0 0x7f0b9c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba0d1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b9c101de0 0x7f0b9c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58768/0 (socket says 172.21.15.7:58768) 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.717+0000 7f0ba0d1f640 1 -- 172.21.15.7:0/3450837172 learned_addr learned my addr 172.21.15.7:0/3450837172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0ba1520640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9c106f10 0x7f0b9c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0b9bfff640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c102790 0x7f0b9c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0ba0d1f640 1 -- 172.21.15.7:0/3450837172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c102790 msgr2=0x7f0b9c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0ba0d1f640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c102790 0x7f0b9c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0ba0d1f640 1 -- 172.21.15.7:0/3450837172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9c106f10 msgr2=0x7f0b9c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0ba0d1f640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9c106f10 0x7f0b9c19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0ba0d1f640 1 -- 172.21.15.7:0/3450837172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9c19d2c0 con 0x7f0b9c101de0 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0b9bfff640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c102790 0x7f0b9c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0ba0d1f640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b9c101de0 0x7f0b9c198310 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005ab0 tx=0x7f0b900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0b99ffb640 1 -- 172.21.15.7:0/3450837172 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9003f030 con 0x7f0b9c101de0 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0b99ffb640 1 -- 172.21.15.7:0/3450837172 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b90003ce0 con 0x7f0b9c101de0 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0ba2f83640 1 -- 172.21.15.7:0/3450837172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9c19d550 con 0x7f0b9c101de0 2024-09-10T10:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0b99ffb640 1 -- 172.21.15.7:0/3450837172 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90035070 con 0x7f0b9c101de0 2024-09-10T10:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.718+0000 7f0ba2f83640 1 -- 172.21.15.7:0/3450837172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9c1a4e90 con 0x7f0b9c101de0 2024-09-10T10:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.719+0000 7f0ba2f83640 1 -- 172.21.15.7:0/3450837172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b9c1032c0 con 0x7f0b9c101de0 2024-09-10T10:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.721+0000 7f0b99ffb640 1 -- 172.21.15.7:0/3450837172 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0b90042030 con 0x7f0b9c101de0 2024-09-10T10:45:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.721+0000 7f0b99ffb640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0b8c0a2c10 0x7f0b8c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.722+0000 7f0b9bfff640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0b8c0a2c10 0x7f0b8c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.722+0000 7f0b99ffb640 1 -- 172.21.15.7:0/3450837172 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f0b900efd20 con 0x7f0b9c101de0 2024-09-10T10:45:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.722+0000 7f0b9bfff640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0b8c0a2c10 0x7f0b8c0a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c1088e0 tx=0x7f0b88005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:42.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.725+0000 7f0b99ffb640 1 -- 172.21.15.7:0/3450837172 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b900bcb30 con 0x7f0b9c101de0 2024-09-10T10:45:42.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.904+0000 7f0ba2f83640 1 -- 172.21.15.7:0/3450837172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b9c1a5060 con 0x7f0b9c101de0 2024-09-10T10:45:42.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.905+0000 7f0b99ffb640 1 -- 172.21.15.7:0/3450837172 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22646 (secure 0 0 0) 0x7f0b900bc430 con 0x7f0b9c101de0 2024-09-10T10:45:42.907 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T10:45:42.907 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:36:33.117443+0000","modified":"2024-09-10T10:44:22.070795+0000","last_up_change":"2024-09-10T10:43:23.913978+0000","last_in_change":"2024-09-10T10:42:59.733844+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:39:37.104114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:07.441548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:09.970273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9239e99-3a62-4883-b2f1-8964ebb54492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6803","nonce":704452390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6805","nonce":704452390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6809","nonce":704452390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6807","nonce":704452390}]},"public_addr":"172.21.15.7:6803/704452390","cluster_addr":"172.21.15.7:6805/704452390","heartbeat_back_addr":"172.21.15.7:6809/704452390","heartbeat_front_addr":"172.21.15.7:6807/704452390","state":["exists","up"]},{"osd":1,"uuid":"d750ff16-48c8-421d-b46b-49f0d2b7f4e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6801","nonce":2409158189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6803","nonce":2409158189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6807","nonce":2409158189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6805","nonce":2409158189}]},"public_addr":"172.21.15.103:6801/2409158189","cluster_addr":"172.21.15.103:6803/2409158189","heartbeat_back_addr":"172.21.15.103:6807/2409158189","heartbeat_front_addr":"172.21.15.103:6805/2409158189","state":["exists","up"]},{"osd":2,"uuid":"1ecb070a-a2f1-499a-99e7-f1967ebb52ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6801","nonce":1657349858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6803","nonce":1657349858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6807","nonce":1657349858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6805","nonce":1657349858}]},"public_addr":"172.21.15.133:6801/1657349858","cluster_addr":"172.21.15.133:6803/1657349858","heartbeat_back_addr":"172.21.15.133:6807/1657349858","heartbeat_front_addr":"172.21.15.133:6805/1657349858","state":["exists","up"]},{"osd":3,"uuid":"8e53d79e-038f-4a7f-9ac9-16b60f47bb4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6811","nonce":871781144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6813","nonce":871781144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6817","nonce":871781144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6815","nonce":871781144}]},"public_addr":"172.21.15.7:6811/871781144","cluster_addr":"172.21.15.7:6813/871781144","heartbeat_back_addr":"172.21.15.7:6817/871781144","heartbeat_front_addr":"172.21.15.7:6815/871781144","state":["exists","up"]},{"osd":4,"uuid":"e4f04c13-594c-43dd-9a98-d2138061b1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6809","nonce":1937243043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6811","nonce":1937243043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6815","nonce":1937243043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6813","nonce":1937243043}]},"public_addr":"172.21.15.103:6809/1937243043","cluster_addr":"172.21.15.103:6811/1937243043","heartbeat_back_addr":"172.21.15.103:6815/1937243043","heartbeat_front_addr":"172.21.15.103:6813/1937243043","state":["exists","up"]},{"osd":5,"uuid":"73143170-5816-407d-ae03-dcf896057681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6809","nonce":577882774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6811","nonce":577882774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6815","nonce":577882774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6813","nonce":577882774}]},"public_addr":"172.21.15.133:6809/577882774","cluster_addr":"172.21.15.133:6811/577882774","heartbeat_back_addr":"172.21.15.133:6815/577882774","heartbeat_front_addr":"172.21.15.133:6813/577882774","state":["exists","up"]},{"osd":6,"uuid":"3a5904d5-36de-47b0-97fa-5a10b24d0c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6819","nonce":2646192734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6821","nonce":2646192734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6825","nonce":2646192734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6823","nonce":2646192734}]},"public_addr":"172.21.15.7:6819/2646192734","cluster_addr":"172.21.15.7:6821/2646192734","heartbeat_back_addr":"172.21.15.7:6825/2646192734","heartbeat_front_addr":"172.21.15.7:6823/2646192734","state":["exists","up"]},{"osd":7,"uuid":"91e79d42-decd-4405-b9f2-9480f7edfca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6817","nonce":3858867236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6819","nonce":3858867236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6823","nonce":3858867236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6821","nonce":3858867236}]},"public_addr":"172.21.15.103:6817/3858867236","cluster_addr":"172.21.15.103:6819/3858867236","heartbeat_back_addr":"172.21.15.103:6823/3858867236","heartbeat_front_addr":"172.21.15.103:6821/3858867236","state":["exists","up"]},{"osd":8,"uuid":"64373754-e81a-430c-967d-4b4f289f3473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6817","nonce":131428321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6819","nonce":131428321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6823","nonce":131428321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6821","nonce":131428321}]},"public_addr":"172.21.15.133:6817/131428321","cluster_addr":"172.21.15.133:6819/131428321","heartbeat_back_addr":"172.21.15.133:6823/131428321","heartbeat_front_addr":"172.21.15.133:6821/131428321","state":["exists","up"]},{"osd":9,"uuid":"91ebe2c9-bda3-4596-ac76-5678c71f850b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6827","nonce":945633240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6829","nonce":945633240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6833","nonce":945633240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6831","nonce":945633240}]},"public_addr":"172.21.15.7:6827/945633240","cluster_addr":"172.21.15.7:6829/945633240","heartbeat_back_addr":"172.21.15.7:6833/945633240","heartbeat_front_addr":"172.21.15.7:6831/945633240","state":["exists","up"]},{"osd":10,"uuid":"2f65568f-89d1-4a67-a222-8e263d34c710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6825","nonce":907835659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6827","nonce":907835659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6831","nonce":907835659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6829","nonce":907835659}]},"public_addr":"172.21.15.103:6825/907835659","cluster_addr":"172.21.15.103:6827/907835659","heartbeat_back_addr":"172.21.15.103:6831/907835659","heartbeat_front_addr":"172.21.15.103:6829/907835659","state":["exists","up"]},{"osd":11,"uuid":"4b49839b-32e9-4105-abee-a253f1aa3f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6825","nonce":2538184038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6827","nonce":2538184038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6831","nonce":2538184038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6829","nonce":2538184038}]},"public_addr":"172.21.15.133:6825/2538184038","cluster_addr":"172.21.15.133:6827/2538184038","heartbeat_back_addr":"172.21.15.133:6831/2538184038","heartbeat_front_addr":"172.21.15.133:6829/2538184038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:44.541279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:10.103733+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:35.290268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:01.070763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:25.666941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:50.925733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:16.486537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:41.464155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:06.610089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:31.994080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:57.000696+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:21.398669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3394895925":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6800/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/262340568":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/3768226338":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/258851660":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3411487770":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/11850255":"2024-09-11T10:36:55.407956+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.907+0000 7f0ba2f83640 1 -- 172.21.15.7:0/3450837172 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0b8c0a2c10 msgr2=0x7f0b8c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.907+0000 7f0ba2f83640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0b8c0a2c10 0x7f0b8c0a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c1088e0 tx=0x7f0b88005d90 comp rx=0 tx=0).stop 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.907+0000 7f0ba2f83640 1 -- 172.21.15.7:0/3450837172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b9c101de0 msgr2=0x7f0b9c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.907+0000 7f0ba2f83640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b9c101de0 0x7f0b9c198310 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005ab0 tx=0x7f0b900042f0 comp rx=0 tx=0).stop 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.907+0000 7f0ba2f83640 1 -- 172.21.15.7:0/3450837172 shutdown_connections 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.907+0000 7f0ba2f83640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f0b8c0a2c10 0x7f0b8c0a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.907+0000 7f0ba2f83640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0b9c106f10 0x7f0b9c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.907+0000 7f0ba2f83640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b9c102790 0x7f0b9c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.907+0000 7f0ba2f83640 1 --2- 172.21.15.7:0/3450837172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b9c101de0 0x7f0b9c198310 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.907+0000 7f0ba2f83640 1 -- 172.21.15.7:0/3450837172 >> 172.21.15.7:0/3450837172 conn(0x7f0b9c0fd8c0 msgr2=0x7f0b9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.908+0000 7f0ba2f83640 1 -- 172.21.15.7:0/3450837172 shutdown_connections 2024-09-10T10:45:42.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:42.908+0000 7f0ba2f83640 1 -- 172.21.15.7:0/3450837172 wait complete. 2024-09-10T10:45:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:42 smithi103 ceph-mon[35160]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:42.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:42 smithi133 ceph-mon[35282]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-10T10:45:43.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T10:44:11.549269+0000', 'modified': '2024-09-10T10:44:22.697096+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24499, 'mds_1': 24482, 'mds_2': 24481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6833/691908645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6832', 'nonce': 691908645}, {'type': 'v1', 'addr': '172.21.15.133:6833', 'nonce': 691908645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6837/2960409846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 2960409846}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 2960409846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/4118505620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 4118505620}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 4118505620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.133:6837/3030361099', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 3030361099}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3030361099}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.103:6839/2537001809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2537001809}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2537001809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6839/1182528850', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 1182528850}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 1182528850}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24499, 'qdb_cluster': [24481, 24482, 24499]} 2024-09-10T10:45:43.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-10T10:45:43.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T10:45:43.801 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:43 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3450837172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:43.922 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:43 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3450837172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:43 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3450837172' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:45:44.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.284+0000 7f621fb1f640 1 -- 172.21.15.7:0/2493127435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f621810c170 msgr2=0x7f621810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:44.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.284+0000 7f621fb1f640 1 --2- 172.21.15.7:0/2493127435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f621810c170 0x7f621810e610 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6210009ca0 tx=0x7f621002a950 comp rx=0 tx=0).stop 2024-09-10T10:45:44.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.284+0000 7f621fb1f640 1 -- 172.21.15.7:0/2493127435 shutdown_connections 2024-09-10T10:45:44.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.284+0000 7f621fb1f640 1 --2- 172.21.15.7:0/2493127435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f621810c170 0x7f621810e610 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.284+0000 7f621fb1f640 1 --2- 172.21.15.7:0/2493127435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6218103660 0x7f621810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.284+0000 7f621fb1f640 1 --2- 172.21.15.7:0/2493127435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6218102d40 0x7f6218103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.284+0000 7f621fb1f640 1 -- 172.21.15.7:0/2493127435 >> 172.21.15.7:0/2493127435 conn(0x7f62180fc7a0 msgr2=0x7f62180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:44.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.284+0000 7f621fb1f640 1 -- 172.21.15.7:0/2493127435 shutdown_connections 2024-09-10T10:45:44.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.284+0000 7f621fb1f640 1 -- 172.21.15.7:0/2493127435 wait complete. 2024-09-10T10:45:44.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.284+0000 7f621fb1f640 1 Processor -- start 2024-09-10T10:45:44.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621fb1f640 1 -- start start 2024-09-10T10:45:44.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621fb1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6218102d40 0x7f621819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621fb1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6218103660 0x7f621819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621fb1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f621810c170 0x7f621819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621fb1f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6218111f70 con 0x7f6218103660 2024-09-10T10:45:44.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621fb1f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6218111df0 con 0x7f621810c170 2024-09-10T10:45:44.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621fb1f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f62181120f0 con 0x7f6218102d40 2024-09-10T10:45:44.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621d0ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6218103660 0x7f621819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621d0ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6218103660 0x7f621819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58784/0 (socket says 172.21.15.7:58784) 2024-09-10T10:45:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621d0ba640 1 -- 172.21.15.7:0/3236718516 learned_addr learned my addr 172.21.15.7:0/3236718516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621d8bb640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6218102d40 0x7f621819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621e0bc640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f621810c170 0x7f621819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621d0ba640 1 -- 172.21.15.7:0/3236718516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6218102d40 msgr2=0x7f621819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621d0ba640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6218102d40 0x7f621819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621d0ba640 1 -- 172.21.15.7:0/3236718516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f621810c170 msgr2=0x7f621819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621d0ba640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f621810c170 0x7f621819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621d0ba640 1 -- 172.21.15.7:0/3236718516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621819f5d0 con 0x7f6218103660 2024-09-10T10:45:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621e0bc640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f621810c170 0x7f621819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.285+0000 7f621d0ba640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6218103660 0x7f621819ab60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f620800c940 tx=0x7f620800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.286+0000 7f620effd640 1 -- 172.21.15.7:0/3236718516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208004220 con 0x7f6218103660 2024-09-10T10:45:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.286+0000 7f621fb1f640 1 -- 172.21.15.7:0/3236718516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621819f8c0 con 0x7f6218103660 2024-09-10T10:45:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.286+0000 7f620effd640 1 -- 172.21.15.7:0/3236718516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62080043c0 con 0x7f6218103660 2024-09-10T10:45:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.286+0000 7f620effd640 1 -- 172.21.15.7:0/3236718516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208004a00 con 0x7f6218103660 2024-09-10T10:45:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.286+0000 7f621fb1f640 1 -- 172.21.15.7:0/3236718516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62181a71a0 con 0x7f6218103660 2024-09-10T10:45:44.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.288+0000 7f620effd640 1 -- 172.21.15.7:0/3236718516 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f62080079b0 con 0x7f6218103660 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.288+0000 7f6221ef8740 1 Processor -- start 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.288+0000 7f6221ef8740 1 -- start start 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e89d0 0x55bb276e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 0x55bb276f3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f6221ef8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bb2764f090 con 0x55bb276f3bb0 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f6221ef8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb275f6a70 con 0x55bb276e89d0 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f6221ef8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bb2764b050 con 0x55bb276e92f0 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621d0ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621d0ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58794/0 (socket says 172.21.15.7:58794) 2024-09-10T10:45:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621d0ba640 1 -- 172.21.15.7:0/3924222225 learned_addr learned my addr 172.21.15.7:0/3924222225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f620effd640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f62040a2c10 0x7f62040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621e0bc640 1 --2- 172.21.15.7:0/3924222225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e89d0 0x55bb276e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621d8bb640 1 --2- 172.21.15.7:0/3924222225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 0x55bb276f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621e0bc640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f62040a2c10 0x7f62040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f620effd640 1 -- 172.21.15.7:0/3236718516 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f62080cf120 con 0x7f6218103660 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621d0ba640 1 -- 172.21.15.7:0/3924222225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 msgr2=0x55bb276f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621d0ba640 1 --2- 172.21.15.7:0/3924222225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 0x55bb276f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621d0ba640 1 -- 172.21.15.7:0/3924222225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e89d0 msgr2=0x55bb276e8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621d0ba640 1 --2- 172.21.15.7:0/3924222225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e89d0 0x55bb276e8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621d0ba640 1 -- 172.21.15.7:0/3924222225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb27603130 con 0x55bb276f3bb0 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.289+0000 7f621d8bb640 1 --2- 172.21.15.7:0/3924222225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 0x55bb276f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f621d0ba640 1 --2- 172.21.15.7:0/3924222225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f5fa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f620809b7c0 tx=0x7f6208007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a37954aa5a03f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:44.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f621e0bc640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f62040a2c10 0x7f62040a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6210009430 tx=0x7f621002a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f6203fff640 1 -- 172.21.15.7:0/3924222225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620801d070 con 0x55bb276f3bb0 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f6203fff640 1 -- 172.21.15.7:0/3924222225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f620809bf30 con 0x55bb276f3bb0 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f6203fff640 1 -- 172.21.15.7:0/3924222225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62080a44a0 con 0x55bb276f3bb0 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f6221ef8740 1 -- 172.21.15.7:0/3924222225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 msgr2=0x55bb276f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f6221ef8740 1 --2- 172.21.15.7:0/3924222225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f5fa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f620809b7c0 tx=0x7f6208007f90 comp rx=0 tx=0).stop 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f6221ef8740 1 -- 172.21.15.7:0/3924222225 shutdown_connections 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f6221ef8740 1 --2- 172.21.15.7:0/3924222225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f5fa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f6221ef8740 1 --2- 172.21.15.7:0/3924222225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 0x55bb276f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f6221ef8740 1 --2- 172.21.15.7:0/3924222225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e89d0 0x55bb276e8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.290+0000 7f6221ef8740 1 -- 172.21.15.7:0/3924222225 >> 172.21.15.7:0/3924222225 conn(0x55bb276e4f60 msgr2=0x55bb276e5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 1 -- 172.21.15.7:0/3924222225 shutdown_connections 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 1 -- 172.21.15.7:0/3924222225 wait complete. 2024-09-10T10:45:44.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 1 -- start start 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 1 Processor -- start 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e89d0 0x55bb276f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e92f0 0x55bb276f0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bb276f6780 con 0x55bb276f3bb0 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb27609f20 con 0x55bb276e92f0 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bb276f6900 con 0x55bb276e89d0 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.291+0000 7f6221ef8740 10 client.0 _ll_register_callbacks cb 0x55bb275d3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621e0bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621e0bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58808/0 (socket says 172.21.15.7:58808) 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621e0bc640 1 -- 172.21.15.7:0/2903302004 learned_addr learned my addr 172.21.15.7:0/2903302004 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:44.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621d8bb640 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e89d0 0x55bb276f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621d0ba640 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e92f0 0x55bb276f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621e0bc640 1 -- 172.21.15.7:0/2903302004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e89d0 msgr2=0x55bb276f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621e0bc640 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e89d0 0x55bb276f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621e0bc640 1 -- 172.21.15.7:0/2903302004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e92f0 msgr2=0x55bb276f0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621e0bc640 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e92f0 0x55bb276f0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621e0bc640 1 -- 172.21.15.7:0/2903302004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb273c02b0 con 0x55bb276f3bb0 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f621e0bc640 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f0cc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6210009a50 tx=0x7f62100095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cc1858f1774e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f62017fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f62017fa640 1 -- 172.21.15.7:0/2903302004 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621004d030 con 0x55bb276f3bb0 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f6221ef8740 1 -- 172.21.15.7:0/2903302004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620809bf30 con 0x55bb276f3bb0 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.292+0000 7f62017fa640 1 -- 172.21.15.7:0/2903302004 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f6210004250 con 0x55bb276f3bb0 2024-09-10T10:45:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f62017fa640 1 -- 172.21.15.7:0/2903302004 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6210034040 con 0x55bb276f3bb0 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f6221ef8740 10 client.15066 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f6221ef8740 1 -- 172.21.15.7:0/2903302004 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bb277020e0 con 0x55bb276f3bb0 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f62017fa640 1 -- 172.21.15.7:0/2903302004 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f620809bf30 con 0x55bb276f3bb0 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f62017fa640 1 -- 172.21.15.7:0/2903302004 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6210048040 con 0x55bb276f3bb0 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f6221ef8740 10 client.15066 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f6221ef8740 10 client.15066 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f6221ef8740 20 client.15066 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x55bb27704720 0x55bb27705b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f6221ef8740 4 client.15066 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f6221ef8740 1 -- 172.21.15.7:0/2903302004 --> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb2765c600 con 0x55bb27704720 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.293+0000 7f621d8bb640 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x55bb27704720 0x55bb27705b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.294+0000 7f621d8bb640 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x55bb27704720 0x55bb27705b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a32911552de085e server_cookie=2b747e99ac75d4be in_seq=0 out_seq=0 2024-09-10T10:45:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.294+0000 7f62017fa640 10 client.15066 ms_handle_connect on v2:172.21.15.133:6836/3030361099 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f62017fa640 1 -- 172.21.15.7:0/2903302004 <== mds.2 v2:172.21.15.133:6836/3030361099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb2765c600 con 0x55bb27704720 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f62017fa640 10 client.15066 handle_command_reply: tid=0 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 client.15066 shutdown 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 -- 172.21.15.7:0/2903302004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 msgr2=0x55bb276f0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f0cc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6210009a50 tx=0x7f62100095c0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 -- 172.21.15.7:0/2903302004 shutdown_connections 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x55bb27704720 0x55bb27705b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb276f0cc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e92f0 0x55bb276f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2903302004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e89d0 0x55bb276f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 -- 172.21.15.7:0/2903302004 >> 172.21.15.7:0/2903302004 conn(0x55bb276e4f60 msgr2=0x55bb276eac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f62017fa640 0 client.15066 ms_handle_reset on v2:172.21.15.133:6836/3030361099 2024-09-10T10:45:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 -- 172.21.15.7:0/2903302004 shutdown_connections 2024-09-10T10:45:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 1 -- 172.21.15.7:0/2903302004 wait complete. 2024-09-10T10:45:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.295+0000 7f6221ef8740 20 client.15066 trim_cache size 0 max 16384 2024-09-10T10:45:44.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f6221ef8740 1 Processor -- start 2024-09-10T10:45:44.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f6221ef8740 1 -- start start 2024-09-10T10:45:44.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276e89d0 0x55bb276f0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 0x55bb276ef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276f3bb0 0x55bb276ef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f6221ef8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bb27703ed0 con 0x55bb276e89d0 2024-09-10T10:45:44.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f6221ef8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb277045a0 con 0x55bb276f3bb0 2024-09-10T10:45:44.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f6221ef8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bb27704c70 con 0x55bb276e92f0 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f621d8bb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276e89d0 0x55bb276f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f621d8bb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276e89d0 0x55bb276f0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58810/0 (socket says 172.21.15.7:58810) 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f621d8bb640 1 -- 172.21.15.7:0/155092339 learned_addr learned my addr 172.21.15.7:0/155092339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f621d0ba640 1 --2- 172.21.15.7:0/155092339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 0x55bb276ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f621d8bb640 1 -- 172.21.15.7:0/155092339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 msgr2=0x55bb276ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f621d8bb640 1 --2- 172.21.15.7:0/155092339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 0x55bb276ef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f621d8bb640 1 -- 172.21.15.7:0/155092339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276f3bb0 msgr2=0x55bb276ef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f621d8bb640 1 --2- 172.21.15.7:0/155092339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276f3bb0 0x55bb276ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.313+0000 7f621d8bb640 1 -- 172.21.15.7:0/155092339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb276f6890 con 0x55bb276e89d0 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f621d8bb640 1 --2- 172.21.15.7:0/155092339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276e89d0 0x55bb276f0040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55bb277063f0 tx=0x7f621400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e5ae70c4ab0b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f62037fe640 1 -- 172.21.15.7:0/155092339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214005020 con 0x55bb276e89d0 2024-09-10T10:45:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f62037fe640 1 -- 172.21.15.7:0/155092339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62140047a0 con 0x55bb276e89d0 2024-09-10T10:45:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f62037fe640 1 -- 172.21.15.7:0/155092339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214011680 con 0x55bb276e89d0 2024-09-10T10:45:44.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f6221ef8740 1 -- 172.21.15.7:0/155092339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276e89d0 msgr2=0x55bb276f0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:44.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f6221ef8740 1 --2- 172.21.15.7:0/155092339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276e89d0 0x55bb276f0040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55bb277063f0 tx=0x7f621400b390 comp rx=0 tx=0).stop 2024-09-10T10:45:44.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f6221ef8740 1 -- 172.21.15.7:0/155092339 shutdown_connections 2024-09-10T10:45:44.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f6221ef8740 1 --2- 172.21.15.7:0/155092339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276f3bb0 0x55bb276ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f6221ef8740 1 --2- 172.21.15.7:0/155092339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e92f0 0x55bb276ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f6221ef8740 1 --2- 172.21.15.7:0/155092339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276e89d0 0x55bb276f0040 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f6221ef8740 1 -- 172.21.15.7:0/155092339 >> 172.21.15.7:0/155092339 conn(0x55bb276e4f60 msgr2=0x55bb276f0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:44.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f6221ef8740 1 -- 172.21.15.7:0/155092339 shutdown_connections 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.314+0000 7f6221ef8740 1 -- 172.21.15.7:0/155092339 wait complete. 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f6221ef8740 1 -- start start 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f6221ef8740 1 Processor -- start 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e89d0 0x55bb27707490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e92f0 0x55bb277079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f6221ef8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb2776ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f6221ef8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bb27705a10 con 0x55bb276f3bb0 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f6221ef8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bb27705340 con 0x55bb276e92f0 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f6221ef8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bb277060e0 con 0x55bb276e89d0 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f6221ef8740 10 client.0 _ll_register_callbacks cb 0x7f621002fc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f621e0bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb2776ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f621e0bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb2776ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58824/0 (socket says 172.21.15.7:58824) 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f621e0bc640 1 -- 172.21.15.7:0/2055586666 learned_addr learned my addr 172.21.15.7:0/2055586666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.315+0000 7f621d0ba640 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e92f0 0x55bb277079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f621d8bb640 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e89d0 0x55bb27707490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f621e0bc640 1 -- 172.21.15.7:0/2055586666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e89d0 msgr2=0x55bb27707490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f621e0bc640 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e89d0 0x55bb27707490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f621e0bc640 1 -- 172.21.15.7:0/2055586666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e92f0 msgr2=0x55bb277079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f621e0bc640 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e92f0 0x55bb277079d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f621e0bc640 1 -- 172.21.15.7:0/2055586666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb2776d3b0 con 0x55bb276f3bb0 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f621e0bc640 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb2776ce20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6210030e80 tx=0x7f621002fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2f89bc793cb855 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f62017fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f62017fa640 1 -- 172.21.15.7:0/2055586666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621004d070 con 0x55bb276f3bb0 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f62017fa640 1 -- 172.21.15.7:0/2055586666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f6210004660 con 0x55bb276f3bb0 2024-09-10T10:45:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f6221ef8740 1 -- 172.21.15.7:0/2055586666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62140047a0 con 0x55bb276f3bb0 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f62017fa640 1 -- 172.21.15.7:0/2055586666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6210004860 con 0x55bb276f3bb0 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f6221ef8740 10 client.15072 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f6221ef8740 1 -- 172.21.15.7:0/2055586666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bb2776e6b0 con 0x55bb276f3bb0 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.316+0000 7f62017fa640 1 -- 172.21.15.7:0/2055586666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62140047a0 con 0x55bb276f3bb0 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.317+0000 7f62017fa640 1 -- 172.21.15.7:0/2055586666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6210048040 con 0x55bb276f3bb0 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.317+0000 7f6221ef8740 10 client.15072 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.317+0000 7f6221ef8740 10 client.15072 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.317+0000 7f6221ef8740 20 client.15072 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.317+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x55bb27773290 0x55bb27771fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.317+0000 7f6221ef8740 4 client.15072 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.317+0000 7f6221ef8740 1 -- 172.21.15.7:0/2055586666 --> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] -- command(tid 0: {"prefix": "status"}) -- 0x55bb27656b10 con 0x55bb27773290 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.317+0000 7f621d8bb640 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x55bb27773290 0x55bb27771fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.317+0000 7f62017fa640 10 client.15072 ms_handle_connect on v2:172.21.15.133:6836/3030361099 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.317+0000 7f621d8bb640 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x55bb27773290 0x55bb27771fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=701816ccbaa37b3 server_cookie=6c165922ae142296 in_seq=0 out_seq=0 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f62017fa640 1 -- 172.21.15.7:0/2055586666 <== mds.2 v2:172.21.15.133:6836/3030361099 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bb27656b10 con 0x55bb27773290 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f62017fa640 10 client.15072 handle_command_reply: tid=0 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 client.15072 shutdown 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 -- 172.21.15.7:0/2055586666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 msgr2=0x55bb2776ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb2776ce20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6210030e80 tx=0x7f621002fba0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 -- 172.21.15.7:0/2055586666 shutdown_connections 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.133:6836/3030361099,v1:172.21.15.133:6837/3030361099] conn(0x55bb27773290 0x55bb27771fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb276f3bb0 0x55bb2776ce20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f62017fa640 0 client.15072 ms_handle_reset on v2:172.21.15.133:6836/3030361099 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bb276e92f0 0x55bb277079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 --2- 172.21.15.7:0/2055586666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb276e89d0 0x55bb27707490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 -- 172.21.15.7:0/2055586666 >> 172.21.15.7:0/2055586666 conn(0x55bb276e4f60 msgr2=0x55bb276f0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:44.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 -- 172.21.15.7:0/2055586666 shutdown_connections 2024-09-10T10:45:44.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 1 -- 172.21.15.7:0/2055586666 wait complete. 2024-09-10T10:45:44.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f6221ef8740 20 client.15072 trim_cache size 0 max 16384 2024-09-10T10:45:44.325 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T10:45:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "65af4d96-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-09-10T10:45:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24481, 2024-09-10T10:45:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T10:45:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T10:45:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T10:45:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 84.252633559000003, 2024-09-10T10:45:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-09-10T10:45:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 81, 2024-09-10T10:45:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 81, 2024-09-10T10:45:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 116.332088703 2024-09-10T10:45:44.326 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T10:45:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f621fb1f640 1 -- 172.21.15.7:0/3236718516 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f62040a2c10 msgr2=0x7f62040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f621fb1f640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f62040a2c10 0x7f62040a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6210009430 tx=0x7f621002a950 comp rx=0 tx=0).stop 2024-09-10T10:45:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.318+0000 7f621fb1f640 1 -- 172.21.15.7:0/3236718516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6218103660 msgr2=0x7f621819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.319+0000 7f621fb1f640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6218103660 0x7f621819ab60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f620800c940 tx=0x7f620800ce00 comp rx=0 tx=0).stop 2024-09-10T10:45:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.319+0000 7f621fb1f640 1 -- 172.21.15.7:0/3236718516 shutdown_connections 2024-09-10T10:45:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.319+0000 7f621fb1f640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f62040a2c10 0x7f62040a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.319+0000 7f621fb1f640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f621810c170 0x7f621819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.319+0000 7f621fb1f640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6218103660 0x7f621819ab60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.319+0000 7f621fb1f640 1 --2- 172.21.15.7:0/3236718516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6218102d40 0x7f621819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.319+0000 7f621fb1f640 1 -- 172.21.15.7:0/3236718516 >> 172.21.15.7:0/3236718516 conn(0x7f62180fc7a0 msgr2=0x7f62180fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.319+0000 7f621fb1f640 1 -- 172.21.15.7:0/3236718516 shutdown_connections 2024-09-10T10:45:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:44.319+0000 7f621fb1f640 1 -- 172.21.15.7:0/3236718516 wait complete. 2024-09-10T10:45:44.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:44 smithi103 ceph-mon[35160]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:44.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:44 smithi007 ceph-mon[30873]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:44.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:44 smithi133 ceph-mon[35282]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:44.980 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T10:45:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.649+0000 7f2e62481640 1 -- 172.21.15.7:0/1831298836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c1009d0 msgr2=0x7f2e5c100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.649+0000 7f2e62481640 1 --2- 172.21.15.7:0/1831298836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c1009d0 0x7f2e5c100db0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009900 tx=0x7f2e4c02a410 comp rx=0 tx=0).stop 2024-09-10T10:45:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.649+0000 7f2e62481640 1 -- 172.21.15.7:0/1831298836 shutdown_connections 2024-09-10T10:45:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.649+0000 7f2e62481640 1 --2- 172.21.15.7:0/1831298836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e5c10e4f0 0x7f2e5c110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.649+0000 7f2e62481640 1 --2- 172.21.15.7:0/1831298836 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c101380 0x7f2e5c10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.649+0000 7f2e62481640 1 --2- 172.21.15.7:0/1831298836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c1009d0 0x7f2e5c100db0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.649+0000 7f2e62481640 1 -- 172.21.15.7:0/1831298836 >> 172.21.15.7:0/1831298836 conn(0x7f2e5c0fc7d0 msgr2=0x7f2e5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.649+0000 7f2e62481640 1 -- 172.21.15.7:0/1831298836 shutdown_connections 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.649+0000 7f2e62481640 1 -- 172.21.15.7:0/1831298836 wait complete. 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.650+0000 7f2e62481640 1 Processor -- start 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.650+0000 7f2e62481640 1 -- start start 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e62481640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c1009d0 0x7f2e5c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e62481640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e5c101380 0x7f2e5c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e62481640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c10e4f0 0x7f2e5c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e62481640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c114340 con 0x7f2e5c1009d0 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e62481640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1141c0 con 0x7f2e5c10e4f0 2024-09-10T10:45:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e62481640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1144c0 con 0x7f2e5c101380 2024-09-10T10:45:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c1009d0 0x7f2e5c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c1009d0 0x7f2e5c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58838/0 (socket says 172.21.15.7:58838) 2024-09-10T10:45:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e5bfff640 1 -- 172.21.15.7:0/3649671601 learned_addr learned my addr 172.21.15.7:0/3649671601 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e5b7fe640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e5c101380 0x7f2e5c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e5bfff640 1 -- 172.21.15.7:0/3649671601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e5c101380 msgr2=0x7f2e5c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e5c101380 0x7f2e5c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e5bfff640 1 -- 172.21.15.7:0/3649671601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c10e4f0 msgr2=0x7f2e5c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c10e4f0 0x7f2e5c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c10e4f0 0x7f2e5c19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.651+0000 7f2e5bfff640 1 -- 172.21.15.7:0/3649671601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c19af70 con 0x7f2e5c1009d0 2024-09-10T10:45:45.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.652+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c1009d0 0x7f2e5c1a0630 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005ab0 tx=0x7f2e4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.652+0000 7f2e597fa640 1 -- 172.21.15.7:0/3649671601 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c03f030 con 0x7f2e5c1009d0 2024-09-10T10:45:45.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.652+0000 7f2e597fa640 1 -- 172.21.15.7:0/3649671601 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c003ce0 con 0x7f2e5c1009d0 2024-09-10T10:45:45.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.652+0000 7f2e62481640 1 -- 172.21.15.7:0/3649671601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c19b200 con 0x7f2e5c1009d0 2024-09-10T10:45:45.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.652+0000 7f2e597fa640 1 -- 172.21.15.7:0/3649671601 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c035070 con 0x7f2e5c1009d0 2024-09-10T10:45:45.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.652+0000 7f2e62481640 1 -- 172.21.15.7:0/3649671601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c1a73e0 con 0x7f2e5c1009d0 2024-09-10T10:45:45.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.654+0000 7f2e597fa640 1 -- 172.21.15.7:0/3649671601 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2e4c042030 con 0x7f2e5c1009d0 2024-09-10T10:45:45.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e6485a740 1 Processor -- start 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e6485a740 1 -- start start 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 0x55e8257e6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e72f0 0x55e8257f1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e6485a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e82574d090 con 0x55e8257f1bb0 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e6485a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e8256f4a70 con 0x55e8257e69d0 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e6485a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e825749050 con 0x55e8257e72f0 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e5b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e5b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257f3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58852/0 (socket says 172.21.15.7:58852) 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e5b7fe640 1 -- 172.21.15.7:0/310496076 learned_addr learned my addr 172.21.15.7:0/310496076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/310496076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e72f0 0x55e8257f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/310496076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 0x55e8257e6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e5b7fe640 1 -- 172.21.15.7:0/310496076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e72f0 msgr2=0x55e8257f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e5b7fe640 1 --2- 172.21.15.7:0/310496076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e72f0 0x55e8257f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e5b7fe640 1 -- 172.21.15.7:0/310496076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 msgr2=0x55e8257e6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e5b7fe640 1 --2- 172.21.15.7:0/310496076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 0x55e8257e6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e5b7fe640 1 -- 172.21.15.7:0/310496076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e825701130 con 0x55e8257f1bb0 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.655+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/310496076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 0x55e8257e6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.656+0000 7f2e597fa640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2e440a2c10 0x7f2e440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.656+0000 7f2e5b7fe640 1 --2- 172.21.15.7:0/310496076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257f3fa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009a40 tx=0x7f2e50009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b869e7037fe7071 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.656+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2e440a2c10 0x7f2e440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.656+0000 7f2e597fa640 1 -- 172.21.15.7:0/3649671601 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f2e4c0efb40 con 0x7f2e5c1009d0 2024-09-10T10:45:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.656+0000 7f2e427fc640 1 -- 172.21.15.7:0/310496076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50017070 con 0x55e8257f1bb0 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.656+0000 7f2e427fc640 1 -- 172.21.15.7:0/310496076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e50013470 con 0x55e8257f1bb0 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.656+0000 7f2e427fc640 1 -- 172.21.15.7:0/310496076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50012630 con 0x55e8257f1bb0 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 -- 172.21.15.7:0/310496076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 msgr2=0x55e8257f3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2e440a2c10 0x7f2e440a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c069760 tx=0x7f2e54007510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 --2- 172.21.15.7:0/310496076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257f3fa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009a40 tx=0x7f2e50009f00 comp rx=0 tx=0).stop 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 -- 172.21.15.7:0/310496076 shutdown_connections 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 --2- 172.21.15.7:0/310496076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257f3fa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 --2- 172.21.15.7:0/310496076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e72f0 0x55e8257f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 --2- 172.21.15.7:0/310496076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 0x55e8257e6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 -- 172.21.15.7:0/310496076 >> 172.21.15.7:0/310496076 conn(0x55e8257e2f60 msgr2=0x55e8257e3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 -- 172.21.15.7:0/310496076 shutdown_connections 2024-09-10T10:45:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 -- 172.21.15.7:0/310496076 wait complete. 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 -- start start 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.657+0000 7f2e6485a740 1 Processor -- start 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 0x55e8257f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e72f0 0x55e8257ee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257eec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e6485a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e8257f4780 con 0x55e8257f1bb0 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e6485a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e825707f20 con 0x55e8257e69d0 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e6485a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e8257f4900 con 0x55e8257e72f0 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e6485a740 10 client.0 _ll_register_callbacks cb 0x55e8256d1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e60a1e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e60a1e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257eec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58856/0 (socket says 172.21.15.7:58856) 2024-09-10T10:45:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e60a1e640 1 -- 172.21.15.7:0/2078332564 learned_addr learned my addr 172.21.15.7:0/2078332564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 0x55e8257f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e5b7fe640 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e72f0 0x55e8257ee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e60a1e640 1 -- 172.21.15.7:0/2078332564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e72f0 msgr2=0x55e8257ee6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e72f0 0x55e8257ee6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e60a1e640 1 -- 172.21.15.7:0/2078332564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 msgr2=0x55e8257f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 0x55e8257f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e60a1e640 1 -- 172.21.15.7:0/2078332564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8254be2b0 con 0x55e8257f1bb0 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 0x55e8257f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257eec30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2e54007c20 tx=0x7f2e54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae7f3301880985f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.658+0000 7f2e3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/2078332564 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5401a050 con 0x55e8257f1bb0 2024-09-10T10:45:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/2078332564 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f2e540129c0 con 0x55e8257f1bb0 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e6485a740 1 -- 172.21.15.7:0/2078332564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e50013470 con 0x55e8257f1bb0 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/2078332564 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2e54012bc0 con 0x55e8257f1bb0 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e6485a740 10 client.15084 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e6485a740 1 -- 172.21.15.7:0/2078332564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e825800080 con 0x55e8257f1bb0 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/2078332564 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e50013470 con 0x55e8257f1bb0 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/2078332564 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2e54011070 con 0x55e8257f1bb0 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e6485a740 10 client.15084 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e6485a740 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e6485a740 20 client.15084 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e6485a740 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x55e825802720 0x55e825803b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e6485a740 4 client.15084 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.659+0000 7f2e6485a740 1 -- 172.21.15.7:0/2078332564 --> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e82575a600 con 0x55e825802720 2024-09-10T10:45:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.660+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x55e825802720 0x55e825803b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.660+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x55e825802720 0x55e825803b90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4bf18ea48fd2bd8c server_cookie=7f27e59f2c0ac948 in_seq=0 out_seq=0 2024-09-10T10:45:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.660+0000 7f2e3b7fe640 10 client.15084 ms_handle_connect on v2:172.21.15.103:6838/2537001809 2024-09-10T10:45:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/2078332564 <== mds.1 v2:172.21.15.103:6838/2537001809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e82575a600 con 0x55e825802720 2024-09-10T10:45:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e3b7fe640 10 client.15084 handle_command_reply: tid=0 2024-09-10T10:45:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 client.15084 shutdown 2024-09-10T10:45:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 -- 172.21.15.7:0/2078332564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 msgr2=0x55e8257eec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257eec30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2e54007c20 tx=0x7f2e54004a40 comp rx=0 tx=0).stop 2024-09-10T10:45:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 -- 172.21.15.7:0/2078332564 shutdown_connections 2024-09-10T10:45:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257f1bb0 0x55e8257eec30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x55e825802720 0x55e825803b90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e72f0 0x55e8257ee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 --2- 172.21.15.7:0/2078332564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e69d0 0x55e8257f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 -- 172.21.15.7:0/2078332564 >> 172.21.15.7:0/2078332564 conn(0x55e8257e2f60 msgr2=0x55e8257e8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e3b7fe640 0 client.15084 ms_handle_reset on v2:172.21.15.103:6838/2537001809 2024-09-10T10:45:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 -- 172.21.15.7:0/2078332564 shutdown_connections 2024-09-10T10:45:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 1 -- 172.21.15.7:0/2078332564 wait complete. 2024-09-10T10:45:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.661+0000 7f2e6485a740 20 client.15084 trim_cache size 0 max 16384 2024-09-10T10:45:45.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e6485a740 1 Processor -- start 2024-09-10T10:45:45.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e6485a740 1 -- start start 2024-09-10T10:45:45.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e69d0 0x55e825802df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e72f0 0x55e8257ee030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257f1bb0 0x55e8257e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e6485a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e825801ed0 con 0x55e8257e69d0 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e6485a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e8258025a0 con 0x55e8257e72f0 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e6485a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e825802c70 con 0x55e8257f1bb0 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e69d0 0x55e825802df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e69d0 0x55e825802df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58870/0 (socket says 172.21.15.7:58870) 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e5bfff640 1 -- 172.21.15.7:0/200996510 learned_addr learned my addr 172.21.15.7:0/200996510 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.679+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/200996510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257f1bb0 0x55e8257e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.680+0000 7f2e5b7fe640 1 --2- 172.21.15.7:0/200996510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e72f0 0x55e8257ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.680+0000 7f2e5bfff640 1 -- 172.21.15.7:0/200996510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257f1bb0 msgr2=0x55e8257e9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.680+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/200996510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257f1bb0 0x55e8257e9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.680+0000 7f2e5bfff640 1 -- 172.21.15.7:0/200996510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e72f0 msgr2=0x55e8257ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.680+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/200996510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e72f0 0x55e8257ee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.680+0000 7f2e5bfff640 1 -- 172.21.15.7:0/200996510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8257e2a00 con 0x55e8257e69d0 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.680+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/200996510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e69d0 0x55e825802df0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0b8c20 tx=0x7f2e4c0f3690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b92361eea31f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.680+0000 7f2e41ffb640 1 -- 172.21.15.7:0/200996510 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c049070 con 0x55e8257e69d0 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.680+0000 7f2e41ffb640 1 -- 172.21.15.7:0/200996510 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c0bc950 con 0x55e8257e69d0 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.680+0000 7f2e41ffb640 1 -- 172.21.15.7:0/200996510 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0ed750 con 0x55e8257e69d0 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 -- 172.21.15.7:0/200996510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e69d0 msgr2=0x55e825802df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 --2- 172.21.15.7:0/200996510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e69d0 0x55e825802df0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0b8c20 tx=0x7f2e4c0f3690 comp rx=0 tx=0).stop 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 -- 172.21.15.7:0/200996510 shutdown_connections 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 --2- 172.21.15.7:0/200996510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257f1bb0 0x55e8257e9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 --2- 172.21.15.7:0/200996510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257e72f0 0x55e8257ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 --2- 172.21.15.7:0/200996510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e69d0 0x55e825802df0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 -- 172.21.15.7:0/200996510 >> 172.21.15.7:0/200996510 conn(0x55e8257e2f60 msgr2=0x55e8257e9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 -- 172.21.15.7:0/200996510 shutdown_connections 2024-09-10T10:45:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 -- 172.21.15.7:0/200996510 wait complete. 2024-09-10T10:45:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 -- start start 2024-09-10T10:45:45.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.681+0000 7f2e6485a740 1 Processor -- start 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e69d0 0x55e8257eaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e72f0 0x55e8257eaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e6485a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257f1bb0 0x55e8257eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e6485a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e825803a10 con 0x55e8257e72f0 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e6485a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e825803340 con 0x55e8257f1bb0 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e6485a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e8258040e0 con 0x55e8257e69d0 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e5b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e72f0 0x55e8257eaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e6485a740 10 client.0 _ll_register_callbacks cb 0x7f2e54008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e5b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e72f0 0x55e8257eaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58880/0 (socket says 172.21.15.7:58880) 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e5b7fe640 1 -- 172.21.15.7:0/4172607750 learned_addr learned my addr 172.21.15.7:0/4172607750 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e69d0 0x55e8257eaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257f1bb0 0x55e8257eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e5b7fe640 1 -- 172.21.15.7:0/4172607750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e69d0 msgr2=0x55e8257eaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e5b7fe640 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e69d0 0x55e8257eaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e5b7fe640 1 -- 172.21.15.7:0/4172607750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257f1bb0 msgr2=0x55e8257eb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e5b7fe640 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257f1bb0 0x55e8257eb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e5b7fe640 1 -- 172.21.15.7:0/4172607750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e82580adf0 con 0x55e8257e72f0 2024-09-10T10:45:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e60a1e640 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257f1bb0 0x55e8257eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e5b7fe640 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e72f0 0x55e8257eaf60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009a40 tx=0x7f2e5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c4a1820b7d9113 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.682+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/4172607750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50029060 con 0x55e8257e72f0 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e6485a740 1 -- 172.21.15.7:0/4172607750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4c0bc950 con 0x55e8257e72f0 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/4172607750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f2e50013880 con 0x55e8257e72f0 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/4172607750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2e50017040 con 0x55e8257e72f0 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e6485a740 10 client.15090 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e6485a740 1 -- 172.21.15.7:0/4172607750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e82580c100 con 0x55e8257e72f0 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/4172607750 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c0bc950 con 0x55e8257e72f0 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/4172607750 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2e5002f3d0 con 0x55e8257e72f0 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e6485a740 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e6485a740 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e6485a740 20 client.15090 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:45.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e6485a740 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x55e825870b10 0x55e82586f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e6485a740 4 client.15090 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e6485a740 1 -- 172.21.15.7:0/4172607750 --> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] -- command(tid 0: {"prefix": "status"}) -- 0x55e825754b10 con 0x55e825870b10 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.683+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x55e825870b10 0x55e82586f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e5bfff640 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x55e825870b10 0x55e82586f820 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bba6a406845ad6 server_cookie=9dfce1469262e120 in_seq=0 out_seq=0 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e3b7fe640 10 client.15090 ms_handle_connect on v2:172.21.15.103:6838/2537001809 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e3b7fe640 1 -- 172.21.15.7:0/4172607750 <== mds.1 v2:172.21.15.103:6838/2537001809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55e825754b10 con 0x55e825870b10 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e3b7fe640 10 client.15090 handle_command_reply: tid=0 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 client.15090 shutdown 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 -- 172.21.15.7:0/4172607750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e72f0 msgr2=0x55e8257eaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e72f0 0x55e8257eaf60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009a40 tx=0x7f2e5000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 -- 172.21.15.7:0/4172607750 shutdown_connections 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e8257f1bb0 0x55e8257eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8257e72f0 0x55e8257eaf60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.103:6838/2537001809,v1:172.21.15.103:6839/2537001809] conn(0x55e825870b10 0x55e82586f820 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 --2- 172.21.15.7:0/4172607750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e8257e69d0 0x55e8257eaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 -- 172.21.15.7:0/4172607750 >> 172.21.15.7:0/4172607750 conn(0x55e8257e2f60 msgr2=0x55e8257e8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e3b7fe640 0 client.15090 ms_handle_reset on v2:172.21.15.103:6838/2537001809 2024-09-10T10:45:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 -- 172.21.15.7:0/4172607750 shutdown_connections 2024-09-10T10:45:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.684+0000 7f2e6485a740 1 -- 172.21.15.7:0/4172607750 wait complete. 2024-09-10T10:45:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e6485a740 20 client.15090 trim_cache size 0 max 16384 2024-09-10T10:45:45.692 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T10:45:45.692 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "65af4d96-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24482, 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 86.625258965, 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 81, 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 81, 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 104.459342725 2024-09-10T10:45:45.693 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T10:45:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 -- 172.21.15.7:0/3649671601 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2e440a2c10 msgr2=0x7f2e440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2e440a2c10 0x7f2e440a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c069760 tx=0x7f2e54007510 comp rx=0 tx=0).stop 2024-09-10T10:45:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 -- 172.21.15.7:0/3649671601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c1009d0 msgr2=0x7f2e5c1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c1009d0 0x7f2e5c1a0630 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005ab0 tx=0x7f2e4c0042f0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 -- 172.21.15.7:0/3649671601 shutdown_connections 2024-09-10T10:45:45.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f2e440a2c10 0x7f2e440a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e5c10e4f0 0x7f2e5c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e5c101380 0x7f2e5c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 --2- 172.21.15.7:0/3649671601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c1009d0 0x7f2e5c1a0630 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:45.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 -- 172.21.15.7:0/3649671601 >> 172.21.15.7:0/3649671601 conn(0x7f2e5c0fc7d0 msgr2=0x7f2e5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:45.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 -- 172.21.15.7:0/3649671601 shutdown_connections 2024-09-10T10:45:45.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:45.685+0000 7f2e62481640 1 -- 172.21.15.7:0/3649671601 wait complete. 2024-09-10T10:45:46.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T10:45:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:46.867 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:46 smithi007 ceph-mon[30873]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:46.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:46 smithi103 ceph-mon[35160]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:46.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:46 smithi133 ceph-mon[35282]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T10:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.095+0000 7f4507e45640 1 -- 172.21.15.7:0/1677142933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4500102da0 msgr2=0x7f4500103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.095+0000 7f4507e45640 1 --2- 172.21.15.7:0/1677142933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4500102da0 0x7f4500103180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009900 tx=0x7f44fc02a480 comp rx=0 tx=0).stop 2024-09-10T10:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.095+0000 7f4507e45640 1 -- 172.21.15.7:0/1677142933 shutdown_connections 2024-09-10T10:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.095+0000 7f4507e45640 1 --2- 172.21.15.7:0/1677142933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f450010c180 0x7f450010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.095+0000 7f4507e45640 1 --2- 172.21.15.7:0/1677142933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45001036c0 0x7f450010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.095+0000 7f4507e45640 1 --2- 172.21.15.7:0/1677142933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4500102da0 0x7f4500103180 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.095+0000 7f4507e45640 1 -- 172.21.15.7:0/1677142933 >> 172.21.15.7:0/1677142933 conn(0x7f45000fc7a0 msgr2=0x7f45000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.095+0000 7f4507e45640 1 -- 172.21.15.7:0/1677142933 shutdown_connections 2024-09-10T10:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.095+0000 7f4507e45640 1 -- 172.21.15.7:0/1677142933 wait complete. 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4507e45640 1 Processor -- start 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4507e45640 1 -- start start 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4507e45640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4500102da0 0x7f4500100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4507e45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45001036c0 0x7f4500100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4507e45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f450010c180 0x7f4500101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4507e45640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4500111f60 con 0x7f4500102da0 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4507e45640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4500111de0 con 0x7f45001036c0 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4507e45640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45001120e0 con 0x7f450010c180 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4505be1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4500102da0 0x7f4500100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4505be1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4500102da0 0x7f4500100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58896/0 (socket says 172.21.15.7:58896) 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.096+0000 7f4505be1640 1 -- 172.21.15.7:0/361301792 learned_addr learned my addr 172.21.15.7:0/361301792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f45053e0640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45001036c0 0x7f4500100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f45063e2640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f450010c180 0x7f4500101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f4505be1640 1 -- 172.21.15.7:0/361301792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f450010c180 msgr2=0x7f4500101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f4505be1640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f450010c180 0x7f4500101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f4505be1640 1 -- 172.21.15.7:0/361301792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45001036c0 msgr2=0x7f4500100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f4505be1640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45001036c0 0x7f4500100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f4505be1640 1 -- 172.21.15.7:0/361301792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45001a6980 con 0x7f4500102da0 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f45063e2640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f450010c180 0x7f4500101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f4505be1640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4500102da0 0x7f4500100a00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f44fc006010 tx=0x7f44fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f44f6ffd640 1 -- 172.21.15.7:0/361301792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc03f030 con 0x7f4500102da0 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f4507e45640 1 -- 172.21.15.7:0/361301792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45001a6c10 con 0x7f4500102da0 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f44f6ffd640 1 -- 172.21.15.7:0/361301792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc003ce0 con 0x7f4500102da0 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f44f6ffd640 1 -- 172.21.15.7:0/361301792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc035070 con 0x7f4500102da0 2024-09-10T10:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.097+0000 7f4507e45640 1 -- 172.21.15.7:0/361301792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45001a70f0 con 0x7f4500102da0 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.099+0000 7f44f6ffd640 1 -- 172.21.15.7:0/361301792 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f44fc042030 con 0x7f4500102da0 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f450a21e740 1 Processor -- start 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f450a21e740 1 -- start start 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f450a21e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f60db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f450a21e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f6b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f450a21e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f6bbb0 0x55b222f6dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f450a21e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b222ec7090 con 0x55b222f609d0 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f450a21e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b222e6ea70 con 0x55b222f6bbb0 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f450a21e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b222ec3050 con 0x55b222f612f0 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f45063e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f60db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f45063e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f60db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58898/0 (socket says 172.21.15.7:58898) 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f44f6ffd640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44ec0a2c10 0x7f44ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f45063e2640 1 -- 172.21.15.7:0/2438646530 learned_addr learned my addr 172.21.15.7:0/2438646530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f45053e0640 1 --2- 172.21.15.7:0/2438646530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f6bbb0 0x55b222f6dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f45063e2640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44ec0a2c10 0x7f44ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f44f6ffd640 1 -- 172.21.15.7:0/361301792 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f44fc0ef6d0 con 0x7f4500102da0 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.100+0000 7f4505be1640 1 --2- 172.21.15.7:0/2438646530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f6b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.101+0000 7f45063e2640 1 -- 172.21.15.7:0/2438646530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 msgr2=0x55b222f6b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.101+0000 7f45063e2640 1 --2- 172.21.15.7:0/2438646530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f6b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.101+0000 7f45063e2640 1 -- 172.21.15.7:0/2438646530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f6bbb0 msgr2=0x55b222f6dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.101+0000 7f45063e2640 1 --2- 172.21.15.7:0/2438646530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f6bbb0 0x55b222f6dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.101+0000 7f45063e2640 1 -- 172.21.15.7:0/2438646530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b222e7b130 con 0x55b222f609d0 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.101+0000 7f45063e2640 1 --2- 172.21.15.7:0/2438646530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f60db0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f44f8014510 tx=0x7f44f8014ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357745fb7d4c72f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.101+0000 7f44ebfff640 1 -- 172.21.15.7:0/2438646530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f802d070 con 0x55b222f609d0 2024-09-10T10:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.101+0000 7f44ebfff640 1 -- 172.21.15.7:0/2438646530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f8020e30 con 0x55b222f609d0 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.101+0000 7f44ebfff640 1 -- 172.21.15.7:0/2438646530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f8028650 con 0x55b222f609d0 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 -- 172.21.15.7:0/2438646530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 msgr2=0x55b222f60db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f45063e2640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44ec0a2c10 0x7f44ec0a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f44f802a000 tx=0x7f44f8026000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 --2- 172.21.15.7:0/2438646530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f60db0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f44f8014510 tx=0x7f44f8014ea0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 -- 172.21.15.7:0/2438646530 shutdown_connections 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 --2- 172.21.15.7:0/2438646530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f6bbb0 0x55b222f6dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 --2- 172.21.15.7:0/2438646530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f6b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 --2- 172.21.15.7:0/2438646530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f60db0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 -- 172.21.15.7:0/2438646530 >> 172.21.15.7:0/2438646530 conn(0x55b222f5cf60 msgr2=0x55b222f5d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 -- 172.21.15.7:0/2438646530 shutdown_connections 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 -- 172.21.15.7:0/2438646530 wait complete. 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 -- start start 2024-09-10T10:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.102+0000 7f450a21e740 1 Processor -- start 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f450a21e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f6b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f450a21e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f68730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f450a21e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f6bbb0 0x55b222f68c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f450a21e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b222f6e780 con 0x55b222f609d0 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f450a21e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b222e81f20 con 0x55b222f6bbb0 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f450a21e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b222f6e900 con 0x55b222f612f0 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f450a21e740 10 client.0 _ll_register_callbacks cb 0x55b222e4b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f4505be1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f6b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f4505be1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f6b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58908/0 (socket says 172.21.15.7:58908) 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f4505be1640 1 -- 172.21.15.7:0/1093582681 learned_addr learned my addr 172.21.15.7:0/1093582681 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f45053e0640 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f68730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f45063e2640 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f6bbb0 0x55b222f68c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f4505be1640 1 -- 172.21.15.7:0/1093582681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 msgr2=0x55b222f68730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f4505be1640 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f68730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f4505be1640 1 -- 172.21.15.7:0/1093582681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f6bbb0 msgr2=0x55b222f68c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f4505be1640 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f6bbb0 0x55b222f68c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f4505be1640 1 -- 172.21.15.7:0/1093582681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b222c382b0 con 0x55b222f609d0 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f45053e0640 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f68730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f4505be1640 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f6b3e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f44fc000c00 tx=0x7f44fc0bbcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b265eea7bc3f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.103+0000 7f44e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f44e97fa640 1 -- 172.21.15.7:0/1093582681 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc049030 con 0x55b222f609d0 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f44e97fa640 1 -- 172.21.15.7:0/1093582681 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f44fc0b8510 con 0x55b222f609d0 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f450a21e740 1 -- 172.21.15.7:0/1093582681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44f8020e30 con 0x55b222f609d0 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f44e97fa640 1 -- 172.21.15.7:0/1093582681 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f44fc0044e0 con 0x55b222f609d0 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f450a21e740 10 client.15102 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f450a21e740 1 -- 172.21.15.7:0/1093582681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b222f7a030 con 0x55b222f609d0 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f44e97fa640 1 -- 172.21.15.7:0/1093582681 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f8020e30 con 0x55b222f609d0 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f44e97fa640 1 -- 172.21.15.7:0/1093582681 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f44fc0c4680 con 0x55b222f609d0 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f450a21e740 10 client.15102 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f450a21e740 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f450a21e740 20 client.15102 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f450a21e740 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b222f7c6e0 0x55b222f7db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f450a21e740 4 client.15102 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f450a21e740 1 -- 172.21.15.7:0/1093582681 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b222ed4600 con 0x55b222f7c6e0 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.104+0000 7f45053e0640 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b222f7c6e0 0x55b222f7db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.105+0000 7f45053e0640 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b222f7c6e0 0x55b222f7db50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9008327bd462c85 server_cookie=d8db88488635ed63 in_seq=0 out_seq=0 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.105+0000 7f44e97fa640 10 client.15102 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f44e97fa640 1 -- 172.21.15.7:0/1093582681 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b222ed4600 con 0x55b222f7c6e0 2024-09-10T10:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f44e97fa640 10 client.15102 handle_command_reply: tid=0 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 client.15102 shutdown 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 -- 172.21.15.7:0/1093582681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 msgr2=0x55b222f6b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f6b3e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f44fc000c00 tx=0x7f44fc0bbcf0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 -- 172.21.15.7:0/1093582681 shutdown_connections 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b222f7c6e0 0x55b222f7db50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f6bbb0 0x55b222f68c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f68730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f44e97fa640 0 client.15102 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 --2- 172.21.15.7:0/1093582681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f6b3e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 -- 172.21.15.7:0/1093582681 >> 172.21.15.7:0/1093582681 conn(0x55b222f5cf60 msgr2=0x55b222f62c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 -- 172.21.15.7:0/1093582681 shutdown_connections 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 1 -- 172.21.15.7:0/1093582681 wait complete. 2024-09-10T10:45:47.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.106+0000 7f450a21e740 20 client.15102 trim_cache size 0 max 16384 2024-09-10T10:45:47.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.124+0000 7f450a21e740 1 Processor -- start 2024-09-10T10:45:47.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.124+0000 7f450a21e740 1 -- start start 2024-09-10T10:45:47.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f450a21e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f7db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f450a21e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f612f0 0x55b222f7b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f450a21e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f6bbb0 0x55b222f7b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f450a21e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b222f7be90 con 0x55b222f609d0 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f450a21e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b222f7c560 con 0x55b222f612f0 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f450a21e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b222f7cc30 con 0x55b222f6bbb0 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f4505be1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f7db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f4505be1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f7db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58920/0 (socket says 172.21.15.7:58920) 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f4505be1640 1 -- 172.21.15.7:0/1101020449 learned_addr learned my addr 172.21.15.7:0/1101020449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f45063e2640 1 --2- 172.21.15.7:0/1101020449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f6bbb0 0x55b222f7b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f45053e0640 1 --2- 172.21.15.7:0/1101020449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f612f0 0x55b222f7b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f4505be1640 1 -- 172.21.15.7:0/1101020449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f6bbb0 msgr2=0x55b222f7b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f4505be1640 1 --2- 172.21.15.7:0/1101020449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f6bbb0 0x55b222f7b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f4505be1640 1 -- 172.21.15.7:0/1101020449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f612f0 msgr2=0x55b222f7b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f4505be1640 1 --2- 172.21.15.7:0/1101020449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f612f0 0x55b222f7b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f4505be1640 1 -- 172.21.15.7:0/1101020449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b222f61120 con 0x55b222f609d0 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f45053e0640 1 --2- 172.21.15.7:0/1101020449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f612f0 0x55b222f7b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f4505be1640 1 --2- 172.21.15.7:0/1101020449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f7db50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0a8040 tx=0x7f44fc02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b9fb463d580ed8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f44eb7fe640 1 -- 172.21.15.7:0/1101020449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc049070 con 0x55b222f609d0 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.125+0000 7f44eb7fe640 1 -- 172.21.15.7:0/1101020449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc0bc5a0 con 0x55b222f609d0 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.126+0000 7f44eb7fe640 1 -- 172.21.15.7:0/1101020449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0b8510 con 0x55b222f609d0 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.126+0000 7f450a21e740 1 -- 172.21.15.7:0/1101020449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 msgr2=0x55b222f7db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.126+0000 7f450a21e740 1 --2- 172.21.15.7:0/1101020449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f7db50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0a8040 tx=0x7f44fc02a480 comp rx=0 tx=0).stop 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.126+0000 7f450a21e740 1 -- 172.21.15.7:0/1101020449 shutdown_connections 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.126+0000 7f450a21e740 1 --2- 172.21.15.7:0/1101020449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f6bbb0 0x55b222f7b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.126+0000 7f450a21e740 1 --2- 172.21.15.7:0/1101020449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f612f0 0x55b222f7b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.126+0000 7f450a21e740 1 --2- 172.21.15.7:0/1101020449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f609d0 0x55b222f7db50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.126+0000 7f450a21e740 1 -- 172.21.15.7:0/1101020449 >> 172.21.15.7:0/1101020449 conn(0x55b222f5cf60 msgr2=0x55b222f57ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.126+0000 7f450a21e740 1 -- 172.21.15.7:0/1101020449 shutdown_connections 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.126+0000 7f450a21e740 1 -- 172.21.15.7:0/1101020449 wait complete. 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.127+0000 7f450a21e740 1 -- start start 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.127+0000 7f450a21e740 1 Processor -- start 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.127+0000 7f450a21e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f609d0 0x55b222f80f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.127+0000 7f450a21e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.127+0000 7f450a21e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f6bbb0 0x55b222fe4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f450a21e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b222f7d9d0 con 0x55b222f6bbb0 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f450a21e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b222f7d300 con 0x55b222f609d0 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f450a21e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b222f7e0a0 con 0x55b222f612f0 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45063e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f6bbb0 0x55b222fe4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f450a21e740 10 client.0 _ll_register_callbacks cb 0x7f44fc03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45063e2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f6bbb0 0x55b222fe4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58926/0 (socket says 172.21.15.7:58926) 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45063e2640 1 -- 172.21.15.7:0/1465731736 learned_addr learned my addr 172.21.15.7:0/1465731736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f4505be1640 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f609d0 0x55b222f80f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45053e0640 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45063e2640 1 -- 172.21.15.7:0/1465731736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 msgr2=0x55b222f814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45063e2640 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45063e2640 1 -- 172.21.15.7:0/1465731736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f609d0 msgr2=0x55b222f80f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45063e2640 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f609d0 0x55b222f80f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45063e2640 1 -- 172.21.15.7:0/1465731736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b222fe5320 con 0x55b222f6bbb0 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45053e0640 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f4505be1640 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f609d0 0x55b222f80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f45063e2640 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f6bbb0 0x55b222fe4de0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f44f8017000 tx=0x7f44f8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8e2a8e4ce04a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f44e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f44e97fa640 1 -- 172.21.15.7:0/1465731736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f802d040 con 0x55b222f6bbb0 2024-09-10T10:45:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f44e97fa640 1 -- 172.21.15.7:0/1465731736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7f44f8013ef0 con 0x55b222f6bbb0 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.128+0000 7f450a21e740 1 -- 172.21.15.7:0/1465731736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44fc0bc5a0 con 0x55b222f6bbb0 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f44e97fa640 1 -- 172.21.15.7:0/1465731736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f44f80140f0 con 0x55b222f6bbb0 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f450a21e740 10 client.15108 fetch_fsmap learned FSMap version 24 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f450a21e740 1 -- 172.21.15.7:0/1465731736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b222fe6430 con 0x55b222f6bbb0 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f44e97fa640 1 -- 172.21.15.7:0/1465731736 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc0bc5a0 con 0x55b222f6bbb0 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f44e97fa640 1 -- 172.21.15.7:0/1465731736 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f44f8027030 con 0x55b222f6bbb0 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f450a21e740 10 client.15108 fetch_fsmap finished waiting for FSMap version 24 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f450a21e740 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f450a21e740 20 client.15108 populate_metadata read hostname 'smithi007' 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f450a21e740 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b222feaea0 0x55b222fe9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f450a21e740 4 client.15108 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f450a21e740 1 -- 172.21.15.7:0/1465731736 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "status"}) -- 0x55b222eceb10 con 0x55b222feaea0 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.129+0000 7f4505be1640 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b222feaea0 0x55b222fe9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f4505be1640 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b222feaea0 0x55b222fe9bb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=966cc39d238c06a server_cookie=34ecf09f0b9fa48c in_seq=0 out_seq=0 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f44e97fa640 10 client.15108 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f44e97fa640 1 -- 172.21.15.7:0/1465731736 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b222eceb10 con 0x55b222feaea0 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f44e97fa640 10 client.15108 handle_command_reply: tid=0 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 client.15108 shutdown 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 -- 172.21.15.7:0/1465731736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f6bbb0 msgr2=0x55b222fe4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f6bbb0 0x55b222fe4de0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f44f8017000 tx=0x7f44f8009f80 comp rx=0 tx=0).stop 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 -- 172.21.15.7:0/1465731736 shutdown_connections 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b222feaea0 0x55b222fe9bb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b222f6bbb0 0x55b222fe4de0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b222f612f0 0x55b222f814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 --2- 172.21.15.7:0/1465731736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b222f609d0 0x55b222f80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f44e97fa640 0 client.15108 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 -- 172.21.15.7:0/1465731736 >> 172.21.15.7:0/1465731736 conn(0x55b222f5cf60 msgr2=0x55b222f5d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 -- 172.21.15.7:0/1465731736 shutdown_connections 2024-09-10T10:45:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 1 -- 172.21.15.7:0/1465731736 wait complete. 2024-09-10T10:45:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.130+0000 7f450a21e740 20 client.15108 trim_cache size 0 max 16384 2024-09-10T10:45:47.143 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T10:45:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "65af4d96-6f60-11ef-bcea-c7b262605968", 2024-09-10T10:45:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T10:45:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24499, 2024-09-10T10:45:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T10:45:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T10:45:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T10:45:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 95.108948701000003, 2024-09-10T10:45:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 24, 2024-09-10T10:45:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 81, 2024-09-10T10:45:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 81, 2024-09-10T10:45:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 114.334353788 2024-09-10T10:45:47.144 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T10:45:47.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 -- 172.21.15.7:0/361301792 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44ec0a2c10 msgr2=0x7f44ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44ec0a2c10 0x7f44ec0a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f44f802a000 tx=0x7f44f8026000 comp rx=0 tx=0).stop 2024-09-10T10:45:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 -- 172.21.15.7:0/361301792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4500102da0 msgr2=0x7f4500100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4500102da0 0x7f4500100a00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f44fc006010 tx=0x7f44fc0042f0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 -- 172.21.15.7:0/361301792 shutdown_connections 2024-09-10T10:45:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7f44ec0a2c10 0x7f44ec0a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f450010c180 0x7f4500101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45001036c0 0x7f4500100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 --2- 172.21.15.7:0/361301792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4500102da0 0x7f4500100a00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 -- 172.21.15.7:0/361301792 >> 172.21.15.7:0/361301792 conn(0x7f45000fc7a0 msgr2=0x7f45000fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 -- 172.21.15.7:0/361301792 shutdown_connections 2024-09-10T10:45:47.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.131+0000 7f4507e45640 1 -- 172.21.15.7:0/361301792 wait complete. 2024-09-10T10:45:47.768 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T10:45:47.778 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T10:45:47.778 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T10:45:47.779 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T10:45:47.779 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-10T10:45:47.780 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T10:45:47.780 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T10:45:47.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.890+0000 7fa7c8986640 1 -- 172.21.15.7:0/1417048433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7c4100a50 msgr2=0x7fa7c4100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.890+0000 7fa7c8986640 1 --2- 172.21.15.7:0/1417048433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7c4100a50 0x7fa7c4100e50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009900 tx=0x7fa7b402a410 comp rx=0 tx=0).stop 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.890+0000 7fa7c8986640 1 -- 172.21.15.7:0/1417048433 shutdown_connections 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.890+0000 7fa7c8986640 1 --2- 172.21.15.7:0/1417048433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7c4102600 0x7fa7c4108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.890+0000 7fa7c8986640 1 --2- 172.21.15.7:0/1417048433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c4101c40 0x7fa7c41020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.890+0000 7fa7c8986640 1 --2- 172.21.15.7:0/1417048433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7c4100a50 0x7fa7c4100e50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.890+0000 7fa7c8986640 1 -- 172.21.15.7:0/1417048433 >> 172.21.15.7:0/1417048433 conn(0x7fa7c40fc1b0 msgr2=0x7fa7c40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.890+0000 7fa7c8986640 1 -- 172.21.15.7:0/1417048433 shutdown_connections 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.890+0000 7fa7c8986640 1 -- 172.21.15.7:0/1417048433 wait complete. 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c8986640 1 Processor -- start 2024-09-10T10:45:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c8986640 1 -- start start 2024-09-10T10:45:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c8986640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7c4100a50 0x7fa7c4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c8986640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c4101c40 0x7fa7c41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c8986640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7c4102600 0x7fa7c419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c8986640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c410d820 con 0x7fa7c4100a50 2024-09-10T10:45:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c8986640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c410d6a0 con 0x7fa7c4101c40 2024-09-10T10:45:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c8986640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c410d9a0 con 0x7fa7c4102600 2024-09-10T10:45:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7c4100a50 0x7fa7c4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7c4100a50 0x7fa7c4197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58934/0 (socket says 172.21.15.7:58934) 2024-09-10T10:45:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.891+0000 7fa7c259c640 1 -- 172.21.15.7:0/2856908434 learned_addr learned my addr 172.21.15.7:0/2856908434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7c1d9b640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c4101c40 0x7fa7c41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7c2d9d640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7c4102600 0x7fa7c419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7c259c640 1 -- 172.21.15.7:0/2856908434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7c4102600 msgr2=0x7fa7c419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7c259c640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7c4102600 0x7fa7c419f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7c259c640 1 -- 172.21.15.7:0/2856908434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c4101c40 msgr2=0x7fa7c41982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7c259c640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c4101c40 0x7fa7c41982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7c259c640 1 -- 172.21.15.7:0/2856908434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c419f9d0 con 0x7fa7c4100a50 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7c259c640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7c4100a50 0x7fa7c4197db0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4031610 tx=0x7fa7b40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7b37fe640 1 -- 172.21.15.7:0/2856908434 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b403f030 con 0x7fa7c4100a50 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7c8986640 1 -- 172.21.15.7:0/2856908434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c419fc60 con 0x7fa7c4100a50 2024-09-10T10:45:47.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7b37fe640 1 -- 172.21.15.7:0/2856908434 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4003ce0 con 0x7fa7c4100a50 2024-09-10T10:45:47.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7b37fe640 1 -- 172.21.15.7:0/2856908434 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4035070 con 0x7fa7c4100a50 2024-09-10T10:45:47.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.892+0000 7fa7c8986640 1 -- 172.21.15.7:0/2856908434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c41a0150 con 0x7fa7c4100a50 2024-09-10T10:45:47.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.893+0000 7fa7c8986640 1 -- 172.21.15.7:0/2856908434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c4107d70 con 0x7fa7c4100a50 2024-09-10T10:45:47.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.894+0000 7fa7b37fe640 1 -- 172.21.15.7:0/2856908434 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa7b4042030 con 0x7fa7c4100a50 2024-09-10T10:45:47.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.895+0000 7fa7b37fe640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7ac0a2cd0 0x7fa7ac0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:47.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.895+0000 7fa7b37fe640 1 -- 172.21.15.7:0/2856908434 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7fa7b40efae0 con 0x7fa7c4100a50 2024-09-10T10:45:47.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.895+0000 7fa7c1d9b640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7ac0a2cd0 0x7fa7ac0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:47.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.896+0000 7fa7c1d9b640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7ac0a2cd0 0x7fa7ac0a5190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8005fd0 tx=0x7fa7b8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:47.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:47.899+0000 7fa7b37fe640 1 -- 172.21.15.7:0/2856908434 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7b40bc890 con 0x7fa7c4100a50 2024-09-10T10:45:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.122+0000 7fa7c8986640 1 -- 172.21.15.7:0/2856908434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fa7c402d950 con 0x7fa7c4100a50 2024-09-10T10:45:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.598+0000 7fa7b37fe640 1 -- 172.21.15.7:0/2856908434 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7fa7b40bc190 con 0x7fa7c4100a50 2024-09-10T10:45:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.600+0000 7fa7c8986640 1 -- 172.21.15.7:0/2856908434 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7ac0a2cd0 msgr2=0x7fa7ac0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.600+0000 7fa7c8986640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7ac0a2cd0 0x7fa7ac0a5190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8005fd0 tx=0x7fa7b8005850 comp rx=0 tx=0).stop 2024-09-10T10:45:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.600+0000 7fa7c8986640 1 -- 172.21.15.7:0/2856908434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7c4100a50 msgr2=0x7fa7c4197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.600+0000 7fa7c8986640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7c4100a50 0x7fa7c4197db0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4031610 tx=0x7fa7b40043b0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.601+0000 7fa7c8986640 1 -- 172.21.15.7:0/2856908434 shutdown_connections 2024-09-10T10:45:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.601+0000 7fa7c8986640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7fa7ac0a2cd0 0x7fa7ac0a5190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.601+0000 7fa7c8986640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7c4102600 0x7fa7c419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.601+0000 7fa7c8986640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7c4101c40 0x7fa7c41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.601+0000 7fa7c8986640 1 --2- 172.21.15.7:0/2856908434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7c4100a50 0x7fa7c4197db0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.601+0000 7fa7c8986640 1 -- 172.21.15.7:0/2856908434 >> 172.21.15.7:0/2856908434 conn(0x7fa7c40fc1b0 msgr2=0x7fa7c4104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.601+0000 7fa7c8986640 1 -- 172.21.15.7:0/2856908434 shutdown_connections 2024-09-10T10:45:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:48.601+0000 7fa7c8986640 1 -- 172.21.15.7:0/2856908434 wait complete. 2024-09-10T10:45:48.621 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T10:45:48.648 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T10:45:48.648 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T10:45:48.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:48.648 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T10:45:48.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:48.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-10T10:45:48.737 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-09-10T10:45:48.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:45:48.739 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-10T10:45:48.739 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:45:48.739 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:48.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T10:45:48.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:48 smithi103 ceph-mon[35160]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:48.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:48 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2856908434' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-10T10:45:48.922 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:48 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: ignoring --setuser ceph since I am not root 2024-09-10T10:45:48.923 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:48 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: ignoring --setgroup ceph since I am not root 2024-09-10T10:45:48.923 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:48 smithi103 ceph-mgr[36528]: -- 172.21.15.103:0/3722076023 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555dc3ba70e0 con 0x555dc3b84800 2024-09-10T10:45:48.923 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:48 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:48.754+0000 7f67d6cdc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:45:48.923 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:48 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:48.811+0000 7f67d6cdc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:45:48.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:48 smithi007 ceph-mon[30873]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:48.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:48 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2856908434' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-10T10:45:48.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:48 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: ignoring --setuser ceph since I am not root 2024-09-10T10:45:48.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:48 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: ignoring --setgroup ceph since I am not root 2024-09-10T10:45:48.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:48 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:48.760+0000 7fd6113c4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:45:48.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:48 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:48.822+0000 7fd6113c4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:45:48.954 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:48 smithi133 ceph-mon[35282]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:45:48.955 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:48 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2856908434' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-10T10:45:48.955 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: ignoring --setuser ceph since I am not root 2024-09-10T10:45:48.955 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: ignoring --setgroup ceph since I am not root 2024-09-10T10:45:48.955 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:48 smithi133 ceph-mgr[37098]: -- 172.21.15.133:0/2585535922 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55582b93d0e0 con 0x55582b91b400 2024-09-10T10:45:48.955 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:48.749+0000 7fc7706b0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:45:48.955 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:48.805+0000 7fc7706b0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:45:48.955 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:48.953+0000 7fc7706b0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:45:49.040 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:45:49.225 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:49.019+0000 7fc7706b0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:45:49.225 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:49.167+0000 7fc7706b0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:45:49.291 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:48 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:48.982+0000 7fd6113c4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:45:49.292 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:49.057+0000 7fd6113c4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:45:49.292 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:49.215+0000 7fd6113c4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:45:49.314 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:48 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:48.959+0000 7f67d6cdc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:45:49.314 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:49.024+0000 7f67d6cdc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:45:49.314 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:49.172+0000 7f67d6cdc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:45:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.496+0000 7efe08238640 1 -- 172.21.15.7:0/2577971533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00100e20 msgr2=0x7efe00101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.496+0000 7efe08238640 1 --2- 172.21.15.7:0/2577971533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00100e20 0x7efe00101200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009900 tx=0x7efdfc02a480 comp rx=0 tx=0).stop 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.496+0000 7efe08238640 1 -- 172.21.15.7:0/2577971533 shutdown_connections 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.496+0000 7efe08238640 1 --2- 172.21.15.7:0/2577971533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00102ef0 0x7efe0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.496+0000 7efe08238640 1 --2- 172.21.15.7:0/2577971533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe00102570 0x7efe001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.496+0000 7efe08238640 1 --2- 172.21.15.7:0/2577971533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00100e20 0x7efe00101200 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.496+0000 7efe08238640 1 -- 172.21.15.7:0/2577971533 >> 172.21.15.7:0/2577971533 conn(0x7efe000fc7d0 msgr2=0x7efe000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.496+0000 7efe08238640 1 -- 172.21.15.7:0/2577971533 shutdown_connections 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.496+0000 7efe08238640 1 -- 172.21.15.7:0/2577971533 wait complete. 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe08238640 1 Processor -- start 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe08238640 1 -- start start 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe08238640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00100e20 0x7efe00198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe08238640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00102570 0x7efe00198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe08238640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe00102ef0 0x7efe0019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe08238640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe00110fd0 con 0x7efe00102570 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe08238640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe00110e50 con 0x7efe00102ef0 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe08238640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efe00111150 con 0x7efe00100e20 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe057d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00102570 0x7efe00198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe057d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00102570 0x7efe00198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58968/0 (socket says 172.21.15.7:58968) 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe057d3640 1 -- 172.21.15.7:0/3083684442 learned_addr learned my addr 172.21.15.7:0/3083684442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.497+0000 7efe05fd4640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00100e20 0x7efe00198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efe067d5640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe00102ef0 0x7efe0019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efe057d3640 1 -- 172.21.15.7:0/3083684442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00100e20 msgr2=0x7efe00198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efe057d3640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00100e20 0x7efe00198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efe057d3640 1 -- 172.21.15.7:0/3083684442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe00102ef0 msgr2=0x7efe0019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efe057d3640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe00102ef0 0x7efe0019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efe057d3640 1 -- 172.21.15.7:0/3083684442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0019d510 con 0x7efe00102570 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efe057d3640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00102570 0x7efe00198aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efdf000e9f0 tx=0x7efdf000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efdf6ffd640 1 -- 172.21.15.7:0/3083684442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf000cd80 con 0x7efe00102570 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efe08238640 1 -- 172.21.15.7:0/3083684442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0019d800 con 0x7efe00102570 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efdf6ffd640 1 -- 172.21.15.7:0/3083684442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf0010910 con 0x7efe00102570 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efdf6ffd640 1 -- 172.21.15.7:0/3083684442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf00188f0 con 0x7efe00102570 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.498+0000 7efe08238640 1 -- 172.21.15.7:0/3083684442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe001a50e0 con 0x7efe00102570 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.499+0000 7efe08238640 1 -- 172.21.15.7:0/3083684442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe00101200 con 0x7efe00102570 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.501+0000 7efdf6ffd640 1 -- 172.21.15.7:0/3083684442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 137900+0+0 (secure 0 0 0) 0x7efdf0010ac0 con 0x7efe00102570 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.501+0000 7efdf6ffd640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efdec0a2ce0 0x7efdec0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.501+0000 7efe05fd4640 1 -- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efdec0a2ce0 msgr2=0x7efdec0a51a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3525096194 2024-09-10T10:45:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.501+0000 7efdf6ffd640 1 -- 172.21.15.7:0/3083684442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8219+0+0 (secure 0 0 0) 0x7efdf00cf910 con 0x7efe00102570 2024-09-10T10:45:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.501+0000 7efe05fd4640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efdec0a2ce0 0x7efdec0a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:45:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.505+0000 7efdf6ffd640 1 -- 172.21.15.7:0/3083684442 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdf009c0d0 con 0x7efe00102570 2024-09-10T10:45:49.616 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:49.310+0000 7fc7706b0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:45:49.616 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:49.367+0000 7fd6113c4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:45:49.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.702+0000 7efe05fd4640 1 -- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efdec0a2ce0 msgr2=0x7efdec0a51a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3525096194 2024-09-10T10:45:49.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.702+0000 7efe05fd4640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efdec0a2ce0 0x7efdec0a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:45:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.720+0000 7efe08238640 1 -- 172.21.15.7:0/3083684442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efe001998c0 con 0x7efe00102570 2024-09-10T10:45:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.721+0000 7efdf6ffd640 1 -- 172.21.15.7:0/3083684442 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7efe001998c0 con 0x7efe00102570 2024-09-10T10:45:49.722 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:45:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 -- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efdec0a2ce0 msgr2=0x7efdec0a51a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:45:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efdec0a2ce0 0x7efdec0a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 -- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00102570 msgr2=0x7efe00198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00102570 0x7efe00198aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efdf000e9f0 tx=0x7efdf000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 -- 172.21.15.7:0/3083684442 shutdown_connections 2024-09-10T10:45:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:6800/3525096194,v1:172.21.15.7:6801/3525096194] conn(0x7efdec0a2ce0 0x7efdec0a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe00102ef0 0x7efe0019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe00102570 0x7efe00198aa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 --2- 172.21.15.7:0/3083684442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe00100e20 0x7efe00198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 -- 172.21.15.7:0/3083684442 >> 172.21.15.7:0/3083684442 conn(0x7efe000fc7d0 msgr2=0x7efe0006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 -- 172.21.15.7:0/3083684442 shutdown_connections 2024-09-10T10:45:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:49.723+0000 7efe08238640 1 -- 172.21.15.7:0/3083684442 wait complete. 2024-09-10T10:45:49.871 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:49 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2856908434' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T10:45:49.871 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:49 smithi103 ceph-mon[35160]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-10T10:45:49.871 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:49.313+0000 7f67d6cdc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:45:49.871 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:49.626+0000 7f67d6cdc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:45:49.872 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:49.687+0000 7f67d6cdc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:45:49.872 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:49.870+0000 7f67d6cdc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:45:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:49 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2856908434' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T10:45:49.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:49 smithi007 ceph-mon[30873]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-10T10:45:49.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:49.703+0000 7fd6113c4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:45:49.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:49.770+0000 7fd6113c4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:45:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:49 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2856908434' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T10:45:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:49 smithi133 ceph-mon[35282]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-10T10:45:49.975 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:49.621+0000 7fc7706b0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:45:49.975 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:49.681+0000 7fc7706b0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:45:49.975 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:49.864+0000 7fc7706b0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:45:50.371 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:49.967+0000 7fd6113c4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:45:50.401 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T10:45:50.401 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:45:50.401 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T10:45:50.401 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:45:50.402 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:50.402 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T10:45:50.402 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:45:50.402 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:45:50.402 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:50.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:50.402 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T10:45:50.419 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:45:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:45:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:45:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T10:45:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:50.419 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42208sec preferred_lft 42208sec 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:50.420 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T10:45:50.421 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T10:45:50.421 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:50.421 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T10:45:50.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-09-10T10:45:50.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-09-10T10:45:50.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-09-10T10:45:50.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T10:45:50.422 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T10:45:50.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:50.622 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:50 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3083684442' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:45:50.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:50.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:50.641 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:45:50.668 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T10:45:50.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:50.670 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T10:45:50.747 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T10:45:50.747 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:45:50.747 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:45:50.747 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:45:50.747 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:45:50.748 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:50.748 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T10:45:50.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:45:50.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:45:50.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:45:50.748 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T10:45:50.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:50.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:50 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3083684442' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:45:50.922 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:50.756+0000 7f67d6cdc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:45:50.922 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:50.823+0000 7f67d6cdc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:45:50.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:50 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3083684442' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:45:50.975 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:50 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:50.749+0000 7fc7706b0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:45:50.975 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:50 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:50.816+0000 7fc7706b0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:45:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:51.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:51.044 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T10:45:51.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:51.060 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-09-10T10:45:51.117 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:51.117 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T10:45:51.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:51.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T10:45:51.117 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T10:45:51.184 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:50.924+0000 7fd6113c4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:45:51.184 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:50.998+0000 7fd6113c4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:45:51.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:51.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:51.242 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T10:45:51.243 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:51.243 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T10:45:51.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T10:45:51.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T10:45:51.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T10:45:51.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T10:45:51.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T10:45:51.243 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T10:45:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:51.418 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:51.419 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T10:45:51.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T10:45:51.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T10:45:51.419 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T10:45:51.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:51.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:51.499 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:45:51.499 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:51.515 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:51.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:45:51.516 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:51.572 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-10T10:45:51.572 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-10T10:45:51.572 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/46e373e7-0ae7-4360-8dac-ecfc0d930e13 2024-09-10T10:45:51.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:51.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/46e373e7-0ae7-4360-8dac-ecfc0d930e13 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-09-10T10:45:51.672 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-09-10T10:45:51.672 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-09-10T10:45:51.672 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-10T10:45:51.784 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:51.523+0000 7fc7706b0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:45:51.784 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:51.638+0000 7fc7706b0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:45:51.801 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:51.538+0000 7f67d6cdc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:45:51.801 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:51.656+0000 7f67d6cdc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:45:51.818 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-09-10T10:45:51.818 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-09-10T10:45:51.818 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-09-10T10:45:51.818 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.103:3300,172.21.15.133:3300" 2024-09-10T10:45:51.818 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:3300,172.21.15.103:3300,172.21.15.133:3300:/volumes/qa/sv_1/46e373e7-0ae7-4360-8dac-ecfc0d930e13 2024-09-10T10:45:51.818 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=65af4d96-6f60-11ef-bcea-c7b262605968" will pass to kernel 2024-09-10T10:45:51.819 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:45:51.819 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:45:51.819 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:45:51.819 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:45:51.819 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:45:51.821 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:45:51.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:51.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:52.054 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:51 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:51.780+0000 7fd6113c4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:45:52.055 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:51 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:51.903+0000 7fd6113c4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:45:52.064 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:51.782+0000 7fc7706b0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:45:52.065 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:51.845+0000 7fc7706b0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:45:52.065 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:51.995+0000 7fc7706b0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:45:52.080 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:51.799+0000 7f67d6cdc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:45:52.080 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:51.862+0000 7f67d6cdc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:45:52.080 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:52.010+0000 7f67d6cdc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:45:52.330 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.053+0000 7fd6113c4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:45:52.331 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.119+0000 7fd6113c4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:45:52.331 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.279+0000 7fd6113c4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:45:52.351 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:52.063+0000 7fc7706b0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:45:52.351 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:52.125+0000 7fc7706b0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:45:52.351 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:52.284+0000 7fc7706b0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:45:52.364 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:52.078+0000 7f67d6cdc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:45:52.364 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:52.138+0000 7f67d6cdc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:45:52.364 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:52.296+0000 7f67d6cdc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:45:52.589 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.352+0000 7fd6113c4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:45:52.589 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.417+0000 7fd6113c4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:45:52.589 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.587+0000 7fd6113c4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:45:52.615 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:52.349+0000 7fc7706b0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:45:52.616 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:52.416+0000 7fc7706b0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:45:52.616 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:52.497+0000 7fc7706b0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:45:52.616 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:52.555+0000 7fc7706b0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:45:52.616 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:52.614+0000 7fc7706b0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:45:52.633 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:52.362+0000 7f67d6cdc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:45:52.633 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:52.430+0000 7f67d6cdc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:45:52.633 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:52.512+0000 7f67d6cdc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:45:52.633 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:52.571+0000 7f67d6cdc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:45:52.633 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:52.630+0000 7f67d6cdc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:45:52.880 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.657+0000 7fd6113c4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:45:52.881 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.728+0000 7fd6113c4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:45:52.881 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.816+0000 7fd6113c4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:45:52.881 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.879+0000 7fd6113c4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:45:53.184 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:52 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:52.942+0000 7fd6113c4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:45:53.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: Standby manager daemon z restarted 2024-09-10T10:45:53.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: Standby manager daemon z started 2024-09-10T10:45:53.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:45:53.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:45:53.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:45:53.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:45:53.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: Standby manager daemon y restarted 2024-09-10T10:45:53.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: Standby manager daemon y started 2024-09-10T10:45:53.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:45:53.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:45:53.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:45:53.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-mon[30873]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:45:53.725 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z[37094]: 2024-09-10T10:45:53.409+0000 7fc7706b0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:45:53.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: Standby manager daemon z restarted 2024-09-10T10:45:53.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: Standby manager daemon z started 2024-09-10T10:45:53.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:45:53.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:45:53.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:45:53.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:45:53.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: Standby manager daemon y restarted 2024-09-10T10:45:53.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: Standby manager daemon y started 2024-09-10T10:45:53.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:45:53.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:45:53.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:45:53.726 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:53 smithi133 ceph-mon[35282]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:45:53.849 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:45:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:45:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:53.918 INFO:teuthology.orchestra.run.smithi007.stdout:client15141 2024-09-10T10:45:53.918 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:45:53.918 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:53.922 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y[36524]: 2024-09-10T10:45:53.420+0000 7f67d6cdc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:45:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: Standby manager daemon z restarted 2024-09-10T10:45:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: Standby manager daemon z started 2024-09-10T10:45:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:45:53.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:45:53.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:45:53.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.133:0/119319497' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:45:53.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: Standby manager daemon y restarted 2024-09-10T10:45:53.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: Standby manager daemon y started 2024-09-10T10:45:53.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:45:53.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:45:53.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:45:53.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:53 smithi103 ceph-mon[35160]: from='mgr.? 172.21.15.103:0/1476411823' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:45:53.941 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:45:54.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:45:54.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:54.062 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:53 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:53.783+0000 7fd6113c4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:45:54.065 INFO:teuthology.orchestra.run.smithi007.stdout:client15141 2024-09-10T10:45:54.066 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:45:54.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:45:54.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:54.146 INFO:teuthology.orchestra.run.smithi007.stdout:client15141 2024-09-10T10:45:54.147 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/65af4d96-6f60-11ef-bcea-c7b262605968.client15141/status') 2024-09-10T10:45:54.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:54.217 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.15141 (3)192.168.0.1:0/1569316 2024-09-10T10:45:54.217 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-09-10T10:45:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-09-10T10:45:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-09-10T10:45:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:67 bytes copied, 2.6697e-05 s, 2.5 MB/s 2024-09-10T10:45:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:54.225 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T10:45:54.238 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T10:45:54.238 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T10:45:54.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-10T10:45:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.347+0000 7f3f6f4af640 1 -- 172.21.15.7:0/3823954189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f681025c0 msgr2=0x7f3f681049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.347+0000 7f3f6f4af640 1 --2- 172.21.15.7:0/3823954189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f681025c0 0x7f3f681049b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009900 tx=0x7f3f6402a410 comp rx=0 tx=0).stop 2024-09-10T10:45:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.347+0000 7f3f6f4af640 1 -- 172.21.15.7:0/3823954189 shutdown_connections 2024-09-10T10:45:54.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.348+0000 7f3f6f4af640 1 --2- 172.21.15.7:0/3823954189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f68107820 0x7f3f68109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.348+0000 7f3f6f4af640 1 --2- 172.21.15.7:0/3823954189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68104ef0 0x7f3f681072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.348+0000 7f3f6f4af640 1 --2- 172.21.15.7:0/3823954189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f681025c0 0x7f3f681049b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.348+0000 7f3f6f4af640 1 -- 172.21.15.7:0/3823954189 >> 172.21.15.7:0/3823954189 conn(0x7f3f680fc1e0 msgr2=0x7f3f680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.348+0000 7f3f6f4af640 1 -- 172.21.15.7:0/3823954189 shutdown_connections 2024-09-10T10:45:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.348+0000 7f3f6f4af640 1 -- 172.21.15.7:0/3823954189 wait complete. 2024-09-10T10:45:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.348+0000 7f3f6f4af640 1 Processor -- start 2024-09-10T10:45:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.348+0000 7f3f6f4af640 1 -- start start 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6f4af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f681025c0 0x7f3f68101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6f4af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68104ef0 0x7f3f68100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6f4af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f68107820 0x7f3f68100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6f4af640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6810d830 con 0x7f3f68107820 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6f4af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6810d6b0 con 0x7f3f68104ef0 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6f4af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6810d9b0 con 0x7f3f681025c0 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6da4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f68107820 0x7f3f68100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6da4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f68107820 0x7f3f68100ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49956/0 (socket says 172.21.15.7:49956) 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6da4c640 1 -- 172.21.15.7:0/1656946883 learned_addr learned my addr 172.21.15.7:0/1656946883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6d24b640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f681025c0 0x7f3f68101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6ca4a640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68104ef0 0x7f3f68100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6da4c640 1 -- 172.21.15.7:0/1656946883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f681025c0 msgr2=0x7f3f68101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6da4c640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f681025c0 0x7f3f68101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6da4c640 1 -- 172.21.15.7:0/1656946883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68104ef0 msgr2=0x7f3f68100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6da4c640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68104ef0 0x7f3f68100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6da4c640 1 -- 172.21.15.7:0/1656946883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f68101340 con 0x7f3f68107820 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6ca4a640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68104ef0 0x7f3f68100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.349+0000 7f3f6da4c640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f68107820 0x7f3f68100ab0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000e9d0 tx=0x7f3f6000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.350+0000 7f3f5e7fc640 1 -- 172.21.15.7:0/1656946883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000cd80 con 0x7f3f68107820 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.350+0000 7f3f5e7fc640 1 -- 172.21.15.7:0/1656946883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f60005020 con 0x7f3f68107820 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.350+0000 7f3f6f4af640 1 -- 172.21.15.7:0/1656946883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f681a1f80 con 0x7f3f68107820 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.350+0000 7f3f5e7fc640 1 -- 172.21.15.7:0/1656946883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600116d0 con 0x7f3f68107820 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.350+0000 7f3f6f4af640 1 -- 172.21.15.7:0/1656946883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f681a2390 con 0x7f3f68107820 2024-09-10T10:45:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.351+0000 7f3f6f4af640 1 -- 172.21.15.7:0/1656946883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f680fdcd0 con 0x7f3f68107820 2024-09-10T10:45:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.353+0000 7f3f5e7fc640 1 -- 172.21.15.7:0/1656946883 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f3f600051d0 con 0x7f3f68107820 2024-09-10T10:45:54.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.354+0000 7f3f5e7fc640 1 -- 172.21.15.7:0/1656946883 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f3f600cf0f0 con 0x7f3f68107820 2024-09-10T10:45:54.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.357+0000 7f3f5e7fc640 1 -- 172.21.15.7:0/1656946883 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f60016030 con 0x7f3f68107820 2024-09-10T10:45:54.417 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:54.061+0000 7fd578d56640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:54.417 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:54.061+0000 7fd578d56640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:54.417 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:54.061+0000 7fd578d56640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:54.417 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:54.061+0000 7fd578d56640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:54.417 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:54.061+0000 7fd578d56640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:54.417 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:54.062+0000 7fd57bd17640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:54.418 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:54.062+0000 7fd57bd17640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:54.418 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:54.062+0000 7fd57bd17640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:54.418 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:54.062+0000 7fd57bd17640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:54.418 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:54.062+0000 7fd57bd17640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:45:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.576+0000 7f3f6f4af640 1 -- 172.21.15.7:0/1656946883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f3f6802d7c0 con 0x7f3f68107820 2024-09-10T10:45:54.672 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: Active manager daemon x restarted 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: Activating manager daemon x 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: mgrmap e20: x(active, starting, since 0.0108944s), standbys: z, y 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:45:54.673 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:45:54.674 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: Manager daemon x is now available 2024-09-10T10:45:54.675 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:54.676 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:45:54.676 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:54 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:45:54.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-10T10:45:54.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: Active manager daemon x restarted 2024-09-10T10:45:54.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: Activating manager daemon x 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: mgrmap e20: x(active, starting, since 0.0108944s), standbys: z, y 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:45:54.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:45:54.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: Manager daemon x is now available 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:45:54.687 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:54 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:45:54.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.805+0000 7f3f5e7fc640 1 -- 172.21.15.7:0/1656946883 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f3f60097980 con 0x7f3f68107820 2024-09-10T10:45:54.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.806+0000 7f3f5e7fc640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3f540a99d0 0x7f3f540abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.810+0000 7f3f6d24b640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3f540a99d0 0x7f3f540abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:54.812+0000 7f3f6d24b640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3f540a99d0 0x7f3f540abdc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3f640314e0 tx=0x7f3f64000bc0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:54.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-10T10:45:54.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: Active manager daemon x restarted 2024-09-10T10:45:54.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: Activating manager daemon x 2024-09-10T10:45:54.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T10:45:54.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: mgrmap e20: x(active, starting, since 0.0108944s), standbys: z, y 2024-09-10T10:45:54.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:45:54.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:45:54.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:45:54.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:45:54.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:45:54.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:45:54.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:45:54.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:45:54.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:45:54.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:45:54.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:45:54.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:45:54.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:45:54.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:45:54.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:45:54.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:45:54.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:45:54.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:45:54.980 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:45:54.980 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:45:54.980 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:45:54.980 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:45:54.980 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:45:54.980 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:45:54.980 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:45:54.980 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:45:54.981 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:45:54.981 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:45:54.981 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:45:54.981 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:45:54.981 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:45:54.981 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:45:54.981 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:45:54.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: Manager daemon x is now available 2024-09-10T10:45:54.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:54.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:45:54.982 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:54 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:45:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.481+0000 7f3f5e7fc640 1 -- 172.21.15.7:0/1656946883 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f3f600940b0 con 0x7f3f68107820 2024-09-10T10:45:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 -- 172.21.15.7:0/1656946883 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3f540a99d0 msgr2=0x7f3f540abdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3f540a99d0 0x7f3f540abdc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3f640314e0 tx=0x7f3f64000bc0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 -- 172.21.15.7:0/1656946883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f68107820 msgr2=0x7f3f68100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f68107820 0x7f3f68100ab0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000e9d0 tx=0x7f3f6000ee90 comp rx=0 tx=0).stop 2024-09-10T10:45:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 -- 172.21.15.7:0/1656946883 shutdown_connections 2024-09-10T10:45:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3f540a99d0 0x7f3f540abdc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f68107820 0x7f3f68100ab0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f68104ef0 0x7f3f68100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 --2- 172.21.15.7:0/1656946883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f681025c0 0x7f3f68101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 -- 172.21.15.7:0/1656946883 >> 172.21.15.7:0/1656946883 conn(0x7f3f680fc1e0 msgr2=0x7f3f68102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 -- 172.21.15.7:0/1656946883 shutdown_connections 2024-09-10T10:45:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.483+0000 7f3f6f4af640 1 -- 172.21.15.7:0/1656946883 wait complete. 2024-09-10T10:45:55.495 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T10:45:55.506 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T10:45:55.555 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T10:45:55.555 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T10:45:55.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-10T10:45:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.671+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3421409399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c41078c0 msgr2=0x7ff6c4109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.671+0000 7ff6c9ab7640 1 --2- 172.21.15.7:0/3421409399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c41078c0 0x7ff6c4109cb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009ca0 tx=0x7ff6bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.671+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3421409399 shutdown_connections 2024-09-10T10:45:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.671+0000 7ff6c9ab7640 1 --2- 172.21.15.7:0/3421409399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c41078c0 0x7ff6c4109cb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.671+0000 7ff6c9ab7640 1 --2- 172.21.15.7:0/3421409399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c4104f90 0x7ff6c4107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.671+0000 7ff6c9ab7640 1 --2- 172.21.15.7:0/3421409399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c4102660 0x7ff6c4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.671+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3421409399 >> 172.21.15.7:0/3421409399 conn(0x7ff6c40fc1e0 msgr2=0x7ff6c40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.672+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3421409399 shutdown_connections 2024-09-10T10:45:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.672+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3421409399 wait complete. 2024-09-10T10:45:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.672+0000 7ff6c9ab7640 1 Processor -- start 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.672+0000 7ff6c9ab7640 1 -- start start 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c9ab7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c4102660 0x7ff6c4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c9ab7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c4104f90 0x7ff6c4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c9ab7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c41078c0 0x7ff6c419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c9ab7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410d800 con 0x7ff6c41078c0 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c9ab7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410d680 con 0x7ff6c4102660 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c9ab7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410d980 con 0x7ff6c4104f90 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c4104f90 0x7ff6c4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c4102660 0x7ff6c4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c41078c0 0x7ff6c419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c4102660 0x7ff6c4197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:34040/0 (socket says 172.21.15.7:34040) 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c4104f90 0x7ff6c4198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:34208/0 (socket says 172.21.15.7:34208) 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c37fe640 1 -- 172.21.15.7:0/3371016008 learned_addr learned my addr 172.21.15.7:0/3371016008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c2ffd640 1 -- 172.21.15.7:0/3371016008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c4102660 msgr2=0x7ff6c4197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c2ffd640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c4102660 0x7ff6c4197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c2ffd640 1 -- 172.21.15.7:0/3371016008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c41078c0 msgr2=0x7ff6c419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c2ffd640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c41078c0 0x7ff6c419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c2ffd640 1 -- 172.21.15.7:0/3371016008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c419fa20 con 0x7ff6c4104f90 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c37fe640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c4102660 0x7ff6c4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c3fff640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c41078c0 0x7ff6c419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.673+0000 7ff6c2ffd640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c4104f90 0x7ff6c4198340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800e9c0 tx=0x7ff6b800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.674+0000 7ff6c0ff9640 1 -- 172.21.15.7:0/3371016008 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b800ccc0 con 0x7ff6c4104f90 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.674+0000 7ff6c0ff9640 1 -- 172.21.15.7:0/3371016008 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6b8010910 con 0x7ff6c4104f90 2024-09-10T10:45:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.674+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3371016008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c419fd10 con 0x7ff6c4104f90 2024-09-10T10:45:55.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.674+0000 7ff6c0ff9640 1 -- 172.21.15.7:0/3371016008 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80188c0 con 0x7ff6c4104f90 2024-09-10T10:45:55.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.674+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3371016008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c41a0200 con 0x7ff6c4104f90 2024-09-10T10:45:55.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.675+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3371016008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c40fdcd0 con 0x7ff6c4104f90 2024-09-10T10:45:55.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.676+0000 7ff6c0ff9640 1 -- 172.21.15.7:0/3371016008 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7ff6b8010ac0 con 0x7ff6c4104f90 2024-09-10T10:45:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.677+0000 7ff6c0ff9640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ff6ac0a2a40 0x7ff6ac0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.677+0000 7ff6c0ff9640 1 -- 172.21.15.7:0/3371016008 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7ff6b80cf0b0 con 0x7ff6c4104f90 2024-09-10T10:45:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.677+0000 7ff6c37fe640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ff6ac0a2a40 0x7ff6ac0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.677+0000 7ff6c37fe640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ff6ac0a2a40 0x7ff6ac0a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4011500 tx=0x7ff6b4005f90 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:55.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.681+0000 7ff6c0ff9640 1 -- 172.21.15.7:0/3371016008 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6b809bb50 con 0x7ff6c4104f90 2024-09-10T10:45:55.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:55 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1656946883' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T10:45:55.725 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:55 smithi133 ceph-mon[35282]: mgrmap e21: x(active, since 1.02299s), standbys: z, y 2024-09-10T10:45:55.884 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1656946883' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T10:45:55.884 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-mon[30873]: mgrmap e21: x(active, since 1.02299s), standbys: z, y 2024-09-10T10:45:55.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.884+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3371016008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7ff6c4062e60 con 0x7ff6c4104f90 2024-09-10T10:45:55.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.892+0000 7ff6c0ff9640 1 -- 172.21.15.7:0/3371016008 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v32)=0 v32) ==== 129+0+0 (secure 0 0 0) 0x7ff6b809fe50 con 0x7ff6c4104f90 2024-09-10T10:45:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3371016008 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ff6ac0a2a40 msgr2=0x7ff6ac0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ff6ac0a2a40 0x7ff6ac0a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4011500 tx=0x7ff6b4005f90 comp rx=0 tx=0).stop 2024-09-10T10:45:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3371016008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c4104f90 msgr2=0x7ff6c4198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c4104f90 0x7ff6c4198340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800e9c0 tx=0x7ff6b800ee80 comp rx=0 tx=0).stop 2024-09-10T10:45:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3371016008 shutdown_connections 2024-09-10T10:45:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ff6ac0a2a40 0x7ff6ac0a4f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6c41078c0 0x7ff6c419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c4104f90 0x7ff6c4198340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 --2- 172.21.15.7:0/3371016008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6c4102660 0x7ff6c4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3371016008 >> 172.21.15.7:0/3371016008 conn(0x7ff6c40fc1e0 msgr2=0x7ff6c4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3371016008 shutdown_connections 2024-09-10T10:45:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:55.895+0000 7ff6c9ab7640 1 -- 172.21.15.7:0/3371016008 wait complete. 2024-09-10T10:45:55.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-10T10:45:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:55 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1656946883' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T10:45:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:55 smithi103 ceph-mon[35160]: mgrmap e21: x(active, since 1.02299s), standbys: z, y 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.026+0000 7f1cacaf1640 1 -- 172.21.15.7:0/1766783733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ca8102620 msgr2=0x7f1ca8104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.026+0000 7f1cacaf1640 1 --2- 172.21.15.7:0/1766783733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ca8102620 0x7f1ca8104a10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009900 tx=0x7f1c9802a410 comp rx=0 tx=0).stop 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.026+0000 7f1cacaf1640 1 -- 172.21.15.7:0/1766783733 shutdown_connections 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.026+0000 7f1cacaf1640 1 --2- 172.21.15.7:0/1766783733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8107880 0x7f1ca8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.026+0000 7f1cacaf1640 1 --2- 172.21.15.7:0/1766783733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ca8104f50 0x7f1ca8107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.026+0000 7f1cacaf1640 1 --2- 172.21.15.7:0/1766783733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ca8102620 0x7f1ca8104a10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.026+0000 7f1cacaf1640 1 -- 172.21.15.7:0/1766783733 >> 172.21.15.7:0/1766783733 conn(0x7f1ca80fc1e0 msgr2=0x7f1ca80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.026+0000 7f1cacaf1640 1 -- 172.21.15.7:0/1766783733 shutdown_connections 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.026+0000 7f1cacaf1640 1 -- 172.21.15.7:0/1766783733 wait complete. 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.027+0000 7f1cacaf1640 1 Processor -- start 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.027+0000 7f1cacaf1640 1 -- start start 2024-09-10T10:45:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.027+0000 7f1cacaf1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8102620 0x7f1ca8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.027+0000 7f1cacaf1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ca8104f50 0x7f1ca8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.027+0000 7f1cacaf1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ca8107880 0x7f1ca819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.027+0000 7f1cacaf1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca810d860 con 0x7f1ca8104f50 2024-09-10T10:45:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.027+0000 7f1cacaf1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca810d6e0 con 0x7f1ca8107880 2024-09-10T10:45:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1cacaf1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca810d9e0 con 0x7f1ca8102620 2024-09-10T10:45:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ca8104f50 0x7f1ca8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ca8104f50 0x7f1ca8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50012/0 (socket says 172.21.15.7:50012) 2024-09-10T10:45:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca5d9b640 1 -- 172.21.15.7:0/620143087 learned_addr learned my addr 172.21.15.7:0/620143087 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca659c640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8102620 0x7f1ca8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca6d9d640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ca8107880 0x7f1ca819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca5d9b640 1 -- 172.21.15.7:0/620143087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8102620 msgr2=0x7f1ca8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca5d9b640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8102620 0x7f1ca8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca5d9b640 1 -- 172.21.15.7:0/620143087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ca8107880 msgr2=0x7f1ca819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca5d9b640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ca8107880 0x7f1ca819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca5d9b640 1 -- 172.21.15.7:0/620143087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca819fa60 con 0x7f1ca8104f50 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca6d9d640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ca8107880 0x7f1ca819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1ca5d9b640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ca8104f50 0x7f1ca8198380 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00e9c0 tx=0x7f1c9c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1c977fe640 1 -- 172.21.15.7:0/620143087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00cd80 con 0x7f1ca8104f50 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1c977fe640 1 -- 172.21.15.7:0/620143087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c9c005500 con 0x7f1ca8104f50 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1c977fe640 1 -- 172.21.15.7:0/620143087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c010690 con 0x7f1ca8104f50 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1cacaf1640 1 -- 172.21.15.7:0/620143087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca819fd50 con 0x7f1ca8104f50 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.028+0000 7f1cacaf1640 1 -- 172.21.15.7:0/620143087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca81a01f0 con 0x7f1ca8104f50 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.030+0000 7f1cacaf1640 1 -- 172.21.15.7:0/620143087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca80fdcd0 con 0x7f1ca8104f50 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.031+0000 7f1c977fe640 1 -- 172.21.15.7:0/620143087 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f1c9c0046c0 con 0x7f1ca8104f50 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.031+0000 7f1c977fe640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1c900a2a40 0x7f1c900a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.032+0000 7f1c977fe640 1 -- 172.21.15.7:0/620143087 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f1c9c0cf120 con 0x7f1ca8104f50 2024-09-10T10:45:56.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.032+0000 7f1ca659c640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1c900a2a40 0x7f1c900a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.032+0000 7f1ca659c640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1c900a2a40 0x7f1c900a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c98031440 tx=0x7f1c980315b0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.036+0000 7f1c977fe640 1 -- 172.21.15.7:0/620143087 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c9c09c210 con 0x7f1ca8104f50 2024-09-10T10:45:56.184 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:55.882+0000 7fd564c2f640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:10:45:55] ENGINE Error in HTTPServer.serve 2024-09-10T10:45:56.184 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: Traceback (most recent call last): 2024-09-10T10:45:56.185 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:45:56.185 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self._connections.run(self.expiration_interval) 2024-09-10T10:45:56.185 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:45:56.185 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self._run(expiration_interval) 2024-09-10T10:45:56.185 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:45:56.185 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:45:56.185 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:45:56.185 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:45:56.185 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:45:56.185 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: s = self.context.wrap_socket( 2024-09-10T10:45:56.186 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:45:56.186 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: return self.sslsocket_class._create( 2024-09-10T10:45:56.186 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:45:56.186 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self.do_handshake() 2024-09-10T10:45:56.186 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:45:56.186 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: self._sslobj.do_handshake() 2024-09-10T10:45:56.186 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:45:56.186 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:45:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:45:56.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.262+0000 7f1cacaf1640 1 -- 172.21.15.7:0/620143087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f1ca802d7c0 con 0x7f1ca8104f50 2024-09-10T10:45:56.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.266+0000 7f1c977fe640 1 -- 172.21.15.7:0/620143087 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) ==== 138+0+0 (secure 0 0 0) 0x7f1c9c09bb10 con 0x7f1ca8104f50 2024-09-10T10:45:56.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.270+0000 7f1cacaf1640 1 -- 172.21.15.7:0/620143087 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1c900a2a40 msgr2=0x7f1c900a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.270+0000 7f1cacaf1640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1c900a2a40 0x7f1c900a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c98031440 tx=0x7f1c980315b0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.270+0000 7f1cacaf1640 1 -- 172.21.15.7:0/620143087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ca8104f50 msgr2=0x7f1ca8198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.270+0000 7f1cacaf1640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ca8104f50 0x7f1ca8198380 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c00e9c0 tx=0x7f1c9c00ee80 comp rx=0 tx=0).stop 2024-09-10T10:45:56.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.270+0000 7f1cacaf1640 1 -- 172.21.15.7:0/620143087 shutdown_connections 2024-09-10T10:45:56.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.270+0000 7f1cacaf1640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ca8107880 0x7f1ca819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.270+0000 7f1cacaf1640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1c900a2a40 0x7f1c900a4f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.270+0000 7f1cacaf1640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ca8104f50 0x7f1ca8198380 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.270+0000 7f1cacaf1640 1 --2- 172.21.15.7:0/620143087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca8102620 0x7f1ca8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.270+0000 7f1cacaf1640 1 -- 172.21.15.7:0/620143087 >> 172.21.15.7:0/620143087 conn(0x7f1ca80fc1e0 msgr2=0x7f1ca8102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:56.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.271+0000 7f1cacaf1640 1 -- 172.21.15.7:0/620143087 shutdown_connections 2024-09-10T10:45:56.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.271+0000 7f1cacaf1640 1 -- 172.21.15.7:0/620143087 wait complete. 2024-09-10T10:45:56.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-10T10:45:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.400+0000 7f5d8854b640 1 -- 172.21.15.7:0/375961518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80101c40 msgr2=0x7f5d801020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.400+0000 7f5d8854b640 1 --2- 172.21.15.7:0/375961518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80101c40 0x7f5d801020c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009900 tx=0x7f5d7002a410 comp rx=0 tx=0).stop 2024-09-10T10:45:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.400+0000 7f5d8854b640 1 -- 172.21.15.7:0/375961518 shutdown_connections 2024-09-10T10:45:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.400+0000 7f5d8854b640 1 --2- 172.21.15.7:0/375961518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d80102600 0x7f5d80108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.400+0000 7f5d8854b640 1 --2- 172.21.15.7:0/375961518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80101c40 0x7f5d801020c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.400+0000 7f5d8854b640 1 --2- 172.21.15.7:0/375961518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80100a50 0x7f5d80100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.400+0000 7f5d8854b640 1 -- 172.21.15.7:0/375961518 >> 172.21.15.7:0/375961518 conn(0x7f5d800fc1b0 msgr2=0x7f5d800fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.400+0000 7f5d8854b640 1 -- 172.21.15.7:0/375961518 shutdown_connections 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.400+0000 7f5d8854b640 1 -- 172.21.15.7:0/375961518 wait complete. 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d8854b640 1 Processor -- start 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d8854b640 1 -- start start 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d8854b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80100a50 0x7f5d801049e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d8854b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d80101c40 0x7f5d80103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d8854b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80102600 0x7f5d80103560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d8854b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8010c700 con 0x7f5d80100a50 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d8854b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8010c580 con 0x7f5d80101c40 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d8854b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8010c880 con 0x7f5d80102600 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d862e7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80100a50 0x7f5d801049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d862e7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80100a50 0x7f5d801049e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50018/0 (socket says 172.21.15.7:50018) 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d862e7640 1 -- 172.21.15.7:0/254379320 learned_addr learned my addr 172.21.15.7:0/254379320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d86ae8640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80102600 0x7f5d80103560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d85ae6640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d80101c40 0x7f5d80103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d862e7640 1 -- 172.21.15.7:0/254379320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80102600 msgr2=0x7f5d80103560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d862e7640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80102600 0x7f5d80103560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d862e7640 1 -- 172.21.15.7:0/254379320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d80101c40 msgr2=0x7f5d80103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d862e7640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d80101c40 0x7f5d80103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d862e7640 1 -- 172.21.15.7:0/254379320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d80103df0 con 0x7f5d80100a50 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.401+0000 7f5d86ae8640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80102600 0x7f5d80103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.402+0000 7f5d862e7640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80100a50 0x7f5d801049e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00e9c0 tx=0x7f5d7c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.402+0000 7f5d777fe640 1 -- 172.21.15.7:0/254379320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00cd80 con 0x7f5d80100a50 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.402+0000 7f5d8854b640 1 -- 172.21.15.7:0/254379320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d801a1ee0 con 0x7f5d80100a50 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.402+0000 7f5d777fe640 1 -- 172.21.15.7:0/254379320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c0035f0 con 0x7f5d80100a50 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.402+0000 7f5d777fe640 1 -- 172.21.15.7:0/254379320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c010690 con 0x7f5d80100a50 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.402+0000 7f5d8854b640 1 -- 172.21.15.7:0/254379320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d801a2340 con 0x7f5d80100a50 2024-09-10T10:45:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.403+0000 7f5d8854b640 1 -- 172.21.15.7:0/254379320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d801a2630 con 0x7f5d80100a50 2024-09-10T10:45:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.404+0000 7f5d777fe640 1 -- 172.21.15.7:0/254379320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f5d7c0037a0 con 0x7f5d80100a50 2024-09-10T10:45:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.405+0000 7f5d777fe640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5d6c0a2a40 0x7f5d6c0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.405+0000 7f5d85ae6640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5d6c0a2a40 0x7f5d6c0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.405+0000 7f5d777fe640 1 -- 172.21.15.7:0/254379320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f5d7c0cf0d0 con 0x7f5d80100a50 2024-09-10T10:45:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.406+0000 7f5d85ae6640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5d6c0a2a40 0x7f5d6c0a4f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5d70031440 tx=0x7f5d700315b0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.409+0000 7f5d777fe640 1 -- 172.21.15.7:0/254379320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d7c09c200 con 0x7f5d80100a50 2024-09-10T10:45:56.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.498+0000 7f5d777fe640 1 -- 172.21.15.7:0/254379320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 22) ==== 137952+0+0 (secure 0 0 0) 0x7f5d7c098170 con 0x7f5d80100a50 2024-09-10T10:45:56.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.602+0000 7f5d8854b640 1 -- 172.21.15.7:0/254379320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f5d801a2800 con 0x7f5d80100a50 2024-09-10T10:45:56.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.606+0000 7f5d777fe640 1 -- 172.21.15.7:0/254379320 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7f5d7c064250 con 0x7f5d80100a50 2024-09-10T10:45:56.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 -- 172.21.15.7:0/254379320 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5d6c0a2a40 msgr2=0x7f5d6c0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5d6c0a2a40 0x7f5d6c0a4f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5d70031440 tx=0x7f5d700315b0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 -- 172.21.15.7:0/254379320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80100a50 msgr2=0x7f5d801049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80100a50 0x7f5d801049e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00e9c0 tx=0x7f5d7c00ee80 comp rx=0 tx=0).stop 2024-09-10T10:45:56.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 -- 172.21.15.7:0/254379320 shutdown_connections 2024-09-10T10:45:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80102600 0x7f5d80103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d80101c40 0x7f5d80103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5d6c0a2a40 0x7f5d6c0a4f00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 --2- 172.21.15.7:0/254379320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d80100a50 0x7f5d801049e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 -- 172.21.15.7:0/254379320 >> 172.21.15.7:0/254379320 conn(0x7f5d800fc1b0 msgr2=0x7f5d800fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 -- 172.21.15.7:0/254379320 shutdown_connections 2024-09-10T10:45:56.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.609+0000 7f5d8854b640 1 -- 172.21.15.7:0/254379320 wait complete. 2024-09-10T10:45:56.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:45:56.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1656946883' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T10:45:56.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:45:56.620 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.621 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.621 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.621 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.621 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: from='client.? ' entity='client.admin' 2024-09-10T10:45:56.621 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.621 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.621 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:56 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/620143087' entity='client.admin' 2024-09-10T10:45:56.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-10T10:45:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.737+0000 7f090cd6f640 1 -- 172.21.15.7:0/1655363367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0908102620 msgr2=0x7f0908108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.737+0000 7f090cd6f640 1 --2- 172.21.15.7:0/1655363367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0908102620 0x7f0908108eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0900009900 tx=0x7f090002a410 comp rx=0 tx=0).stop 2024-09-10T10:45:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.737+0000 7f090cd6f640 1 -- 172.21.15.7:0/1655363367 shutdown_connections 2024-09-10T10:45:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.737+0000 7f090cd6f640 1 --2- 172.21.15.7:0/1655363367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0908102620 0x7f0908108eb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.737+0000 7f090cd6f640 1 --2- 172.21.15.7:0/1655363367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0908101c60 0x7f09081020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.737+0000 7f090cd6f640 1 --2- 172.21.15.7:0/1655363367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908100a70 0x7f0908100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.737+0000 7f090cd6f640 1 -- 172.21.15.7:0/1655363367 >> 172.21.15.7:0/1655363367 conn(0x7f09080fc1f0 msgr2=0x7f09080fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.737+0000 7f090cd6f640 1 -- 172.21.15.7:0/1655363367 shutdown_connections 2024-09-10T10:45:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.737+0000 7f090cd6f640 1 -- 172.21.15.7:0/1655363367 wait complete. 2024-09-10T10:45:56.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090cd6f640 1 Processor -- start 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090cd6f640 1 -- start start 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090cd6f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0908100a70 0x7f090806aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090cd6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0908101c60 0x7f090819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090cd6f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908102620 0x7f090819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090cd6f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f090810c720 con 0x7f0908100a70 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090cd6f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f090810c5a0 con 0x7f0908101c60 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090cd6f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f090810c8a0 con 0x7f0908102620 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0908100a70 0x7f090806aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0908100a70 0x7f090806aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50044/0 (socket says 172.21.15.7:50044) 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f0906d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908102620 0x7f090819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f090659c640 1 -- 172.21.15.7:0/3765168881 learned_addr learned my addr 172.21.15.7:0/3765168881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:56.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.738+0000 7f0905d9b640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0908101c60 0x7f090819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f090659c640 1 -- 172.21.15.7:0/3765168881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908102620 msgr2=0x7f090819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f090659c640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908102620 0x7f090819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f090659c640 1 -- 172.21.15.7:0/3765168881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0908101c60 msgr2=0x7f090819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f090659c640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0908101c60 0x7f090819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f090659c640 1 -- 172.21.15.7:0/3765168881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090819fa00 con 0x7f0908100a70 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f0906d9d640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908102620 0x7f090819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f090659c640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0908100a70 0x7f090806aed0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f08f800ebc0 tx=0x7f08f800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f08f77fe640 1 -- 172.21.15.7:0/3765168881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8003b70 con 0x7f0908100a70 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f08f77fe640 1 -- 172.21.15.7:0/3765168881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f8003d10 con 0x7f0908100a70 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f090cd6f640 1 -- 172.21.15.7:0/3765168881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090819fcf0 con 0x7f0908100a70 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f08f77fe640 1 -- 172.21.15.7:0/3765168881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8019980 con 0x7f0908100a70 2024-09-10T10:45:56.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.739+0000 7f090cd6f640 1 -- 172.21.15.7:0/3765168881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09081a0190 con 0x7f0908100a70 2024-09-10T10:45:56.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.741+0000 7f090cd6f640 1 -- 172.21.15.7:0/3765168881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0908107d90 con 0x7f0908100a70 2024-09-10T10:45:56.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.742+0000 7f08f77fe640 1 -- 172.21.15.7:0/3765168881 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 22) ==== 137952+0+0 (secure 0 0 0) 0x7f08f8019b20 con 0x7f0908100a70 2024-09-10T10:45:56.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.742+0000 7f08f77fe640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08f00a2da0 0x7f08f00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.743+0000 7f0905d9b640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08f00a2da0 0x7f08f00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.743+0000 7f08f77fe640 1 -- 172.21.15.7:0/3765168881 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f08f80cfbf0 con 0x7f0908100a70 2024-09-10T10:45:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.743+0000 7f0905d9b640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08f00a2da0 0x7f08f00a5260 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f090806ac40 tx=0x7f08fc00b390 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.747+0000 7f08f77fe640 1 -- 172.21.15.7:0/3765168881 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08f809c810 con 0x7f0908100a70 2024-09-10T10:45:56.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:45:56.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1656946883' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T10:45:56.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:45:56.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: from='client.? ' entity='client.admin' 2024-09-10T10:45:56.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:56 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/620143087' entity='client.admin' 2024-09-10T10:45:56.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:56.968+0000 7f090cd6f640 1 -- 172.21.15.7:0/3765168881 --> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0908062630 con 0x7f08f00a2da0 2024-09-10T10:45:56.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:45:56.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1656946883' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T10:45:56.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:45:56.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: from='client.? ' entity='client.admin' 2024-09-10T10:45:56.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:56.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:56 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/620143087' entity='client.admin' 2024-09-10T10:45:57.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.033+0000 7f08f77fe640 1 -- 172.21.15.7:0/3765168881 <== mgr.15132 v2:172.21.15.7:6800/354580929 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0908062630 con 0x7f08f00a2da0 2024-09-10T10:45:57.034 INFO:teuthology.orchestra.run.smithi007.stdout:Schedule set for path / 2024-09-10T10:45:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 -- 172.21.15.7:0/3765168881 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08f00a2da0 msgr2=0x7f08f00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08f00a2da0 0x7f08f00a5260 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f090806ac40 tx=0x7f08fc00b390 comp rx=0 tx=0).stop 2024-09-10T10:45:57.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 -- 172.21.15.7:0/3765168881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0908100a70 msgr2=0x7f090806aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0908100a70 0x7f090806aed0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f08f800ebc0 tx=0x7f08f800c6a0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 -- 172.21.15.7:0/3765168881 shutdown_connections 2024-09-10T10:45:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908102620 0x7f090819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0908101c60 0x7f090819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08f00a2da0 0x7f08f00a5260 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 --2- 172.21.15.7:0/3765168881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0908100a70 0x7f090806aed0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 -- 172.21.15.7:0/3765168881 >> 172.21.15.7:0/3765168881 conn(0x7f09080fc1f0 msgr2=0x7f0908104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 -- 172.21.15.7:0/3765168881 shutdown_connections 2024-09-10T10:45:57.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.035+0000 7f090cd6f640 1 -- 172.21.15.7:0/3765168881 wait complete. 2024-09-10T10:45:57.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-10T10:45:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.156+0000 7efedb303640 1 -- 172.21.15.7:0/4079063107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efed41025d0 msgr2=0x7efed41049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.156+0000 7efedb303640 1 --2- 172.21.15.7:0/4079063107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efed41025d0 0x7efed41049c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efed0009900 tx=0x7efed002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 -- 172.21.15.7:0/4079063107 shutdown_connections 2024-09-10T10:45:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 --2- 172.21.15.7:0/4079063107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed4107830 0x7efed4109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 --2- 172.21.15.7:0/4079063107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed4104f00 0x7efed41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 --2- 172.21.15.7:0/4079063107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efed41025d0 0x7efed41049c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 -- 172.21.15.7:0/4079063107 >> 172.21.15.7:0/4079063107 conn(0x7efed40fc1b0 msgr2=0x7efed40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 -- 172.21.15.7:0/4079063107 shutdown_connections 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 -- 172.21.15.7:0/4079063107 wait complete. 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 Processor -- start 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 -- start start 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efed41025d0 0x7efed4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed4104f00 0x7efed41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed4107830 0x7efed419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efed410d810 con 0x7efed4104f00 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efed410d690 con 0x7efed41025d0 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.157+0000 7efedb303640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efed410d990 con 0x7efed4107830 2024-09-10T10:45:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed889e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed4104f00 0x7efed41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed909f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efed41025d0 0x7efed4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed889e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed4104f00 0x7efed41983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50062/0 (socket says 172.21.15.7:50062) 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed909f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efed41025d0 0x7efed4197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:34136/0 (socket says 172.21.15.7:34136) 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed889e640 1 -- 172.21.15.7:0/1374676841 learned_addr learned my addr 172.21.15.7:0/1374676841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed98a0640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed4107830 0x7efed419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed889e640 1 -- 172.21.15.7:0/1374676841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed4107830 msgr2=0x7efed419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed889e640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed4107830 0x7efed419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed889e640 1 -- 172.21.15.7:0/1374676841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efed41025d0 msgr2=0x7efed4197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed889e640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efed41025d0 0x7efed4197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed889e640 1 -- 172.21.15.7:0/1374676841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed419fa90 con 0x7efed4104f00 2024-09-10T10:45:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed909f640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efed41025d0 0x7efed4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efed889e640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed4104f00 0x7efed41983b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efec400e9a0 tx=0x7efec400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efeca7fc640 1 -- 172.21.15.7:0/1374676841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec400cd80 con 0x7efed4104f00 2024-09-10T10:45:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efeca7fc640 1 -- 172.21.15.7:0/1374676841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efec40035f0 con 0x7efed4104f00 2024-09-10T10:45:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efedb303640 1 -- 172.21.15.7:0/1374676841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed419fd80 con 0x7efed4104f00 2024-09-10T10:45:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efeca7fc640 1 -- 172.21.15.7:0/1374676841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4010670 con 0x7efed4104f00 2024-09-10T10:45:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.158+0000 7efedb303640 1 -- 172.21.15.7:0/1374676841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed41a0270 con 0x7efed4104f00 2024-09-10T10:45:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.160+0000 7efebffff640 1 -- 172.21.15.7:0/1374676841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed4062df0 con 0x7efed4104f00 2024-09-10T10:45:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.161+0000 7efeca7fc640 1 -- 172.21.15.7:0/1374676841 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 22) ==== 137952+0+0 (secure 0 0 0) 0x7efec40037a0 con 0x7efed4104f00 2024-09-10T10:45:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.162+0000 7efeca7fc640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7efec00a2da0 0x7efec00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.162+0000 7efeca7fc640 1 -- 172.21.15.7:0/1374676841 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7efec40cf680 con 0x7efed4104f00 2024-09-10T10:45:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.162+0000 7efed909f640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7efec00a2da0 0x7efec00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.162+0000 7efed909f640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7efec00a2da0 0x7efec00a5260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efed00314e0 tx=0x7efed0000bc0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.166+0000 7efeca7fc640 1 -- 172.21.15.7:0/1374676841 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efec409c240 con 0x7efed4104f00 2024-09-10T10:45:57.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.371+0000 7efebffff640 1 -- 172.21.15.7:0/1374676841 --> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7efed4063230 con 0x7efec00a2da0 2024-09-10T10:45:57.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.435+0000 7efeca7fc640 1 -- 172.21.15.7:0/1374676841 <== mgr.15132 v2:172.21.15.7:6800/354580929 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7efed4063230 con 0x7efec00a2da0 2024-09-10T10:45:57.436 INFO:teuthology.orchestra.run.smithi007.stdout:Retention added to path / 2024-09-10T10:45:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 -- 172.21.15.7:0/1374676841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7efec00a2da0 msgr2=0x7efec00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7efec00a2da0 0x7efec00a5260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efed00314e0 tx=0x7efed0000bc0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 -- 172.21.15.7:0/1374676841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed4104f00 msgr2=0x7efed41983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed4104f00 0x7efed41983b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efec400e9a0 tx=0x7efec400ee60 comp rx=0 tx=0).stop 2024-09-10T10:45:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 -- 172.21.15.7:0/1374676841 shutdown_connections 2024-09-10T10:45:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed4107830 0x7efed419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7efec00a2da0 0x7efec00a5260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed4104f00 0x7efed41983b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 --2- 172.21.15.7:0/1374676841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efed41025d0 0x7efed4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 -- 172.21.15.7:0/1374676841 >> 172.21.15.7:0/1374676841 conn(0x7efed40fc1b0 msgr2=0x7efed4102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 -- 172.21.15.7:0/1374676841 shutdown_connections 2024-09-10T10:45:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.437+0000 7efebffff640 1 -- 172.21.15.7:0/1374676841 wait complete. 2024-09-10T10:45:57.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-10T10:45:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.557+0000 7f1d739b9640 1 -- 172.21.15.7:0/4116946243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d6c1035c0 msgr2=0x7f1d6c10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.557+0000 7f1d739b9640 1 --2- 172.21.15.7:0/4116946243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d6c1035c0 0x7f1d6c10baa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009900 tx=0x7f1d5c02a410 comp rx=0 tx=0).stop 2024-09-10T10:45:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.558+0000 7f1d739b9640 1 -- 172.21.15.7:0/4116946243 shutdown_connections 2024-09-10T10:45:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.558+0000 7f1d739b9640 1 --2- 172.21.15.7:0/4116946243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c10c0d0 0x7f1d6c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.558+0000 7f1d739b9640 1 --2- 172.21.15.7:0/4116946243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d6c1035c0 0x7f1d6c10baa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.558+0000 7f1d739b9640 1 --2- 172.21.15.7:0/4116946243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d6c102ca0 0x7f1d6c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.558+0000 7f1d739b9640 1 -- 172.21.15.7:0/4116946243 >> 172.21.15.7:0/4116946243 conn(0x7f1d6c0fc6b0 msgr2=0x7f1d6c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.558+0000 7f1d739b9640 1 -- 172.21.15.7:0/4116946243 shutdown_connections 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.558+0000 7f1d739b9640 1 -- 172.21.15.7:0/4116946243 wait complete. 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.558+0000 7f1d739b9640 1 Processor -- start 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.558+0000 7f1d739b9640 1 -- start start 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.558+0000 7f1d739b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d6c102ca0 0x7f1d6c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d739b9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c1035c0 0x7f1d6c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d739b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d6c10c0d0 0x7f1d6c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d739b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c111f20 con 0x7f1d6c102ca0 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d739b9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c111da0 con 0x7f1d6c1035c0 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d739b9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1120a0 con 0x7f1d6c10c0d0 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d71755640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d6c102ca0 0x7f1d6c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d71755640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d6c102ca0 0x7f1d6c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50084/0 (socket says 172.21.15.7:50084) 2024-09-10T10:45:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d71755640 1 -- 172.21.15.7:0/2340731442 learned_addr learned my addr 172.21.15.7:0/2340731442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d70f54640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c1035c0 0x7f1d6c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d71f56640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d6c10c0d0 0x7f1d6c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d71755640 1 -- 172.21.15.7:0/2340731442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d6c10c0d0 msgr2=0x7f1d6c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d71755640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d6c10c0d0 0x7f1d6c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d71755640 1 -- 172.21.15.7:0/2340731442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c1035c0 msgr2=0x7f1d6c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d71755640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c1035c0 0x7f1d6c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d71755640 1 -- 172.21.15.7:0/2340731442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c19d350 con 0x7f1d6c102ca0 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d70f54640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c1035c0 0x7f1d6c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d71755640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d6c102ca0 0x7f1d6c198360 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800e9f0 tx=0x7f1d6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.559+0000 7f1d627fc640 1 -- 172.21.15.7:0/2340731442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800cd80 con 0x7f1d6c102ca0 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.560+0000 7f1d739b9640 1 -- 172.21.15.7:0/2340731442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6c19d640 con 0x7f1d6c102ca0 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.560+0000 7f1d627fc640 1 -- 172.21.15.7:0/2340731442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d68005500 con 0x7f1d6c102ca0 2024-09-10T10:45:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.560+0000 7f1d627fc640 1 -- 172.21.15.7:0/2340731442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68010690 con 0x7f1d6c102ca0 2024-09-10T10:45:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.560+0000 7f1d739b9640 1 -- 172.21.15.7:0/2340731442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6c1a4ed0 con 0x7f1d6c102ca0 2024-09-10T10:45:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.561+0000 7f1d739b9640 1 -- 172.21.15.7:0/2340731442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6c103080 con 0x7f1d6c102ca0 2024-09-10T10:45:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.562+0000 7f1d627fc640 1 -- 172.21.15.7:0/2340731442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 22) ==== 137952+0+0 (secure 0 0 0) 0x7f1d680046c0 con 0x7f1d6c102ca0 2024-09-10T10:45:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.563+0000 7f1d627fc640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1d580a2da0 0x7f1d580a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.563+0000 7f1d627fc640 1 -- 172.21.15.7:0/2340731442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f1d680cf830 con 0x7f1d6c102ca0 2024-09-10T10:45:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.563+0000 7f1d70f54640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1d580a2da0 0x7f1d580a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.564+0000 7f1d70f54640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1d580a2da0 0x7f1d580a5260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c006010 tx=0x7f1d5c031510 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.567+0000 7f1d627fc640 1 -- 172.21.15.7:0/2340731442 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d6809c430 con 0x7f1d6c102ca0 2024-09-10T10:45:57.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.773+0000 7f1d739b9640 1 -- 172.21.15.7:0/2340731442 --> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1d6c062630 con 0x7f1d580a2da0 2024-09-10T10:45:57.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.780+0000 7f1d627fc640 1 -- 172.21.15.7:0/2340731442 <== mgr.15132 v2:172.21.15.7:6800/354580929 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f1d6c062630 con 0x7f1d580a2da0 2024-09-10T10:45:57.782 INFO:teuthology.orchestra.run.smithi007.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-10T00:00:00", "created": "2024-09-10T10:45:56", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-10T10:45:57.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.782+0000 7f1d739b9640 1 -- 172.21.15.7:0/2340731442 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1d580a2da0 msgr2=0x7f1d580a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.782+0000 7f1d739b9640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1d580a2da0 0x7f1d580a5260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c006010 tx=0x7f1d5c031510 comp rx=0 tx=0).stop 2024-09-10T10:45:57.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.782+0000 7f1d739b9640 1 -- 172.21.15.7:0/2340731442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d6c102ca0 msgr2=0x7f1d6c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.782+0000 7f1d739b9640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d6c102ca0 0x7f1d6c198360 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800e9f0 tx=0x7f1d6800edf0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.782+0000 7f1d739b9640 1 -- 172.21.15.7:0/2340731442 shutdown_connections 2024-09-10T10:45:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.782+0000 7f1d739b9640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d6c10c0d0 0x7f1d6c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.782+0000 7f1d739b9640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1d6c1035c0 0x7f1d6c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.782+0000 7f1d739b9640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1d580a2da0 0x7f1d580a5260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.782+0000 7f1d739b9640 1 --2- 172.21.15.7:0/2340731442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d6c102ca0 0x7f1d6c198360 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.782+0000 7f1d739b9640 1 -- 172.21.15.7:0/2340731442 >> 172.21.15.7:0/2340731442 conn(0x7f1d6c0fc6b0 msgr2=0x7f1d6c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:57.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.783+0000 7f1d739b9640 1 -- 172.21.15.7:0/2340731442 shutdown_connections 2024-09-10T10:45:57.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.783+0000 7f1d739b9640 1 -- 172.21.15.7:0/2340731442 wait complete. 2024-09-10T10:45:57.795 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: [10/Sep/2024:10:45:55] ENGINE Bus STARTING 2024-09-10T10:45:57.795 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:45:57.795 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: [10/Sep/2024:10:45:55] ENGINE Serving on https://172.21.15.7:7150 2024-09-10T10:45:57.795 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: [10/Sep/2024:10:45:55] ENGINE Error in HTTPServer.serve 2024-09-10T10:45:57.796 INFO:journalctl@ceph.mon.a.smithi007.stdout: Traceback (most recent call last): 2024-09-10T10:45:57.796 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:45:57.796 INFO:journalctl@ceph.mon.a.smithi007.stdout: self._connections.run(self.expiration_interval) 2024-09-10T10:45:57.796 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:45:57.796 INFO:journalctl@ceph.mon.a.smithi007.stdout: self._run(expiration_interval) 2024-09-10T10:45:57.796 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:45:57.797 INFO:journalctl@ceph.mon.a.smithi007.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:45:57.797 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:45:57.797 INFO:journalctl@ceph.mon.a.smithi007.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:45:57.797 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:45:57.797 INFO:journalctl@ceph.mon.a.smithi007.stdout: s = self.context.wrap_socket( 2024-09-10T10:45:57.797 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:45:57.797 INFO:journalctl@ceph.mon.a.smithi007.stdout: return self.sslsocket_class._create( 2024-09-10T10:45:57.798 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:45:57.798 INFO:journalctl@ceph.mon.a.smithi007.stdout: self.do_handshake() 2024-09-10T10:45:57.798 INFO:journalctl@ceph.mon.a.smithi007.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:45:57.798 INFO:journalctl@ceph.mon.a.smithi007.stdout: self._sslobj.do_handshake() 2024-09-10T10:45:57.798 INFO:journalctl@ceph.mon.a.smithi007.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:45:57.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: [10/Sep/2024:10:45:55] ENGINE Serving on http://172.21.15.7:8765 2024-09-10T10:45:57.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: [10/Sep/2024:10:45:55] ENGINE Bus STARTED 2024-09-10T10:45:57.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-10T10:45:57.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-10T10:45:57.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: Unable to set osd_memory_target on smithi103 to 184593920: error parsing value: Value '184593920' is below minimum 939524096 2024-09-10T10:45:57.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-10T10:45:57.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: Unable to set osd_memory_target on smithi133 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-10T10:45:57.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/254379320' entity='client.admin' 2024-09-10T10:45:57.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:57.802 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:57.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-10T10:45:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.913+0000 7f6306635640 1 -- 172.21.15.7:0/1022662551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63001009f0 msgr2=0x7f6300100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.913+0000 7f6306635640 1 --2- 172.21.15.7:0/1022662551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63001009f0 0x7f6300100e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009930 tx=0x7f62f402a420 comp rx=0 tx=0).stop 2024-09-10T10:45:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.914+0000 7f6306635640 1 -- 172.21.15.7:0/1022662551 shutdown_connections 2024-09-10T10:45:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.914+0000 7f6306635640 1 --2- 172.21.15.7:0/1022662551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6300101390 0x7f630010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.914+0000 7f6306635640 1 --2- 172.21.15.7:0/1022662551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63001009f0 0x7f6300100e50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.914+0000 7f6306635640 1 --2- 172.21.15.7:0/1022662551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300106a00 0x7f6300106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.914+0000 7f6306635640 1 -- 172.21.15.7:0/1022662551 >> 172.21.15.7:0/1022662551 conn(0x7f63000fc6b0 msgr2=0x7f63000fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.914+0000 7f6306635640 1 -- 172.21.15.7:0/1022662551 shutdown_connections 2024-09-10T10:45:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.914+0000 7f6306635640 1 -- 172.21.15.7:0/1022662551 wait complete. 2024-09-10T10:45:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.914+0000 7f6306635640 1 Processor -- start 2024-09-10T10:45:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.914+0000 7f6306635640 1 -- start start 2024-09-10T10:45:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6306635640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63001009f0 0x7f63001983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6306635640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300101390 0x7f6300198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6306635640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300106a00 0x7f630019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6306635640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6300111f30 con 0x7f6300106a00 2024-09-10T10:45:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6306635640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6300111db0 con 0x7f6300101390 2024-09-10T10:45:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6306635640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63001120b0 con 0x7f63001009f0 2024-09-10T10:45:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6304bd2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300106a00 0x7f630019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6304bd2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300106a00 0x7f630019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50106/0 (socket says 172.21.15.7:50106) 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6304bd2640 1 -- 172.21.15.7:0/1694571577 learned_addr learned my addr 172.21.15.7:0/1694571577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f62ff7fe640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300101390 0x7f6300198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f62fffff640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63001009f0 0x7f63001983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6304bd2640 1 -- 172.21.15.7:0/1694571577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63001009f0 msgr2=0x7f63001983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6304bd2640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63001009f0 0x7f63001983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6304bd2640 1 -- 172.21.15.7:0/1694571577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300101390 msgr2=0x7f6300198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6304bd2640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300101390 0x7f6300198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6304bd2640 1 -- 172.21.15.7:0/1694571577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630019d3b0 con 0x7f6300106a00 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6304bd2640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300106a00 0x7f630019cc80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f62f800eb10 tx=0x7f62f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f62fd7fa640 1 -- 172.21.15.7:0/1694571577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8004b20 con 0x7f6300106a00 2024-09-10T10:45:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6306635640 1 -- 172.21.15.7:0/1694571577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630019d6a0 con 0x7f6300106a00 2024-09-10T10:45:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f62fd7fa640 1 -- 172.21.15.7:0/1694571577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f8004cc0 con 0x7f6300106a00 2024-09-10T10:45:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f62fd7fa640 1 -- 172.21.15.7:0/1694571577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8010760 con 0x7f6300106a00 2024-09-10T10:45:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.915+0000 7f6306635640 1 -- 172.21.15.7:0/1694571577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63001a4f30 con 0x7f6300106a00 2024-09-10T10:45:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.917+0000 7f6306635640 1 -- 172.21.15.7:0/1694571577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6300100e50 con 0x7f6300106a00 2024-09-10T10:45:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.918+0000 7f62fd7fa640 1 -- 172.21.15.7:0/1694571577 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 22) ==== 137952+0+0 (secure 0 0 0) 0x7f62f8005020 con 0x7f6300106a00 2024-09-10T10:45:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.918+0000 7f62fd7fa640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62e80a2da0 0x7f62e80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.919+0000 7f62fffff640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62e80a2da0 0x7f62e80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.919+0000 7f62fd7fa640 1 -- 172.21.15.7:0/1694571577 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f62f80cfab0 con 0x7f6300106a00 2024-09-10T10:45:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.919+0000 7f62fffff640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62e80a2da0 0x7f62e80a5260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009b50 tx=0x7f62f0005f90 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: [10/Sep/2024:10:45:55] ENGINE Bus STARTING 2024-09-10T10:45:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:45:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: [10/Sep/2024:10:45:55] ENGINE Serving on https://172.21.15.7:7150 2024-09-10T10:45:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: [10/Sep/2024:10:45:55] ENGINE Error in HTTPServer.serve 2024-09-10T10:45:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout: Traceback (most recent call last): 2024-09-10T10:45:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: self._run(expiration_interval) 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: s = self.context.wrap_socket( 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: return self.sslsocket_class._create( 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:45:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout: self.do_handshake() 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: self._sslobj.do_handshake() 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: [10/Sep/2024:10:45:55] ENGINE Serving on http://172.21.15.7:8765 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: [10/Sep/2024:10:45:55] ENGINE Bus STARTED 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: Unable to set osd_memory_target on smithi103 to 184593920: error parsing value: Value '184593920' is below minimum 939524096 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: Unable to set osd_memory_target on smithi133 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-10T10:45:57.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/254379320' entity='client.admin' 2024-09-10T10:45:57.926 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.926 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.926 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.926 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.926 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.926 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.926 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:57.926 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:57.923+0000 7f62fd7fa640 1 -- 172.21.15.7:0/1694571577 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62f809c6d0 con 0x7f6300106a00 2024-09-10T10:45:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: [10/Sep/2024:10:45:55] ENGINE Bus STARTING 2024-09-10T10:45:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: [10/Sep/2024:10:45:55] ENGINE Serving on https://172.21.15.7:7150 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: [10/Sep/2024:10:45:55] ENGINE Error in HTTPServer.serve 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout: Traceback (most recent call last): 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout: self._connections.run(self.expiration_interval) 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout: self._run(expiration_interval) 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:45:57.976 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout: s = self.context.wrap_socket( 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout: return self.sslsocket_class._create( 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout: self.do_handshake() 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout: self._sslobj.do_handshake() 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: [10/Sep/2024:10:45:55] ENGINE Serving on http://172.21.15.7:8765 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: [10/Sep/2024:10:45:55] ENGINE Bus STARTED 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-10T10:45:57.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: Unable to set osd_memory_target on smithi103 to 184593920: error parsing value: Value '184593920' is below minimum 939524096 2024-09-10T10:45:57.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-10T10:45:57.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: Unable to set osd_memory_target on smithi133 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-10T10:45:57.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/254379320' entity='client.admin' 2024-09-10T10:45:57.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:57.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:45:57.979 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:57.980 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:58.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.134+0000 7f6306635640 1 -- 172.21.15.7:0/1694571577 --> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f6300062630 con 0x7f62e80a2da0 2024-09-10T10:45:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.140+0000 7f62fd7fa640 1 -- 172.21.15.7:0/1694571577 <== mgr.15132 v2:172.21.15.7:6800/354580929 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f6300062630 con 0x7f62e80a2da0 2024-09-10T10:45:58.141 INFO:teuthology.orchestra.run.smithi007.stdout:/ 1m 6m3h 2024-09-10T10:45:58.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 -- 172.21.15.7:0/1694571577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62e80a2da0 msgr2=0x7f62e80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62e80a2da0 0x7f62e80a5260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009b50 tx=0x7f62f0005f90 comp rx=0 tx=0).stop 2024-09-10T10:45:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 -- 172.21.15.7:0/1694571577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300106a00 msgr2=0x7f630019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300106a00 0x7f630019cc80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f62f800eb10 tx=0x7f62f800c2d0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 -- 172.21.15.7:0/1694571577 shutdown_connections 2024-09-10T10:45:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62e80a2da0 0x7f62e80a5260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6300106a00 0x7f630019cc80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6300101390 0x7f6300198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 --2- 172.21.15.7:0/1694571577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63001009f0 0x7f63001983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 -- 172.21.15.7:0/1694571577 >> 172.21.15.7:0/1694571577 conn(0x7f63000fc6b0 msgr2=0x7f63000fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:58.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 -- 172.21.15.7:0/1694571577 shutdown_connections 2024-09-10T10:45:58.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T10:45:58.142+0000 7f6306635640 1 -- 172.21.15.7:0/1694571577 wait complete. 2024-09-10T10:45:58.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-10T10:45:58.181 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-10T10:45:58.193 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-10T10:45:58.232 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T10:45:58.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-10T10:45:58.234 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T10:45:58.286 INFO:tasks.workunit:Pulling workunits from ref 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:45:58.287 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T10:45:58.287 INFO:tasks.workunit:timeout=3h 2024-09-10T10:45:58.287 INFO:tasks.workunit:cleanup=True 2024-09-10T10:45:58.287 DEBUG:teuthology.orchestra.run.smithi007:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-10T10:45:58.304 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-10T10:45:58.304 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:45:58.304 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-10T10:45:58.304 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-10 10:44:29.375025489 +0000 2024-09-10T10:45:58.304 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-10 10:44:29.375025489 +0000 2024-09-10T10:45:58.304 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-10 10:45:53.816413151 +0000 2024-09-10T10:45:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-10 10:44:29.375025489 +0000 2024-09-10T10:45:58.304 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:58.304 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T10:45:58.377 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:45:58.434 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T10:45:58.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi007 to 177.0M 2024-09-10T10:45:58.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Unable to set osd_memory_target on smithi007 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:58.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi007 to 177.0M 2024-09-10T10:45:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Unable to set osd_memory_target on smithi007 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-10T10:45:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-10T10:45:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T10:45:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T10:45:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:45:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:45:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:58.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:58.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:58.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi007 to 177.0M 2024-09-10T10:45:58.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Unable to set osd_memory_target on smithi007 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-10T10:45:58.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi007:/etc/ceph/ceph.conf 2024-09-10T10:45:58.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-10T10:45:58.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi133:/etc/ceph/ceph.conf 2024-09-10T10:45:58.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:58.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:45:58.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:45:58.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:45:58.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:45:58.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:45:58.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:45:58.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: Updating smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:45:58.977 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:45:58.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:45:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:59 smithi103 ceph-mon[35160]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:59 smithi103 ceph-mon[35160]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:45:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:59 smithi103 ceph-mon[35160]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:45:59 smithi103 ceph-mon[35160]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-10T10:45:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:59 smithi007 ceph-mon[30873]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:59 smithi007 ceph-mon[30873]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:45:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:59 smithi007 ceph-mon[30873]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:45:59 smithi007 ceph-mon[30873]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-10T10:45:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:59 smithi133 ceph-mon[35282]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:59 smithi133 ceph-mon[35282]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:45:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:59 smithi133 ceph-mon[35282]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:45:59.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:45:59 smithi133 ceph-mon[35282]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-10T10:46:00.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:46:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:46:00.446+0000 7fd555290640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:46:00.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:46:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:46:00.446+0000 7fd555290640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:46:00.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:46:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:46:00.446+0000 7fd555290640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:46:00.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:46:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:46:00.446+0000 7fd555290640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:46:00.934 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 10:46:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T10:46:00.446+0000 7fd555290640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:46:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:01 smithi103 ceph-mon[35160]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T10:46:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:01 smithi007 ceph-mon[30873]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T10:46:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:01 smithi133 ceph-mon[35282]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T10:46:03.777 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:03 smithi133 ceph-mon[35282]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-09-10T10:46:03.778 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:03 smithi133 ceph-mon[35282]: mgrmap e24: x(active, since 8s), standbys: z, y 2024-09-10T10:46:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:03 smithi103 ceph-mon[35160]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-09-10T10:46:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:03 smithi103 ceph-mon[35160]: mgrmap e24: x(active, since 8s), standbys: z, y 2024-09-10T10:46:03.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:03 smithi007 ceph-mon[30873]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-09-10T10:46:03.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:03 smithi007 ceph-mon[30873]: mgrmap e24: x(active, since 8s), standbys: z, y 2024-09-10T10:46:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:05 smithi103 ceph-mon[35160]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-09-10T10:46:05.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:05 smithi007 ceph-mon[30873]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-09-10T10:46:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:05 smithi133 ceph-mon[35282]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-09-10T10:46:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:07 smithi103 ceph-mon[35160]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-09-10T10:46:07.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:07 smithi007 ceph-mon[30873]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-09-10T10:46:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:07 smithi133 ceph-mon[35282]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-09-10T10:46:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:09 smithi103 ceph-mon[35160]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 62 op/s 2024-09-10T10:46:09.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:09 smithi007 ceph-mon[30873]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 62 op/s 2024-09-10T10:46:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:09 smithi133 ceph-mon[35282]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 62 op/s 2024-09-10T10:46:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.375+0000 7f4439077640 1 -- 172.21.15.7:0/524335795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4434102c90 msgr2=0x7f4434103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.375+0000 7f4439077640 1 --2- 172.21.15.7:0/524335795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4434102c90 0x7f4434103070 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4424009900 tx=0x7f442402a480 comp rx=0 tx=0).stop 2024-09-10T10:46:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.375+0000 7f4439077640 1 -- 172.21.15.7:0/524335795 shutdown_connections 2024-09-10T10:46:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.375+0000 7f4439077640 1 --2- 172.21.15.7:0/524335795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f443410c0b0 0x7f443410e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.375+0000 7f4439077640 1 --2- 172.21.15.7:0/524335795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44341035b0 0x7f443410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.375+0000 7f4439077640 1 --2- 172.21.15.7:0/524335795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4434102c90 0x7f4434103070 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.375+0000 7f4439077640 1 -- 172.21.15.7:0/524335795 >> 172.21.15.7:0/524335795 conn(0x7f44340fc6e0 msgr2=0x7f44340feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.376+0000 7f4439077640 1 -- 172.21.15.7:0/524335795 shutdown_connections 2024-09-10T10:46:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.376+0000 7f4439077640 1 -- 172.21.15.7:0/524335795 wait complete. 2024-09-10T10:46:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.376+0000 7f4439077640 1 Processor -- start 2024-09-10T10:46:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4439077640 1 -- start start 2024-09-10T10:46:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4439077640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4434102c90 0x7f44341982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4439077640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44341035b0 0x7f44341987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4439077640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f443410c0b0 0x7f443419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4439077640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4434111e90 con 0x7f4434102c90 2024-09-10T10:46:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4439077640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4434111d10 con 0x7f44341035b0 2024-09-10T10:46:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4439077640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4434112010 con 0x7f443410c0b0 2024-09-10T10:46:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4432d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4434102c90 0x7f44341982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4432d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4434102c90 0x7f44341982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56416/0 (socket says 172.21.15.7:56416) 2024-09-10T10:46:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4432d9d640 1 -- 172.21.15.7:0/3807158466 learned_addr learned my addr 172.21.15.7:0/3807158466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f443359e640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f443410c0b0 0x7f443419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f443259c640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44341035b0 0x7f44341987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4432d9d640 1 -- 172.21.15.7:0/3807158466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f443410c0b0 msgr2=0x7f443419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4432d9d640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f443410c0b0 0x7f443419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4432d9d640 1 -- 172.21.15.7:0/3807158466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44341035b0 msgr2=0x7f44341987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4432d9d640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44341035b0 0x7f44341987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f4432d9d640 1 -- 172.21.15.7:0/3807158466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443419d290 con 0x7f4434102c90 2024-09-10T10:46:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.377+0000 7f443259c640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44341035b0 0x7f44341987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.378+0000 7f4432d9d640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4434102c90 0x7f44341982a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4424006010 tx=0x7f44240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.378+0000 7f441bfff640 1 -- 172.21.15.7:0/3807158466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442403f030 con 0x7f4434102c90 2024-09-10T10:46:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.378+0000 7f441bfff640 1 -- 172.21.15.7:0/3807158466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4424003ce0 con 0x7f4434102c90 2024-09-10T10:46:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.378+0000 7f4439077640 1 -- 172.21.15.7:0/3807158466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443419d520 con 0x7f4434102c90 2024-09-10T10:46:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.378+0000 7f441bfff640 1 -- 172.21.15.7:0/3807158466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424035070 con 0x7f4434102c90 2024-09-10T10:46:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.378+0000 7f4439077640 1 -- 172.21.15.7:0/3807158466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44341a4e10 con 0x7f4434102c90 2024-09-10T10:46:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.380+0000 7f441bfff640 1 -- 172.21.15.7:0/3807158466 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) ==== 138065+0+0 (secure 0 0 0) 0x7f4424042030 con 0x7f4434102c90 2024-09-10T10:46:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443be0b740 1 Processor -- start 2024-09-10T10:46:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443be0b740 1 -- start start 2024-09-10T10:46:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443be0b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d955af0 0x55f64d955ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443be0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d960770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443be0b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d9630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443be0b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f64d8ba2f0 con 0x55f64d960cb0 2024-09-10T10:46:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443be0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f64d862270 con 0x55f64d956410 2024-09-10T10:46:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443be0b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f64d8b62b0 con 0x55f64d955af0 2024-09-10T10:46:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d9630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f441bfff640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f441c0a2ec0 0x7f441c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d9630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56428/0 (socket says 172.21.15.7:56428) 2024-09-10T10:46:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443259c640 1 -- 172.21.15.7:0/2397252441 learned_addr learned my addr 172.21.15.7:0/2397252441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f4432d9d640 1 --2- 172.21.15.7:0/2397252441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d960770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443359e640 1 --2- 172.21.15.7:0/2397252441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d955af0 0x55f64d955ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443359e640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f441c0a2ec0 0x7f441c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443259c640 1 -- 172.21.15.7:0/2397252441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d955af0 msgr2=0x55f64d955ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443259c640 1 --2- 172.21.15.7:0/2397252441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d955af0 0x55f64d955ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443259c640 1 -- 172.21.15.7:0/2397252441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 msgr2=0x55f64d960770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.381+0000 7f443259c640 1 --2- 172.21.15.7:0/2397252441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d960770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.382+0000 7f443259c640 1 -- 172.21.15.7:0/2397252441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64d861340 con 0x55f64d960cb0 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.382+0000 7f441bfff640 1 -- 172.21.15.7:0/3807158466 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f44240efcb0 con 0x7f4434102c90 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.382+0000 7f443259c640 1 --2- 172.21.15.7:0/2397252441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d9630a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4428008c10 tx=0x7f442800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bc94480a68b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.382+0000 7f443359e640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f441c0a2ec0 0x7f441c0a5380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f442c009fd0 tx=0x7f442c009600 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.382+0000 7f44197fa640 1 -- 172.21.15.7:0/2397252441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428014070 con 0x55f64d960cb0 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.382+0000 7f44197fa640 1 -- 172.21.15.7:0/2397252441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44280047a0 con 0x55f64d960cb0 2024-09-10T10:46:11.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.382+0000 7f44197fa640 1 -- 172.21.15.7:0/2397252441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428010690 con 0x55f64d960cb0 2024-09-10T10:46:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 -- 172.21.15.7:0/2397252441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 msgr2=0x55f64d9630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 --2- 172.21.15.7:0/2397252441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d9630a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4428008c10 tx=0x7f442800e720 comp rx=0 tx=0).stop 2024-09-10T10:46:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 -- 172.21.15.7:0/2397252441 shutdown_connections 2024-09-10T10:46:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 --2- 172.21.15.7:0/2397252441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d9630a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 --2- 172.21.15.7:0/2397252441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d960770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 --2- 172.21.15.7:0/2397252441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d955af0 0x55f64d955ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 -- 172.21.15.7:0/2397252441 >> 172.21.15.7:0/2397252441 conn(0x55f64d952060 msgr2=0x55f64d952440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 -- 172.21.15.7:0/2397252441 shutdown_connections 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 -- 172.21.15.7:0/2397252441 wait complete. 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 -- start start 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 Processor -- start 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d95e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d960cb0 0x55f64d95e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f64d963880 con 0x55f64d955af0 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.383+0000 7f443be0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f64d843290 con 0x55f64d956410 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f443be0b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f64d963a00 con 0x55f64d960cb0 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f443be0b740 10 client.0 _ll_register_callbacks cb 0x55f64d874140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4432d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4432d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56438/0 (socket says 172.21.15.7:56438) 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4432d9d640 1 -- 172.21.15.7:0/3860786724 learned_addr learned my addr 172.21.15.7:0/3860786724 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f443259c640 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d95e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4432d9d640 1 -- 172.21.15.7:0/3860786724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d960cb0 msgr2=0x55f64d95e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4432d9d640 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d960cb0 0x55f64d95e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4432d9d640 1 -- 172.21.15.7:0/3860786724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 msgr2=0x55f64d95e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4432d9d640 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d95e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4432d9d640 1 -- 172.21.15.7:0/3860786724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64d5cd210 con 0x55f64d955af0 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f443259c640 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d95e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4432d9d640 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95db70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f44240bbff0 tx=0x7f44240bbd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57a81dca5e12d2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4416ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4416ff5640 1 -- 172.21.15.7:0/3860786724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424049040 con 0x55f64d955af0 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4416ff5640 1 -- 172.21.15.7:0/3860786724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f44240b8720 con 0x55f64d955af0 2024-09-10T10:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f443be0b740 1 -- 172.21.15.7:0/3860786724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44280047a0 con 0x55f64d955af0 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f4416ff5640 1 -- 172.21.15.7:0/3860786724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44240bc7b0 con 0x55f64d955af0 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f443be0b740 10 client.15246 fetch_fsmap learned FSMap version 25 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.384+0000 7f443be0b740 1 -- 172.21.15.7:0/3860786724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f64d699620 con 0x55f64d955af0 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.385+0000 7f4416ff5640 1 -- 172.21.15.7:0/3860786724 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44280047a0 con 0x55f64d955af0 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.385+0000 7f4416ff5640 1 -- 172.21.15.7:0/3860786724 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44240d8020 con 0x55f64d955af0 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.385+0000 7f443be0b740 10 client.15246 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.385+0000 7f443be0b740 10 client.15246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.385+0000 7f443be0b740 20 client.15246 populate_metadata read hostname 'smithi007' 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.385+0000 7f443be0b740 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f64d970be0 0x55f64d973070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.385+0000 7f443be0b740 4 client.15246 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.385+0000 7f443be0b740 1 -- 172.21.15.7:0/3860786724 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f64d8c7a20 con 0x55f64d970be0 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.385+0000 7f443259c640 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f64d970be0 0x55f64d973070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.386+0000 7f443259c640 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f64d970be0 0x55f64d973070 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79602429c84db6e3 server_cookie=4c580e7da0dfbdaf in_seq=0 out_seq=0 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.386+0000 7f4416ff5640 10 client.15246 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.386+0000 7f4416ff5640 1 -- 172.21.15.7:0/3860786724 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f64d8c7a20 con 0x55f64d970be0 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.386+0000 7f4416ff5640 10 client.15246 handle_command_reply: tid=0 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.386+0000 7f443be0b740 1 client.15246 shutdown 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.386+0000 7f443be0b740 1 -- 172.21.15.7:0/3860786724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 msgr2=0x55f64d95db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.386+0000 7f443be0b740 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95db70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f44240bbff0 tx=0x7f44240bbd00 comp rx=0 tx=0).stop 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.386+0000 7f443be0b740 1 -- 172.21.15.7:0/3860786724 shutdown_connections 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.386+0000 7f443be0b740 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f64d970be0 0x55f64d973070 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.387+0000 7f443be0b740 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d960cb0 0x55f64d95e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.387+0000 7f443be0b740 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d95e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.387+0000 7f443be0b740 1 --2- 172.21.15.7:0/3860786724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95db70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.387+0000 7f4416ff5640 0 client.15246 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.387+0000 7f443be0b740 1 -- 172.21.15.7:0/3860786724 >> 172.21.15.7:0/3860786724 conn(0x55f64d952060 msgr2=0x55f64d957d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.387+0000 7f443be0b740 1 -- 172.21.15.7:0/3860786724 shutdown_connections 2024-09-10T10:46:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.387+0000 7f443be0b740 1 -- 172.21.15.7:0/3860786724 wait complete. 2024-09-10T10:46:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.387+0000 7f443be0b740 20 client.15246 trim_cache size 0 max 16384 2024-09-10T10:46:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.401+0000 7f443be0b740 1 Processor -- start 2024-09-10T10:46:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443be0b740 1 -- start start 2024-09-10T10:46:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443be0b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d955af0 0x55f64d95e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443be0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d978920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443be0b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d978e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443be0b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44341affa0 con 0x55f64d960cb0 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443be0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44341ad870 con 0x55f64d956410 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443be0b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44341ab6c0 con 0x55f64d955af0 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d978e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d978e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56446/0 (socket says 172.21.15.7:56446) 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443359e640 1 -- 172.21.15.7:0/2679009176 learned_addr learned my addr 172.21.15.7:0/2679009176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443259c640 1 --2- 172.21.15.7:0/2679009176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d978920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f4432d9d640 1 --2- 172.21.15.7:0/2679009176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d955af0 0x55f64d95e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443359e640 1 -- 172.21.15.7:0/2679009176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d955af0 msgr2=0x55f64d95e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443359e640 1 --2- 172.21.15.7:0/2679009176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d955af0 0x55f64d95e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443359e640 1 -- 172.21.15.7:0/2679009176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 msgr2=0x55f64d978920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443359e640 1 --2- 172.21.15.7:0/2679009176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d978920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443359e640 1 -- 172.21.15.7:0/2679009176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64d951e90 con 0x55f64d960cb0 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.402+0000 7f443359e640 1 --2- 172.21.15.7:0/2679009176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d978e60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f442c009470 tx=0x7f442c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae8453fa2973589 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f4418ff9640 1 -- 172.21.15.7:0/2679009176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c02d070 con 0x55f64d960cb0 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f4418ff9640 1 -- 172.21.15.7:0/2679009176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f442c01eda0 con 0x55f64d960cb0 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f4418ff9640 1 -- 172.21.15.7:0/2679009176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c028a50 con 0x55f64d960cb0 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f443be0b740 1 -- 172.21.15.7:0/2679009176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 msgr2=0x55f64d978e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f443be0b740 1 --2- 172.21.15.7:0/2679009176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d978e60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f442c009470 tx=0x7f442c004ab0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f443be0b740 1 -- 172.21.15.7:0/2679009176 shutdown_connections 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f443be0b740 1 --2- 172.21.15.7:0/2679009176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d960cb0 0x55f64d978e60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f443be0b740 1 --2- 172.21.15.7:0/2679009176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d978920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f443be0b740 1 --2- 172.21.15.7:0/2679009176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d955af0 0x55f64d95e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f443be0b740 1 -- 172.21.15.7:0/2679009176 >> 172.21.15.7:0/2679009176 conn(0x55f64d952060 msgr2=0x55f64d95fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f443be0b740 1 -- 172.21.15.7:0/2679009176 shutdown_connections 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f443be0b740 1 -- 172.21.15.7:0/2679009176 wait complete. 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.403+0000 7f443be0b740 1 -- start start 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f443be0b740 1 Processor -- start 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f443be0b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f443be0b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d95b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f443be0b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d960cb0 0x55f64d95b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f443be0b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44341abd30 con 0x55f64d955af0 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f443be0b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44341ad1a0 con 0x55f64d956410 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f443be0b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44341ac400 con 0x55f64d960cb0 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f443be0b740 10 client.0 _ll_register_callbacks cb 0x7f4424005750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f4432d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f4432d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56450/0 (socket says 172.21.15.7:56450) 2024-09-10T10:46:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f4432d9d640 1 -- 172.21.15.7:0/516750104 learned_addr learned my addr 172.21.15.7:0/516750104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f443259c640 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d95b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f443359e640 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d960cb0 0x55f64d95b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f4432d9d640 1 -- 172.21.15.7:0/516750104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d960cb0 msgr2=0x55f64d95b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f4432d9d640 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d960cb0 0x55f64d95b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f4432d9d640 1 -- 172.21.15.7:0/516750104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 msgr2=0x55f64d95b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f4432d9d640 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d95b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f4432d9d640 1 -- 172.21.15.7:0/516750104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64d9da8f0 con 0x55f64d955af0 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f4432d9d640 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95ab40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4424004620 tx=0x7f44240ba5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935bde479b4f39dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.404+0000 7f4416ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f4416ff5640 1 -- 172.21.15.7:0/516750104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424049040 con 0x55f64d955af0 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f4416ff5640 1 -- 172.21.15.7:0/516750104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f44240bc310 con 0x55f64d955af0 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f443be0b740 1 -- 172.21.15.7:0/516750104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f442c01eda0 con 0x55f64d955af0 2024-09-10T10:46:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f4416ff5640 1 -- 172.21.15.7:0/516750104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44240c4d20 con 0x55f64d955af0 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f443be0b740 10 client.15252 fetch_fsmap learned FSMap version 25 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f443be0b740 1 -- 172.21.15.7:0/516750104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f64d9dbb90 con 0x55f64d955af0 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f4416ff5640 1 -- 172.21.15.7:0/516750104 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f442c01eda0 con 0x55f64d955af0 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f4416ff5640 1 -- 172.21.15.7:0/516750104 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44240d8070 con 0x55f64d955af0 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f443be0b740 10 client.15252 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f443be0b740 10 client.15252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f443be0b740 20 client.15252 populate_metadata read hostname 'smithi007' 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f443be0b740 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f64d9dc120 0x55f64d9de510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f443be0b740 4 client.15252 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.405+0000 7f443be0b740 1 -- 172.21.15.7:0/516750104 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f64d8c1d70 con 0x55f64d9dc120 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.406+0000 7f443259c640 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f64d9dc120 0x55f64d9de510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.406+0000 7f443259c640 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f64d9dc120 0x55f64d9de510 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67e54c305e7a576 server_cookie=c7360be9bdbb4ad3 in_seq=0 out_seq=0 2024-09-10T10:46:11.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.406+0000 7f4416ff5640 10 client.15252 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f4416ff5640 1 -- 172.21.15.7:0/516750104 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f64d8c1d70 con 0x55f64d9dc120 2024-09-10T10:46:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f4416ff5640 10 client.15252 handle_command_reply: tid=0 2024-09-10T10:46:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 client.15252 shutdown 2024-09-10T10:46:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 -- 172.21.15.7:0/516750104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 msgr2=0x55f64d95ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95ab40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4424004620 tx=0x7f44240ba5c0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 -- 172.21.15.7:0/516750104 shutdown_connections 2024-09-10T10:46:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f64d9dc120 0x55f64d9de510 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f64d960cb0 0x55f64d95b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f4416ff5640 0 client.15252 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64d956410 0x55f64d95b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 --2- 172.21.15.7:0/516750104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f64d955af0 0x55f64d95ab40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 -- 172.21.15.7:0/516750104 >> 172.21.15.7:0/516750104 conn(0x55f64d952060 msgr2=0x55f64d957550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 -- 172.21.15.7:0/516750104 shutdown_connections 2024-09-10T10:46:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 1 -- 172.21.15.7:0/516750104 wait complete. 2024-09-10T10:46:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.417+0000 7f443be0b740 20 client.15252 trim_cache size 0 max 16384 2024-09-10T10:46:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:46:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:46:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:46:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 -- 172.21.15.7:0/3807158466 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f441c0a2ec0 msgr2=0x7f441c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f441c0a2ec0 0x7f441c0a5380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f442c009fd0 tx=0x7f442c009600 comp rx=0 tx=0).stop 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 -- 172.21.15.7:0/3807158466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4434102c90 msgr2=0x7f44341982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4434102c90 0x7f44341982a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4424006010 tx=0x7f44240042f0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 -- 172.21.15.7:0/3807158466 shutdown_connections 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f443410c0b0 0x7f443419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44341035b0 0x7f44341987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f441c0a2ec0 0x7f441c0a5380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 --2- 172.21.15.7:0/3807158466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4434102c90 0x7f44341982a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 -- 172.21.15.7:0/3807158466 >> 172.21.15.7:0/3807158466 conn(0x7f44340fc6e0 msgr2=0x7f44340fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 -- 172.21.15.7:0/3807158466 shutdown_connections 2024-09-10T10:46:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:11.418+0000 7f4439077640 1 -- 172.21.15.7:0/3807158466 wait complete. 2024-09-10T10:46:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:11 smithi103 ceph-mon[35160]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-09-10T10:46:11.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:11 smithi007 ceph-mon[30873]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-09-10T10:46:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:11 smithi133 ceph-mon[35282]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-09-10T10:46:13.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:13 smithi103 ceph-mon[35160]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 64 op/s 2024-09-10T10:46:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:13 smithi007 ceph-mon[30873]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 64 op/s 2024-09-10T10:46:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:13 smithi133 ceph-mon[35282]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 100 KiB/s wr, 64 op/s 2024-09-10T10:46:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:15 smithi103 ceph-mon[35160]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-10T10:46:15.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:15 smithi007 ceph-mon[30873]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-10T10:46:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:15 smithi133 ceph-mon[35282]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-10T10:46:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:17 smithi103 ceph-mon[35160]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-09-10T10:46:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:17 smithi007 ceph-mon[30873]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-09-10T10:46:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:17 smithi133 ceph-mon[35282]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-09-10T10:46:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:19 smithi103 ceph-mon[35160]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-10T10:46:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:19 smithi007 ceph-mon[30873]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-10T10:46:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:19 smithi133 ceph-mon[35282]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-10T10:46:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:21 smithi103 ceph-mon[35160]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-10T10:46:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:21 smithi007 ceph-mon[30873]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-10T10:46:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:21 smithi133 ceph-mon[35282]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-10T10:46:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:23 smithi103 ceph-mon[35160]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-10T10:46:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:23 smithi007 ceph-mon[30873]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-10T10:46:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:23 smithi133 ceph-mon[35282]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-10T10:46:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.516+0000 7fd0c7752640 1 -- 172.21.15.7:0/4040273027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c01069c0 msgr2=0x7fd0c0106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.516+0000 7fd0c7752640 1 --2- 172.21.15.7:0/4040273027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c01069c0 0x7fd0c0106da0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009930 tx=0x7fd0bc02a420 comp rx=0 tx=0).stop 2024-09-10T10:46:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.516+0000 7fd0c7752640 1 -- 172.21.15.7:0/4040273027 shutdown_connections 2024-09-10T10:46:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.516+0000 7fd0c7752640 1 --2- 172.21.15.7:0/4040273027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0c0101350 0x7fd0c010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.516+0000 7fd0c7752640 1 --2- 172.21.15.7:0/4040273027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0c01009b0 0x7fd0c0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.516+0000 7fd0c7752640 1 --2- 172.21.15.7:0/4040273027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c01069c0 0x7fd0c0106da0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.516+0000 7fd0c7752640 1 -- 172.21.15.7:0/4040273027 >> 172.21.15.7:0/4040273027 conn(0x7fd0c0077990 msgr2=0x7fd0c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.516+0000 7fd0c7752640 1 -- 172.21.15.7:0/4040273027 shutdown_connections 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.516+0000 7fd0c7752640 1 -- 172.21.15.7:0/4040273027 wait complete. 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c7752640 1 Processor -- start 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c7752640 1 -- start start 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c7752640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c01009b0 0x7fd0c00fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c7752640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0c0101350 0x7fd0c00fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c7752640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0c01069c0 0x7fd0c00fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c7752640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0110dc0 con 0x7fd0c01009b0 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c7752640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0110c40 con 0x7fd0c0101350 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c7752640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0110f40 con 0x7fd0c01069c0 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c54ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c01009b0 0x7fd0c00fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c54ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c01009b0 0x7fd0c00fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59356/0 (socket says 172.21.15.7:59356) 2024-09-10T10:46:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c54ee640 1 -- 172.21.15.7:0/547551035 learned_addr learned my addr 172.21.15.7:0/547551035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.517+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0c0101350 0x7fd0c00fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0c5cef640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0c01069c0 0x7fd0c00fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0c54ee640 1 -- 172.21.15.7:0/547551035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0c01069c0 msgr2=0x7fd0c00fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0c01069c0 0x7fd0c00fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0c54ee640 1 -- 172.21.15.7:0/547551035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0c0101350 msgr2=0x7fd0c00fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0c0101350 0x7fd0c00fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0c54ee640 1 -- 172.21.15.7:0/547551035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c01a6850 con 0x7fd0c01009b0 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0c5cef640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0c01069c0 0x7fd0c00fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c01009b0 0x7fd0c00fcde0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009230 tx=0x7fd0bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0b67fc640 1 -- 172.21.15.7:0/547551035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc03f030 con 0x7fd0c01009b0 2024-09-10T10:46:24.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0b67fc640 1 -- 172.21.15.7:0/547551035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0bc003ad0 con 0x7fd0c01009b0 2024-09-10T10:46:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0c7752640 1 -- 172.21.15.7:0/547551035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c01a6ae0 con 0x7fd0c01009b0 2024-09-10T10:46:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0b67fc640 1 -- 172.21.15.7:0/547551035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc035070 con 0x7fd0c01009b0 2024-09-10T10:46:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.518+0000 7fd0c7752640 1 -- 172.21.15.7:0/547551035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c01a7020 con 0x7fd0c01009b0 2024-09-10T10:46:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0b67fc640 1 -- 172.21.15.7:0/547551035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) ==== 138065+0+0 (secure 0 0 0) 0x7fd0bc042030 con 0x7fd0c01009b0 2024-09-10T10:46:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0ca5cd740 1 Processor -- start 2024-09-10T10:46:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0ca5cd740 1 -- start start 2024-09-10T10:46:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40de410 0x558fd40e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40e8cb0 0x558fd40eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558fd40422f0 con 0x558fd40e8cb0 2024-09-10T10:46:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fd3fea270 con 0x558fd40de410 2024-09-10T10:46:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558fd403e2b0 con 0x558fd40ddaf0 2024-09-10T10:46:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0c4ced640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40e8cb0 0x558fd40eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0c4ced640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40e8cb0 0x558fd40eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59362/0 (socket says 172.21.15.7:59362) 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0c4ced640 1 -- 172.21.15.7:0/3375471212 learned_addr learned my addr 172.21.15.7:0/3375471212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0c5cef640 1 --2- 172.21.15.7:0/3375471212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.521+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/3375471212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40de410 0x558fd40e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0b67fc640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd0ac0a2f40 0x7fd0ac0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0c4ced640 1 -- 172.21.15.7:0/3375471212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 msgr2=0x558fd40dded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/3375471212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40dded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0c4ced640 1 -- 172.21.15.7:0/3375471212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40de410 msgr2=0x558fd40e8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/3375471212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40de410 0x558fd40e8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0c4ced640 1 -- 172.21.15.7:0/3375471212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fd3fe9340 con 0x558fd40e8cb0 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0c5cef640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd0ac0a2f40 0x7fd0ac0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0b67fc640 1 -- 172.21.15.7:0/547551035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fd0bc0efbd0 con 0x7fd0c01009b0 2024-09-10T10:46:24.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/3375471212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40e8cb0 0x558fd40eb0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000bcb0 tx=0x7fd0b00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4bba293a292440 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/3375471212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0004030 con 0x558fd40e8cb0 2024-09-10T10:46:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/3375471212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b00041d0 con 0x558fd40e8cb0 2024-09-10T10:46:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.522+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/3375471212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00117b0 con 0x558fd40e8cb0 2024-09-10T10:46:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.523+0000 7fd0c5cef640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd0ac0a2f40 0x7fd0ac0a5400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8005fd0 tx=0x7fd0b8004a40 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.523+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/3375471212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40e8cb0 msgr2=0x558fd40eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.523+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/3375471212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40e8cb0 0x558fd40eb0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000bcb0 tx=0x7fd0b00107a0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.523+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/3375471212 shutdown_connections 2024-09-10T10:46:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.523+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/3375471212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40e8cb0 0x558fd40eb0a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.523+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/3375471212 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40de410 0x558fd40e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.523+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/3375471212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.523+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/3375471212 >> 172.21.15.7:0/3375471212 conn(0x558fd40da060 msgr2=0x558fd40da440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.523+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/3375471212 shutdown_connections 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.523+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/3375471212 wait complete. 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.524+0000 7fd0ca5cd740 1 -- start start 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.524+0000 7fd0ca5cd740 1 Processor -- start 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.524+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.524+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.524+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd40e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.524+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558fd40eb880 con 0x558fd40de410 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.524+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fd3fcb290 con 0x558fd40e8cb0 2024-09-10T10:46:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.524+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558fd40eba00 con 0x558fd40ddaf0 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0ca5cd740 10 client.0 _ll_register_callbacks cb 0x558fd3ffc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c4ced640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c4ced640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40e60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59374/0 (socket says 172.21.15.7:59374) 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c4ced640 1 -- 172.21.15.7:0/2111502290 learned_addr learned my addr 172.21.15.7:0/2111502290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c5cef640 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd40e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c4ced640 1 -- 172.21.15.7:0/2111502290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 msgr2=0x558fd40e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c4ced640 1 -- 172.21.15.7:0/2111502290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 msgr2=0x558fd40e6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd40e6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c4ced640 1 -- 172.21.15.7:0/2111502290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fd3d55210 con 0x558fd40de410 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40e60c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0011010 tx=0x7fd0b000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1065066c6fa637d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/2111502290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b002a060 con 0x558fd40de410 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/2111502290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0b00041d0 con 0x558fd40de410 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.525+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/2111502290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fd0b00035f0 con 0x558fd40de410 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/2111502290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0b00037f0 con 0x558fd40de410 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0ca5cd740 10 client.15264 fetch_fsmap learned FSMap version 25 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/2111502290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558fd3e21620 con 0x558fd40de410 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/2111502290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b00041d0 con 0x558fd40de410 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/2111502290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd0b00047e0 con 0x558fd40de410 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0ca5cd740 10 client.15264 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:46:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0ca5cd740 10 client.15264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0ca5cd740 20 client.15264 populate_metadata read hostname 'smithi007' 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558fd40f8c50 0x558fd40fb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0ca5cd740 4 client.15264 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/2111502290 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fd404fa20 con 0x558fd40f8c50 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.526+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558fd40f8c50 0x558fd40fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.527+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558fd40f8c50 0x558fd40fb040 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1511df903cc7c2c server_cookie=3ad840642b76ca36 in_seq=0 out_seq=0 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.527+0000 7fd0a8ff9640 10 client.15264 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/2111502290 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fd404fa20 con 0x558fd40f8c50 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0a8ff9640 10 client.15264 handle_command_reply: tid=0 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 client.15264 shutdown 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/2111502290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 msgr2=0x558fd40e60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40e60c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0011010 tx=0x7fd0b000bfd0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/2111502290 shutdown_connections 2024-09-10T10:46:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558fd40f8c50 0x558fd40fb040 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd40e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40e60c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0a8ff9640 0 client.15264 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/2111502290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/2111502290 >> 172.21.15.7:0/2111502290 conn(0x558fd40da060 msgr2=0x558fd40dfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/2111502290 shutdown_connections 2024-09-10T10:46:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/2111502290 wait complete. 2024-09-10T10:46:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.528+0000 7fd0ca5cd740 20 client.15264 trim_cache size 0 max 16384 2024-09-10T10:46:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.544+0000 7fd0ca5cd740 1 Processor -- start 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.544+0000 7fd0ca5cd740 1 -- start start 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.544+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40ddaf0 0x558fd40e0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.544+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40de410 0x558fd40e6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.544+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd4100670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.544+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01b21c0 con 0x558fd40ddaf0 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.544+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01afa90 con 0x558fd40e8cb0 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.544+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01ad8e0 con 0x558fd40de410 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c54ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40ddaf0 0x558fd40e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c54ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40ddaf0 0x558fd40e0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59380/0 (socket says 172.21.15.7:59380) 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c54ee640 1 -- 172.21.15.7:0/3425123173 learned_addr learned my addr 172.21.15.7:0/3425123173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c5cef640 1 --2- 172.21.15.7:0/3425123173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd4100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/3425123173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40de410 0x558fd40e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c54ee640 1 -- 172.21.15.7:0/3425123173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40de410 msgr2=0x558fd40e6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/3425123173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40de410 0x558fd40e6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c54ee640 1 -- 172.21.15.7:0/3425123173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 msgr2=0x558fd4100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/3425123173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd4100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c54ee640 1 -- 172.21.15.7:0/3425123173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fd40d9e90 con 0x558fd40ddaf0 2024-09-10T10:46:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/3425123173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40ddaf0 0x558fd40e0560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0bbf00 tx=0x7fd0bc043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed8bb9a92aa635a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0aaffd640 1 -- 172.21.15.7:0/3425123173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc049040 con 0x558fd40ddaf0 2024-09-10T10:46:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0aaffd640 1 -- 172.21.15.7:0/3425123173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0bc0bc670 con 0x558fd40ddaf0 2024-09-10T10:46:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.545+0000 7fd0aaffd640 1 -- 172.21.15.7:0/3425123173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc03bd60 con 0x558fd40ddaf0 2024-09-10T10:46:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/3425123173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40ddaf0 msgr2=0x558fd40e0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/3425123173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40ddaf0 0x558fd40e0560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0bbf00 tx=0x7fd0bc043000 comp rx=0 tx=0).stop 2024-09-10T10:46:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/3425123173 shutdown_connections 2024-09-10T10:46:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/3425123173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd4100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/3425123173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40de410 0x558fd40e6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/3425123173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40ddaf0 0x558fd40e0560 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/3425123173 >> 172.21.15.7:0/3425123173 conn(0x558fd40da060 msgr2=0x558fd40e7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/3425123173 shutdown_connections 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/3425123173 wait complete. 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 -- start start 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.546+0000 7fd0ca5cd740 1 Processor -- start 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40f1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0ca5cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd40eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01adf50 con 0x558fd40de410 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01af3c0 con 0x558fd40e8cb0 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0ca5cd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01ae620 con 0x558fd40ddaf0 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0ca5cd740 10 client.0 _ll_register_callbacks cb 0x7fd0b80098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c4ced640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c4ced640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40ec530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59396/0 (socket says 172.21.15.7:59396) 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c4ced640 1 -- 172.21.15.7:0/340821975 learned_addr learned my addr 172.21.15.7:0/340821975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c5cef640 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd40eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c4ced640 1 -- 172.21.15.7:0/340821975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 msgr2=0x558fd40f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40f1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c4ced640 1 -- 172.21.15.7:0/340821975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 msgr2=0x558fd40eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd40eca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c4ced640 1 -- 172.21.15.7:0/340821975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fd40ed0b0 con 0x558fd40de410 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.547+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0c4ced640 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40ec530 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x558fd40e53f0 tx=0x7fd0b0004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64640e49b83d6cf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/340821975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b003b060 con 0x558fd40de410 2024-09-10T10:46:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/340821975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fd0b00035f0 con 0x558fd40de410 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/340821975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0bc0bc670 con 0x558fd40de410 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/340821975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0b0003dc0 con 0x558fd40de410 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0ca5cd740 10 client.15270 fetch_fsmap learned FSMap version 25 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/340821975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558fd416bc90 con 0x558fd40de410 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/340821975 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0bc0bc670 con 0x558fd40de410 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/340821975 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd0b0011ce0 con 0x558fd40de410 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0ca5cd740 10 client.15270 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0ca5cd740 10 client.15270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0ca5cd740 20 client.15270 populate_metadata read hostname 'smithi007' 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558fd416c350 0x558fd416e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0ca5cd740 4 client.15270 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:46:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.548+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/340821975 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558fd4049d70 con 0x558fd416c350 2024-09-10T10:46:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.549+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558fd416c350 0x558fd416e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.549+0000 7fd0c54ee640 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558fd416c350 0x558fd416e770 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34129fc495247503 server_cookie=7197e8f512207295 in_seq=0 out_seq=0 2024-09-10T10:46:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.549+0000 7fd0a8ff9640 10 client.15270 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0a8ff9640 1 -- 172.21.15.7:0/340821975 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558fd4049d70 con 0x558fd416c350 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0a8ff9640 10 client.15270 handle_command_reply: tid=0 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 client.15270 shutdown 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/340821975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 msgr2=0x558fd40ec530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40ec530 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x558fd40e53f0 tx=0x7fd0b0004840 comp rx=0 tx=0).stop 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/340821975 shutdown_connections 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558fd416c350 0x558fd416e770 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fd40e8cb0 0x558fd40eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd40de410 0x558fd40ec530 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 --2- 172.21.15.7:0/340821975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558fd40ddaf0 0x558fd40f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/340821975 >> 172.21.15.7:0/340821975 conn(0x558fd40da060 msgr2=0x558fd40d89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0a8ff9640 0 client.15270 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/340821975 shutdown_connections 2024-09-10T10:46:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 1 -- 172.21.15.7:0/340821975 wait complete. 2024-09-10T10:46:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.553+0000 7fd0ca5cd740 20 client.15270 trim_cache size 0 max 16384 2024-09-10T10:46:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:46:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:46:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:46:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:46:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 -- 172.21.15.7:0/547551035 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd0ac0a2f40 msgr2=0x7fd0ac0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd0ac0a2f40 0x7fd0ac0a5400 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8005fd0 tx=0x7fd0b8004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 -- 172.21.15.7:0/547551035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c01009b0 msgr2=0x7fd0c00fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c01009b0 0x7fd0c00fcde0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009230 tx=0x7fd0bc004270 comp rx=0 tx=0).stop 2024-09-10T10:46:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 -- 172.21.15.7:0/547551035 shutdown_connections 2024-09-10T10:46:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0c01069c0 0x7fd0c00fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0c0101350 0x7fd0c00fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd0ac0a2f40 0x7fd0ac0a5400 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 --2- 172.21.15.7:0/547551035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c01009b0 0x7fd0c00fcde0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 -- 172.21.15.7:0/547551035 >> 172.21.15.7:0/547551035 conn(0x7fd0c0077990 msgr2=0x7fd0c0069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 -- 172.21.15.7:0/547551035 shutdown_connections 2024-09-10T10:46:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:24.554+0000 7fd0c7752640 1 -- 172.21.15.7:0/547551035 wait complete. 2024-09-10T10:46:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:25 smithi103 ceph-mon[35160]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-10T10:46:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:25 smithi007 ceph-mon[30873]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-10T10:46:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:25 smithi133 ceph-mon[35282]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-10T10:46:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:27 smithi103 ceph-mon[35160]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-10T10:46:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:27 smithi007 ceph-mon[30873]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-10T10:46:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:27 smithi133 ceph-mon[35282]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-10T10:46:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:29 smithi103 ceph-mon[35160]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:46:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:29 smithi007 ceph-mon[30873]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:46:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:29 smithi133 ceph-mon[35282]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:46:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:31 smithi103 ceph-mon[35160]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:46:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:31 smithi007 ceph-mon[30873]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:46:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:31 smithi133 ceph-mon[35282]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:46:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:33 smithi103 ceph-mon[35160]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T10:46:33.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:33 smithi007 ceph-mon[30873]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T10:46:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:33 smithi133 ceph-mon[35282]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T10:46:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:35 smithi103 ceph-mon[35160]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:46:35.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:35 smithi007 ceph-mon[30873]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:46:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:35 smithi133 ceph-mon[35282]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:46:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.689+0000 7f637a1d6640 1 -- 172.21.15.7:0/3599964040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63741009b0 msgr2=0x7f6374100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.689+0000 7f637a1d6640 1 --2- 172.21.15.7:0/3599964040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63741009b0 0x7f6374100e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6368009900 tx=0x7f636802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.689+0000 7f637a1d6640 1 -- 172.21.15.7:0/3599964040 shutdown_connections 2024-09-10T10:46:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.689+0000 7f637a1d6640 1 --2- 172.21.15.7:0/3599964040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6374101350 0x7f637410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.689+0000 7f637a1d6640 1 --2- 172.21.15.7:0/3599964040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63741009b0 0x7f6374100e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.689+0000 7f637a1d6640 1 --2- 172.21.15.7:0/3599964040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63741069c0 0x7f6374106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.689+0000 7f637a1d6640 1 -- 172.21.15.7:0/3599964040 >> 172.21.15.7:0/3599964040 conn(0x7f6374077990 msgr2=0x7f63740fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.689+0000 7f637a1d6640 1 -- 172.21.15.7:0/3599964040 shutdown_connections 2024-09-10T10:46:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.689+0000 7f637a1d6640 1 -- 172.21.15.7:0/3599964040 wait complete. 2024-09-10T10:46:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.690+0000 7f637a1d6640 1 Processor -- start 2024-09-10T10:46:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f637a1d6640 1 -- start start 2024-09-10T10:46:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f637a1d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63741009b0 0x7f6374199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f637a1d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6374101350 0x7f63741996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f637a1d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63741069c0 0x7f63741a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f637a1d6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6374110e00 con 0x7f63741069c0 2024-09-10T10:46:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f637a1d6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6374110c80 con 0x7f63741009b0 2024-09-10T10:46:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f637a1d6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6374110f80 con 0x7f6374101350 2024-09-10T10:46:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f63737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63741009b0 0x7f6374199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6374101350 0x7f63741996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6374101350 0x7f63741996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:57546/0 (socket says 172.21.15.7:57546) 2024-09-10T10:46:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f63737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63741009b0 0x7f6374199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:41070/0 (socket says 172.21.15.7:41070) 2024-09-10T10:46:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f6373fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63741069c0 0x7f63741a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.691+0000 7f6372ffd640 1 -- 172.21.15.7:0/2245186058 learned_addr learned my addr 172.21.15.7:0/2245186058 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.692+0000 7f6372ffd640 1 -- 172.21.15.7:0/2245186058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63741009b0 msgr2=0x7f6374199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.692+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63741009b0 0x7f6374199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.692+0000 7f6372ffd640 1 -- 172.21.15.7:0/2245186058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63741069c0 msgr2=0x7f63741a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.692+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63741069c0 0x7f63741a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.692+0000 7f6372ffd640 1 -- 172.21.15.7:0/2245186058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63741a4750 con 0x7f6374101350 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.692+0000 7f6373fff640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63741069c0 0x7f63741a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.692+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6374101350 0x7f63741996d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6368006010 tx=0x7f6368003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.692+0000 7f6370ff9640 1 -- 172.21.15.7:0/2245186058 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636803f070 con 0x7f6374101350 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.692+0000 7f6370ff9640 1 -- 172.21.15.7:0/2245186058 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6368031c30 con 0x7f6374101350 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.693+0000 7f637a1d6640 1 -- 172.21.15.7:0/2245186058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63741a4920 con 0x7f6374101350 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.693+0000 7f6370ff9640 1 -- 172.21.15.7:0/2245186058 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368035030 con 0x7f6374101350 2024-09-10T10:46:37.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.693+0000 7f637a1d6640 1 -- 172.21.15.7:0/2245186058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741a4d80 con 0x7f6374101350 2024-09-10T10:46:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.696+0000 7f6370ff9640 1 -- 172.21.15.7:0/2245186058 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 138065+0+0 (secure 0 0 0) 0x7f6368042070 con 0x7f6374101350 2024-09-10T10:46:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.696+0000 7f637d04f740 1 Processor -- start 2024-09-10T10:46:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.696+0000 7f637d04f740 1 -- start start 2024-09-10T10:46:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.696+0000 7f637d04f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac2dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.696+0000 7f637d04f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 0x55bbaac38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.696+0000 7f637d04f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 0x55bbaac3b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.696+0000 7f637d04f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bbaab922f0 con 0x55bbaac2dae0 2024-09-10T10:46:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.696+0000 7f637d04f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bbaab3a270 con 0x55bbaac38ca0 2024-09-10T10:46:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.696+0000 7f637d04f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bbaab8e2b0 con 0x55bbaac2e400 2024-09-10T10:46:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 0x55bbaac3b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 0x55bbaac3b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:41074/0 (socket says 172.21.15.7:41074) 2024-09-10T10:46:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6372ffd640 1 -- 172.21.15.7:0/2392724782 learned_addr learned my addr 172.21.15.7:0/2392724782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6373fff640 1 --2- 172.21.15.7:0/2392724782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac2dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f63737fe640 1 --2- 172.21.15.7:0/2392724782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 0x55bbaac38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6372ffd640 1 -- 172.21.15.7:0/2392724782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 msgr2=0x55bbaac38760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2392724782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 0x55bbaac38760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6372ffd640 1 -- 172.21.15.7:0/2392724782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 msgr2=0x55bbaac2dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6370ff9640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f635c0a2ec0 0x7f635c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2392724782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac2dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6372ffd640 1 -- 172.21.15.7:0/2392724782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbaab39340 con 0x55bbaac38ca0 2024-09-10T10:46:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f6373fff640 1 --2- 172.21.15.7:0/2392724782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac2dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.697+0000 7f63737fe640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f635c0a2ec0 0x7f635c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.698+0000 7f6370ff9640 1 -- 172.21.15.7:0/2245186058 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f63680efbf0 con 0x7f6374101350 2024-09-10T10:46:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.698+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2392724782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 0x55bbaac3b090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f63680ed2d0 tx=0x7f63680b8060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d769e6e9f5c9eb17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.698+0000 7f63737fe640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f635c0a2ec0 0x7f635c0a5380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x55bbaac2aeb0 tx=0x7f6364005f90 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.698+0000 7f6359ffb640 1 -- 172.21.15.7:0/2392724782 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368049070 con 0x55bbaac38ca0 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.698+0000 7f6359ffb640 1 -- 172.21.15.7:0/2392724782 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636803b460 con 0x55bbaac38ca0 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.698+0000 7f6359ffb640 1 -- 172.21.15.7:0/2392724782 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63680bbd60 con 0x55bbaac38ca0 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.699+0000 7f637d04f740 1 -- 172.21.15.7:0/2392724782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 msgr2=0x55bbaac3b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.699+0000 7f637d04f740 1 --2- 172.21.15.7:0/2392724782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 0x55bbaac3b090 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f63680ed2d0 tx=0x7f63680b8060 comp rx=0 tx=0).stop 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.699+0000 7f637d04f740 1 -- 172.21.15.7:0/2392724782 shutdown_connections 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.699+0000 7f637d04f740 1 --2- 172.21.15.7:0/2392724782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 0x55bbaac3b090 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.699+0000 7f637d04f740 1 --2- 172.21.15.7:0/2392724782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 0x55bbaac38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.699+0000 7f637d04f740 1 --2- 172.21.15.7:0/2392724782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac2dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.699+0000 7f637d04f740 1 -- 172.21.15.7:0/2392724782 >> 172.21.15.7:0/2392724782 conn(0x55bbaac2a050 msgr2=0x55bbaac2a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.699+0000 7f637d04f740 1 -- 172.21.15.7:0/2392724782 shutdown_connections 2024-09-10T10:46:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.699+0000 7f637d04f740 1 -- 172.21.15.7:0/2392724782 wait complete. 2024-09-10T10:46:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.700+0000 7f637d04f740 1 -- start start 2024-09-10T10:46:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.700+0000 7f637d04f740 1 Processor -- start 2024-09-10T10:46:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.700+0000 7f637d04f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac35c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.700+0000 7f637d04f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 0x55bbaac361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.700+0000 7f63737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac35c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.700+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 0x55bbaac361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f63737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac35c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37854/0 (socket says 172.21.15.7:37854) 2024-09-10T10:46:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f6372ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 0x55bbaac361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:57572/0 (socket says 172.21.15.7:57572) 2024-09-10T10:46:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.700+0000 7f637d04f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 0x55bbaac366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f63737fe640 1 -- 172.21.15.7:0/2715287307 learned_addr learned my addr 172.21.15.7:0/2715287307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bbaab3a270 con 0x55bbaac2dae0 2024-09-10T10:46:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bbaab1b290 con 0x55bbaac38ca0 2024-09-10T10:46:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bbaac3f9e0 con 0x55bbaac2e400 2024-09-10T10:46:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f63737fe640 1 -- 172.21.15.7:0/2715287307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 msgr2=0x55bbaac361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f63737fe640 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 0x55bbaac361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f63737fe640 1 -- 172.21.15.7:0/2715287307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 msgr2=0x55bbaac366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f63737fe640 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 0x55bbaac366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f63737fe640 1 -- 172.21.15.7:0/2715287307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbaa8a5210 con 0x55bbaac2dae0 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f637d04f740 10 client.0 _ll_register_callbacks cb 0x55bbaab4c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bbaab9fa20 con 0x55bbaac2dae0 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f63737fe640 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac35c60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f63640045e0 tx=0x7f6364004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1876058d772cc24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.701+0000 7f6352ffd640 10 client.15282 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f6352ffd640 1 -- 172.21.15.7:0/2715287307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636401e680 con 0x55bbaac2dae0 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f6352ffd640 1 -- 172.21.15.7:0/2715287307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f636401e820 con 0x55bbaac2dae0 2024-09-10T10:46:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f6352ffd640 1 -- 172.21.15.7:0/2715287307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f636401ea60 con 0x55bbaac2dae0 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f637d04f740 10 client.15282 fetch_fsmap learned FSMap version 25 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55bbaa971620 con 0x55bbaac2dae0 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f6352ffd640 1 -- 172.21.15.7:0/2715287307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6364027070 con 0x55bbaac2dae0 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f637d04f740 10 client.15282 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f637d04f740 10 client.15282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f637d04f740 20 client.15282 populate_metadata read hostname 'smithi007' 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f637d04f740 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bbaac4cbb0 0x55bbaac4f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f637d04f740 4 client.15282 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.702+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbaab99d70 con 0x55bbaac4cbb0 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.703+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bbaac4cbb0 0x55bbaac4f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.703+0000 7f6352ffd640 10 client.15282 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.703+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bbaac4cbb0 0x55bbaac4f040 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=978ade125dca0fd6 server_cookie=890c33155bacdb30 in_seq=0 out_seq=0 2024-09-10T10:46:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f6352ffd640 1 -- 172.21.15.7:0/2715287307 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbaab99d70 con 0x55bbaac4cbb0 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f6352ffd640 10 client.15282 handle_command_reply: tid=0 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f637d04f740 1 client.15282 shutdown 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 msgr2=0x55bbaac35c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f637d04f740 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac35c60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f63640045e0 tx=0x7f6364004730 comp rx=0 tx=0).stop 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 shutdown_connections 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f637d04f740 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bbaac4cbb0 0x55bbaac4f040 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f637d04f740 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac38ca0 0x55bbaac366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f6352ffd640 0 client.15282 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f637d04f740 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac2e400 0x55bbaac361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f637d04f740 1 --2- 172.21.15.7:0/2715287307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2dae0 0x55bbaac35c60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.704+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 >> 172.21.15.7:0/2715287307 conn(0x55bbaac2a050 msgr2=0x55bbaac2fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.705+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 shutdown_connections 2024-09-10T10:46:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.705+0000 7f637d04f740 1 -- 172.21.15.7:0/2715287307 wait complete. 2024-09-10T10:46:37.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.705+0000 7f637d04f740 20 client.15282 trim_cache size 0 max 16384 2024-09-10T10:46:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.733+0000 7f637d04f740 1 Processor -- start 2024-09-10T10:46:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f637d04f740 1 -- start start 2024-09-10T10:46:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f637d04f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac36330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f637d04f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac38ca0 0x55bbaac54920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f637d04f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 0x55bbaac54e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f637d04f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ade10 con 0x55bbaac2e400 2024-09-10T10:46:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f637d04f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ab6e0 con 0x55bbaac4cbb0 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f637d04f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63741a9550 con 0x55bbaac38ca0 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f63737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac36330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f63737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac36330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37870/0 (socket says 172.21.15.7:37870) 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f63737fe640 1 -- 172.21.15.7:0/1984676839 learned_addr learned my addr 172.21.15.7:0/1984676839 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f6372ffd640 1 --2- 172.21.15.7:0/1984676839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac38ca0 0x55bbaac54920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f6373fff640 1 --2- 172.21.15.7:0/1984676839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 0x55bbaac54e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f63737fe640 1 -- 172.21.15.7:0/1984676839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac38ca0 msgr2=0x55bbaac54920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f63737fe640 1 --2- 172.21.15.7:0/1984676839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac38ca0 0x55bbaac54920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f63737fe640 1 -- 172.21.15.7:0/1984676839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 msgr2=0x55bbaac54e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f63737fe640 1 --2- 172.21.15.7:0/1984676839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 0x55bbaac54e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.734+0000 7f63737fe640 1 -- 172.21.15.7:0/1984676839 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbaac2ace0 con 0x55bbaac2e400 2024-09-10T10:46:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.735+0000 7f63737fe640 1 --2- 172.21.15.7:0/1984676839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac36330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6364012d50 tx=0x7f6364023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d08cc902539d52d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.735+0000 7f63597fa640 1 -- 172.21.15.7:0/1984676839 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636401e880 con 0x55bbaac2e400 2024-09-10T10:46:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.735+0000 7f63597fa640 1 -- 172.21.15.7:0/1984676839 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bbaab9fa20 con 0x55bbaac2e400 2024-09-10T10:46:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.735+0000 7f63597fa640 1 -- 172.21.15.7:0/1984676839 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364004dc0 con 0x55bbaac2e400 2024-09-10T10:46:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.736+0000 7f637d04f740 1 -- 172.21.15.7:0/1984676839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 msgr2=0x55bbaac36330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.736+0000 7f637d04f740 1 --2- 172.21.15.7:0/1984676839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac36330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6364012d50 tx=0x7f6364023000 comp rx=0 tx=0).stop 2024-09-10T10:46:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.736+0000 7f637d04f740 1 -- 172.21.15.7:0/1984676839 shutdown_connections 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.736+0000 7f637d04f740 1 --2- 172.21.15.7:0/1984676839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 0x55bbaac54e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.736+0000 7f637d04f740 1 --2- 172.21.15.7:0/1984676839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac38ca0 0x55bbaac54920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.736+0000 7f637d04f740 1 --2- 172.21.15.7:0/1984676839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac36330 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.736+0000 7f637d04f740 1 -- 172.21.15.7:0/1984676839 >> 172.21.15.7:0/1984676839 conn(0x55bbaac2a050 msgr2=0x55bbaac35c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.736+0000 7f637d04f740 1 -- 172.21.15.7:0/1984676839 shutdown_connections 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.736+0000 7f637d04f740 1 -- 172.21.15.7:0/1984676839 wait complete. 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.737+0000 7f637d04f740 1 -- start start 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.737+0000 7f637d04f740 1 Processor -- start 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f637d04f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac45410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f637d04f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac38ca0 0x55bbaac40730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f63737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac45410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f637d04f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 0x55bbaac40c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f63737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac45410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37874/0 (socket says 172.21.15.7:37874) 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f637d04f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63741a9ba0 con 0x55bbaac2e400 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f637d04f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63741ab010 con 0x55bbaac4cbb0 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f637d04f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63741aa270 con 0x55bbaac38ca0 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f63737fe640 1 -- 172.21.15.7:0/2306304128 learned_addr learned my addr 172.21.15.7:0/2306304128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f637d04f740 10 client.0 _ll_register_callbacks cb 0x7f63640044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f6373fff640 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 0x55bbaac40c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac38ca0 0x55bbaac40730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f63737fe640 1 -- 172.21.15.7:0/2306304128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac38ca0 msgr2=0x55bbaac40730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f63737fe640 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac38ca0 0x55bbaac40730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f63737fe640 1 -- 172.21.15.7:0/2306304128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 msgr2=0x55bbaac40c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f63737fe640 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 0x55bbaac40c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f63737fe640 1 -- 172.21.15.7:0/2306304128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbaac41470 con 0x55bbaac2e400 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f6373fff640 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 0x55bbaac40c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f63737fe640 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac45410 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6364009710 tx=0x7f6364009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2682b5cf84d26748 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.738+0000 7f6352ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f6352ffd640 1 -- 172.21.15.7:0/2306304128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636401ec00 con 0x55bbaac2e400 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f6352ffd640 1 -- 172.21.15.7:0/2306304128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f6364034a70 con 0x55bbaac2e400 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f637d04f740 1 -- 172.21.15.7:0/2306304128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bbaab9fa20 con 0x55bbaac2e400 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f6352ffd640 1 -- 172.21.15.7:0/2306304128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6364034c70 con 0x55bbaac2e400 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f637d04f740 10 client.15288 fetch_fsmap learned FSMap version 25 2024-09-10T10:46:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f637d04f740 1 -- 172.21.15.7:0/2306304128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bbaacb7930 con 0x55bbaac2e400 2024-09-10T10:46:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f6352ffd640 1 -- 172.21.15.7:0/2306304128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bbaab9fa20 con 0x55bbaac2e400 2024-09-10T10:46:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f6352ffd640 1 -- 172.21.15.7:0/2306304128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f636403b020 con 0x55bbaac2e400 2024-09-10T10:46:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f637d04f740 10 client.15288 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:46:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f637d04f740 10 client.15288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f637d04f740 20 client.15288 populate_metadata read hostname 'smithi007' 2024-09-10T10:46:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.739+0000 7f637d04f740 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bbaacb7f80 0x55bbaacba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.740+0000 7f637d04f740 4 client.15288 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:46:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.740+0000 7f637d04f740 1 -- 172.21.15.7:0/2306304128 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbaab16900 con 0x55bbaacb7f80 2024-09-10T10:46:37.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.740+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bbaacb7f80 0x55bbaacba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.740+0000 7f6352ffd640 10 client.15288 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.740+0000 7f6372ffd640 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bbaacb7f80 0x55bbaacba370 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3225974fbb4c63dd server_cookie=7bbd149606327c87 in_seq=0 out_seq=0 2024-09-10T10:46:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f6352ffd640 1 -- 172.21.15.7:0/2306304128 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbaab16900 con 0x55bbaacb7f80 2024-09-10T10:46:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f6352ffd640 10 client.15288 handle_command_reply: tid=0 2024-09-10T10:46:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 client.15288 shutdown 2024-09-10T10:46:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 -- 172.21.15.7:0/2306304128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 msgr2=0x55bbaac45410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac45410 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6364009710 tx=0x7f6364009e10 comp rx=0 tx=0).stop 2024-09-10T10:46:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 -- 172.21.15.7:0/2306304128 shutdown_connections 2024-09-10T10:46:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bbaacb7f80 0x55bbaacba370 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f6352ffd640 0 client.15288 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bbaac4cbb0 0x55bbaac40c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bbaac38ca0 0x55bbaac40730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 --2- 172.21.15.7:0/2306304128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbaac2e400 0x55bbaac45410 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 -- 172.21.15.7:0/2306304128 >> 172.21.15.7:0/2306304128 conn(0x55bbaac2a050 msgr2=0x55bbaac2fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 -- 172.21.15.7:0/2306304128 shutdown_connections 2024-09-10T10:46:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 1 -- 172.21.15.7:0/2306304128 wait complete. 2024-09-10T10:46:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.748+0000 7f637d04f740 20 client.15288 trim_cache size 0 max 16384 2024-09-10T10:46:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:46:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:46:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:46:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:46:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 -- 172.21.15.7:0/2245186058 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f635c0a2ec0 msgr2=0x7f635c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f635c0a2ec0 0x7f635c0a5380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x55bbaac2aeb0 tx=0x7f6364005f90 comp rx=0 tx=0).stop 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 -- 172.21.15.7:0/2245186058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6374101350 msgr2=0x7f63741996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6374101350 0x7f63741996d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6368006010 tx=0x7f6368003c10 comp rx=0 tx=0).stop 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 -- 172.21.15.7:0/2245186058 shutdown_connections 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f635c0a2ec0 0x7f635c0a5380 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63741069c0 0x7f63741a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6374101350 0x7f63741996d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 --2- 172.21.15.7:0/2245186058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63741009b0 0x7f6374199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 -- 172.21.15.7:0/2245186058 >> 172.21.15.7:0/2245186058 conn(0x7f6374077990 msgr2=0x7f6374103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 -- 172.21.15.7:0/2245186058 shutdown_connections 2024-09-10T10:46:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:37.749+0000 7f637a1d6640 1 -- 172.21.15.7:0/2245186058 wait complete. 2024-09-10T10:46:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:37 smithi103 ceph-mon[35160]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-10T10:46:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:37 smithi007 ceph-mon[30873]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-10T10:46:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:37 smithi133 ceph-mon[35282]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-10T10:46:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:39 smithi103 ceph-mon[35160]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T10:46:39.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:39 smithi007 ceph-mon[30873]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T10:46:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:39 smithi133 ceph-mon[35282]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T10:46:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:41 smithi103 ceph-mon[35160]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T10:46:41.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:41 smithi007 ceph-mon[30873]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T10:46:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:41 smithi133 ceph-mon[35282]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T10:46:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:43 smithi103 ceph-mon[35160]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-10T10:46:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:43 smithi007 ceph-mon[30873]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-10T10:46:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:43 smithi133 ceph-mon[35282]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-10T10:46:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:45 smithi103 ceph-mon[35160]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-10T10:46:45.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:45 smithi007 ceph-mon[30873]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-10T10:46:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:45 smithi133 ceph-mon[35282]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-10T10:46:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:47 smithi103 ceph-mon[35160]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-10T10:46:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:47 smithi007 ceph-mon[30873]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-10T10:46:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:47 smithi133 ceph-mon[35282]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-10T10:46:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:49 smithi103 ceph-mon[35160]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T10:46:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:49 smithi007 ceph-mon[30873]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T10:46:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:49 smithi133 ceph-mon[35282]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T10:46:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.882+0000 7f36058b7640 1 -- 172.21.15.7:0/38627606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36001009b0 msgr2=0x7f3600100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.882+0000 7f36058b7640 1 --2- 172.21.15.7:0/38627606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36001009b0 0x7f3600100e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009930 tx=0x7f35f402a420 comp rx=0 tx=0).stop 2024-09-10T10:46:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.883+0000 7f36058b7640 1 -- 172.21.15.7:0/38627606 shutdown_connections 2024-09-10T10:46:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.883+0000 7f36058b7640 1 --2- 172.21.15.7:0/38627606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3600101350 0x7f360010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.883+0000 7f36058b7640 1 --2- 172.21.15.7:0/38627606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36001009b0 0x7f3600100e10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.883+0000 7f36058b7640 1 --2- 172.21.15.7:0/38627606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36001069c0 0x7f3600106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.883+0000 7f36058b7640 1 -- 172.21.15.7:0/38627606 >> 172.21.15.7:0/38627606 conn(0x7f3600077990 msgr2=0x7f36000fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.883+0000 7f36058b7640 1 -- 172.21.15.7:0/38627606 shutdown_connections 2024-09-10T10:46:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.883+0000 7f36058b7640 1 -- 172.21.15.7:0/38627606 wait complete. 2024-09-10T10:46:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.884+0000 7f36058b7640 1 Processor -- start 2024-09-10T10:46:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.884+0000 7f36058b7640 1 -- start start 2024-09-10T10:46:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.885+0000 7f36058b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36001009b0 0x7f3600196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.885+0000 7f36058b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3600101350 0x7f36001966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.885+0000 7f36058b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36001069c0 0x7f360019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.885+0000 7f36058b7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f360010fd00 con 0x7f36001069c0 2024-09-10T10:46:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.885+0000 7f36058b7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f360010fb80 con 0x7f3600101350 2024-09-10T10:46:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.885+0000 7f35ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36001069c0 0x7f360019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.885+0000 7f35fe7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3600101350 0x7f36001966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.885+0000 7f35feffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36001009b0 0x7f3600196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.885+0000 7f35fe7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3600101350 0x7f36001966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:34320/0 (socket says 172.21.15.7:34320) 2024-09-10T10:46:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.886+0000 7f35fe7fc640 1 -- 172.21.15.7:0/4030407841 learned_addr learned my addr 172.21.15.7:0/4030407841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.886+0000 7f36058b7640 1 -- 172.21.15.7:0/4030407841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f360010fe80 con 0x7f36001009b0 2024-09-10T10:46:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.886+0000 7f35ff7fe640 1 -- 172.21.15.7:0/4030407841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36001009b0 msgr2=0x7f3600196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.886+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36001009b0 0x7f3600196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.886+0000 7f35ff7fe640 1 -- 172.21.15.7:0/4030407841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3600101350 msgr2=0x7f36001966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.886+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3600101350 0x7f36001966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.886+0000 7f35ff7fe640 1 -- 172.21.15.7:0/4030407841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360019b160 con 0x7f36001069c0 2024-09-10T10:46:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.886+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36001069c0 0x7f360019aa30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f35f8010f90 tx=0x7f35f800e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.887+0000 7f36048b5640 1 -- 172.21.15.7:0/4030407841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f801a070 con 0x7f36001069c0 2024-09-10T10:46:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.887+0000 7f36048b5640 1 -- 172.21.15.7:0/4030407841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f800be40 con 0x7f36001069c0 2024-09-10T10:46:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.887+0000 7f36058b7640 1 -- 172.21.15.7:0/4030407841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360019b450 con 0x7f36001069c0 2024-09-10T10:46:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.887+0000 7f36048b5640 1 -- 172.21.15.7:0/4030407841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8005770 con 0x7f36001069c0 2024-09-10T10:46:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.887+0000 7f36058b7640 1 -- 172.21.15.7:0/4030407841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36001a2ce0 con 0x7f36001069c0 2024-09-10T10:46:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.891+0000 7f3608650740 1 Processor -- start 2024-09-10T10:46:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.891+0000 7f36048b5640 1 -- 172.21.15.7:0/4030407841 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) ==== 138065+0+0 (secure 0 0 0) 0x7f35f8014050 con 0x7f36001069c0 2024-09-10T10:46:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.891+0000 7f3608650740 1 -- start start 2024-09-10T10:46:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.891+0000 7f3608650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19286af0 0x558b19286ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.891+0000 7f3608650740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 0x558b19291770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.891+0000 7f3608650740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.891+0000 7f3608650740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558b191eb2f0 con 0x558b19291cb0 2024-09-10T10:46:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.891+0000 7f3608650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558b19193270 con 0x558b19286af0 2024-09-10T10:46:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.891+0000 7f3608650740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558b191e72b0 con 0x558b19287410 2024-09-10T10:46:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f35fe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f35feffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 0x558b19291770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f35fe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192940a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41984/0 (socket says 172.21.15.7:41984) 2024-09-10T10:46:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f35fe7fc640 1 -- 172.21.15.7:0/2861422154 learned_addr learned my addr 172.21.15.7:0/2861422154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f35feffd640 1 -- 172.21.15.7:0/2861422154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19286af0 msgr2=0x558b19286ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f35feffd640 1 --2- 172.21.15.7:0/2861422154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19286af0 0x558b19286ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f35feffd640 1 -- 172.21.15.7:0/2861422154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 msgr2=0x558b192940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f35feffd640 1 --2- 172.21.15.7:0/2861422154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192940a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f35feffd640 1 -- 172.21.15.7:0/2861422154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b19192340 con 0x558b19287410 2024-09-10T10:46:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f35fe7fc640 1 --2- 172.21.15.7:0/2861422154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.892+0000 7f36048b5640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f35e80a2f40 0x7f35e80a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.893+0000 7f35feffd640 1 --2- 172.21.15.7:0/2861422154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 0x558b19291770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f35f0005fd0 tx=0x7f35f000b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d09c406b4225080c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.893+0000 7f36048b5640 1 -- 172.21.15.7:0/4030407841 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f35f80d1760 con 0x7f36001069c0 2024-09-10T10:46:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.893+0000 7f35fe7fc640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f35e80a2f40 0x7f35e80a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.893+0000 7f35e5ffb640 1 -- 172.21.15.7:0/2861422154 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0020070 con 0x558b19287410 2024-09-10T10:46:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.893+0000 7f35e5ffb640 1 -- 172.21.15.7:0/2861422154 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f001b8f0 con 0x558b19287410 2024-09-10T10:46:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.893+0000 7f35e5ffb640 1 -- 172.21.15.7:0/2861422154 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f000d860 con 0x558b19287410 2024-09-10T10:46:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.894+0000 7f3608650740 1 -- 172.21.15.7:0/2861422154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 msgr2=0x558b19291770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.894+0000 7f35fe7fc640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f35e80a2f40 0x7f35e80a5400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x558b192843e0 tx=0x7f35f4009620 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.894+0000 7f3608650740 1 --2- 172.21.15.7:0/2861422154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 0x558b19291770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f35f0005fd0 tx=0x7f35f000b730 comp rx=0 tx=0).stop 2024-09-10T10:46:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.894+0000 7f3608650740 1 -- 172.21.15.7:0/2861422154 shutdown_connections 2024-09-10T10:46:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.894+0000 7f3608650740 1 --2- 172.21.15.7:0/2861422154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.894+0000 7f3608650740 1 --2- 172.21.15.7:0/2861422154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 0x558b19291770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.894+0000 7f3608650740 1 --2- 172.21.15.7:0/2861422154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19286af0 0x558b19286ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.894+0000 7f3608650740 1 -- 172.21.15.7:0/2861422154 >> 172.21.15.7:0/2861422154 conn(0x558b19283060 msgr2=0x558b19283440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.895+0000 7f3608650740 1 -- 172.21.15.7:0/2861422154 shutdown_connections 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.895+0000 7f3608650740 1 -- 172.21.15.7:0/2861422154 wait complete. 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.895+0000 7f3608650740 1 -- start start 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.895+0000 7f3608650740 1 Processor -- start 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f3608650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19286af0 0x558b1928ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f3608650740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19287410 0x558b1928f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f3608650740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19291cb0 0x558b1928f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f3608650740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558b19294880 con 0x558b19287410 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f3608650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558b19174290 con 0x558b19286af0 2024-09-10T10:46:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f3608650740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558b19294a00 con 0x558b19291cb0 2024-09-10T10:46:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35fe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19287410 0x558b1928f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f3608650740 10 client.0 _ll_register_callbacks cb 0x558b191a5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35fe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19287410 0x558b1928f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41994/0 (socket says 172.21.15.7:41994) 2024-09-10T10:46:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35fe7fc640 1 -- 172.21.15.7:0/1565271336 learned_addr learned my addr 172.21.15.7:0/1565271336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35ff7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19291cb0 0x558b1928f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35feffd640 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19286af0 0x558b1928ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35ff7fe640 1 -- 172.21.15.7:0/1565271336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19286af0 msgr2=0x558b1928ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19286af0 0x558b1928ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35ff7fe640 1 -- 172.21.15.7:0/1565271336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19287410 msgr2=0x558b1928f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19287410 0x558b1928f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35ff7fe640 1 -- 172.21.15.7:0/1565271336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b18efe210 con 0x558b19291cb0 2024-09-10T10:46:50.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19291cb0 0x558b1928f640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35f80ced00 tx=0x7f35f809e270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75cf7e243b88e524 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.896+0000 7f35deffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.897+0000 7f35deffd640 1 -- 172.21.15.7:0/1565271336 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f80cf010 con 0x558b19291cb0 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.897+0000 7f35deffd640 1 -- 172.21.15.7:0/1565271336 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f35f80cf1b0 con 0x558b19291cb0 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.897+0000 7f3608650740 1 -- 172.21.15.7:0/1565271336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f001b8f0 con 0x558b19291cb0 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.897+0000 7f35deffd640 1 -- 172.21.15.7:0/1565271336 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35f809ce80 con 0x558b19291cb0 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.897+0000 7f3608650740 10 client.24988 fetch_fsmap learned FSMap version 25 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.897+0000 7f3608650740 1 -- 172.21.15.7:0/1565271336 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b18fca620 con 0x558b19291cb0 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.897+0000 7f35deffd640 1 -- 172.21.15.7:0/1565271336 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f001b8f0 con 0x558b19291cb0 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.898+0000 7f35deffd640 1 -- 172.21.15.7:0/1565271336 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f35f8099980 con 0x558b19291cb0 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.898+0000 7f3608650740 10 client.24988 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.898+0000 7f3608650740 10 client.24988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.898+0000 7f3608650740 20 client.24988 populate_metadata read hostname 'smithi007' 2024-09-10T10:46:50.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.898+0000 7f3608650740 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558b192a1ca0 0x558b192a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.898+0000 7f3608650740 4 client.24988 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.898+0000 7f3608650740 1 -- 172.21.15.7:0/1565271336 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b191f8a20 con 0x558b192a1ca0 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.898+0000 7f35feffd640 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558b192a1ca0 0x558b192a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.899+0000 7f35deffd640 10 client.24988 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.899+0000 7f35feffd640 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558b192a1ca0 0x558b192a4090 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7427edfa0255434 server_cookie=b39fca1db0a04a77 in_seq=0 out_seq=0 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f35deffd640 1 -- 172.21.15.7:0/1565271336 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b191f8a20 con 0x558b192a1ca0 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f35deffd640 10 client.24988 handle_command_reply: tid=0 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 client.24988 shutdown 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 -- 172.21.15.7:0/1565271336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19291cb0 msgr2=0x558b1928f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19291cb0 0x558b1928f640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f35f80ced00 tx=0x7f35f809e270 comp rx=0 tx=0).stop 2024-09-10T10:46:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 -- 172.21.15.7:0/1565271336 shutdown_connections 2024-09-10T10:46:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558b192a1ca0 0x558b192a4090 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19291cb0 0x558b1928f640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19287410 0x558b1928f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f35deffd640 0 client.24988 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 --2- 172.21.15.7:0/1565271336 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19286af0 0x558b1928ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 -- 172.21.15.7:0/1565271336 >> 172.21.15.7:0/1565271336 conn(0x558b19283060 msgr2=0x558b19288d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 -- 172.21.15.7:0/1565271336 shutdown_connections 2024-09-10T10:46:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 1 -- 172.21.15.7:0/1565271336 wait complete. 2024-09-10T10:46:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.900+0000 7f3608650740 20 client.24988 trim_cache size 0 max 16384 2024-09-10T10:46:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.930+0000 7f3608650740 1 Processor -- start 2024-09-10T10:46:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.930+0000 7f3608650740 1 -- start start 2024-09-10T10:46:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f3608650740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19286af0 0x558b1928f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f3608650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19287410 0x558b192a9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f3608650740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f3608650740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36001ade80 con 0x558b19291cb0 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f3608650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36001ab750 con 0x558b19287410 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f3608650740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f36001a95a0 con 0x558b19286af0 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f35ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f35feffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19286af0 0x558b1928f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f35ff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192a9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42002/0 (socket says 172.21.15.7:42002) 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f35feffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19286af0 0x558b1928f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48266/0 (socket says 172.21.15.7:48266) 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f35ff7fe640 1 -- 172.21.15.7:0/611113448 learned_addr learned my addr 172.21.15.7:0/611113448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f35fe7fc640 1 --2- 172.21.15.7:0/611113448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19287410 0x558b192a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f35ff7fe640 1 -- 172.21.15.7:0/611113448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19286af0 msgr2=0x558b1928f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/611113448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19286af0 0x558b1928f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f35ff7fe640 1 -- 172.21.15.7:0/611113448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19287410 msgr2=0x558b192a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.931+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/611113448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19287410 0x558b192a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.932+0000 7f35ff7fe640 1 -- 172.21.15.7:0/611113448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b19287240 con 0x558b19291cb0 2024-09-10T10:46:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.932+0000 7f35feffd640 1 --2- 172.21.15.7:0/611113448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19286af0 0x558b1928f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.932+0000 7f35fe7fc640 1 --2- 172.21.15.7:0/611113448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19287410 0x558b192a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.932+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/611113448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192a9eb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35f809a3b0 tx=0x7f35f800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3d772191bdf839 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.932+0000 7f35e57fa640 1 -- 172.21.15.7:0/611113448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f809cc10 con 0x558b19291cb0 2024-09-10T10:46:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.932+0000 7f35e57fa640 1 -- 172.21.15.7:0/611113448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f809c8f0 con 0x558b19291cb0 2024-09-10T10:46:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.932+0000 7f35e57fa640 1 -- 172.21.15.7:0/611113448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8005a70 con 0x558b19291cb0 2024-09-10T10:46:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.933+0000 7f3608650740 1 -- 172.21.15.7:0/611113448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 msgr2=0x558b192a9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.933+0000 7f3608650740 1 --2- 172.21.15.7:0/611113448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192a9eb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35f809a3b0 tx=0x7f35f800ef30 comp rx=0 tx=0).stop 2024-09-10T10:46:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.933+0000 7f3608650740 1 -- 172.21.15.7:0/611113448 shutdown_connections 2024-09-10T10:46:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.933+0000 7f3608650740 1 --2- 172.21.15.7:0/611113448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19291cb0 0x558b192a9eb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.933+0000 7f3608650740 1 --2- 172.21.15.7:0/611113448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19287410 0x558b192a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.933+0000 7f3608650740 1 --2- 172.21.15.7:0/611113448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19286af0 0x558b1928f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.933+0000 7f3608650740 1 -- 172.21.15.7:0/611113448 >> 172.21.15.7:0/611113448 conn(0x558b19283060 msgr2=0x558b19290c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.933+0000 7f3608650740 1 -- 172.21.15.7:0/611113448 shutdown_connections 2024-09-10T10:46:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.933+0000 7f3608650740 1 -- 172.21.15.7:0/611113448 wait complete. 2024-09-10T10:46:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.933+0000 7f3608650740 1 -- start start 2024-09-10T10:46:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f3608650740 1 Processor -- start 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f3608650740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19286af0 0x558b1928baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f3608650740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 0x558b1928c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f3608650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19291cb0 0x558b1928c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f3608650740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36001a9c10 con 0x558b19286af0 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f3608650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36001ab080 con 0x558b19291cb0 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f3608650740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f36001aa2e0 con 0x558b19287410 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f35feffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19286af0 0x558b1928baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f35feffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19286af0 0x558b1928baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42018/0 (socket says 172.21.15.7:42018) 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f35feffd640 1 -- 172.21.15.7:0/796396148 learned_addr learned my addr 172.21.15.7:0/796396148 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f3608650740 10 client.0 _ll_register_callbacks cb 0x7f35f8099cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19291cb0 0x558b1928c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.934+0000 7f35fe7fc640 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 0x558b1928c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35ff7fe640 1 -- 172.21.15.7:0/796396148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 msgr2=0x558b1928c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 0x558b1928c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35ff7fe640 1 -- 172.21.15.7:0/796396148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19286af0 msgr2=0x558b1928baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19286af0 0x558b1928baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35ff7fe640 1 -- 172.21.15.7:0/796396148 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b1930b940 con 0x558b19291cb0 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35feffd640 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19286af0 0x558b1928baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35ff7fe640 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19291cb0 0x558b1928c570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35f80a1fa0 tx=0x7f35f8005190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a58b5c842aa30127 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35deffd640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35deffd640 1 -- 172.21.15.7:0/796396148 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f809e130 con 0x558b19291cb0 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35deffd640 1 -- 172.21.15.7:0/796396148 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f35f8005a10 con 0x558b19291cb0 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f3608650740 1 -- 172.21.15.7:0/796396148 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f809c8f0 con 0x558b19291cb0 2024-09-10T10:46:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f35deffd640 1 -- 172.21.15.7:0/796396148 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35f80cf220 con 0x558b19291cb0 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f3608650740 10 client.24998 fetch_fsmap learned FSMap version 25 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.935+0000 7f3608650740 1 -- 172.21.15.7:0/796396148 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558b1930cb90 con 0x558b19291cb0 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.936+0000 7f35deffd640 1 -- 172.21.15.7:0/796396148 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f809c8f0 con 0x558b19291cb0 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.936+0000 7f35deffd640 1 -- 172.21.15.7:0/796396148 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f35f80a3000 con 0x558b19291cb0 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.936+0000 7f3608650740 10 client.24998 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.936+0000 7f3608650740 10 client.24998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.936+0000 7f3608650740 20 client.24998 populate_metadata read hostname 'smithi007' 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.936+0000 7f3608650740 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558b192a1ca0 0x558b1930f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.937+0000 7f3608650740 4 client.24998 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.937+0000 7f3608650740 1 -- 172.21.15.7:0/796396148 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b191f2d70 con 0x558b192a1ca0 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.937+0000 7f35feffd640 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558b192a1ca0 0x558b1930f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.937+0000 7f35deffd640 10 client.24998 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.937+0000 7f35feffd640 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558b192a1ca0 0x558b1930f020 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7313925047893b0e server_cookie=26bebf94cf2d0977 in_seq=0 out_seq=0 2024-09-10T10:46:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.941+0000 7f35deffd640 1 -- 172.21.15.7:0/796396148 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b191f2d70 con 0x558b192a1ca0 2024-09-10T10:46:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.941+0000 7f35deffd640 10 client.24998 handle_command_reply: tid=0 2024-09-10T10:46:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.941+0000 7f3608650740 1 client.24998 shutdown 2024-09-10T10:46:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 1 -- 172.21.15.7:0/796396148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19291cb0 msgr2=0x558b1928c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19291cb0 0x558b1928c570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35f80a1fa0 tx=0x7f35f8005190 comp rx=0 tx=0).stop 2024-09-10T10:46:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 1 -- 172.21.15.7:0/796396148 shutdown_connections 2024-09-10T10:46:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558b192a1ca0 0x558b1930f020 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b19291cb0 0x558b1928c570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f35deffd640 0 client.24998 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:46:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558b19287410 0x558b1928c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 1 --2- 172.21.15.7:0/796396148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b19286af0 0x558b1928baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 1 -- 172.21.15.7:0/796396148 >> 172.21.15.7:0/796396148 conn(0x558b19283060 msgr2=0x558b192819a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 1 -- 172.21.15.7:0/796396148 shutdown_connections 2024-09-10T10:46:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 1 -- 172.21.15.7:0/796396148 wait complete. 2024-09-10T10:46:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.942+0000 7f3608650740 20 client.24998 trim_cache size 0 max 16384 2024-09-10T10:46:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:46:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:46:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:46:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:46:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 -- 172.21.15.7:0/4030407841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f35e80a2f40 msgr2=0x7f35e80a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f35e80a2f40 0x7f35e80a5400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x558b192843e0 tx=0x7f35f4009620 comp rx=0 tx=0).stop 2024-09-10T10:46:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 -- 172.21.15.7:0/4030407841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36001069c0 msgr2=0x7f360019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36001069c0 0x7f360019aa30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f35f8010f90 tx=0x7f35f800e490 comp rx=0 tx=0).stop 2024-09-10T10:46:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 -- 172.21.15.7:0/4030407841 shutdown_connections 2024-09-10T10:46:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f35e80a2f40 0x7f35e80a5400 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36001069c0 0x7f360019aa30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3600101350 0x7f36001966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 --2- 172.21.15.7:0/4030407841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36001009b0 0x7f3600196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 -- 172.21.15.7:0/4030407841 >> 172.21.15.7:0/4030407841 conn(0x7f3600077990 msgr2=0x7f3600103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 -- 172.21.15.7:0/4030407841 shutdown_connections 2024-09-10T10:46:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:46:50.943+0000 7f36058b7640 1 -- 172.21.15.7:0/4030407841 wait complete. 2024-09-10T10:46:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:51 smithi103 ceph-mon[35160]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-10T10:46:51.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:51 smithi007 ceph-mon[30873]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-10T10:46:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:51 smithi133 ceph-mon[35282]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-10T10:46:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:53 smithi103 ceph-mon[35160]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T10:46:53.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:53 smithi007 ceph-mon[30873]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T10:46:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:53 smithi133 ceph-mon[35282]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-10T10:46:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:55 smithi103 ceph-mon[35160]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:46:55.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:55 smithi007 ceph-mon[30873]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:46:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:55 smithi133 ceph-mon[35282]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T10:46:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:57 smithi103 ceph-mon[35160]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:46:57.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:57 smithi007 ceph-mon[30873]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:46:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:57 smithi133 ceph-mon[35282]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:46:58.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:46:58.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:46:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:46:58.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:46:58.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:46:58.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:46:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:46:58.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:46:58.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:46:58.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:46:58.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:46:58.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:46:59.010 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 91% (11604/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T10:46:59.321 INFO:tasks.workunit.client.0.smithi007.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr: git switch -c 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr:Or undo this operation with: 2024-09-10T10:46:59.322 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-10T10:46:59.323 INFO:tasks.workunit.client.0.smithi007.stderr: git switch - 2024-09-10T10:46:59.323 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-10T10:46:59.323 INFO:tasks.workunit.client.0.smithi007.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T10:46:59.323 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-09-10T10:46:59.323 INFO:tasks.workunit.client.0.smithi007.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:46:59.327 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T10:46:59.395 INFO:tasks.workunit.client.0.smithi007.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T10:46:59.398 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T10:46:59.398 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T10:46:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:46:59 smithi103 ceph-mon[35160]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:46:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:46:59 smithi007 ceph-mon[30873]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:46:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:46:59 smithi133 ceph-mon[35282]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:47:00.098 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T10:47:00.184 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T10:47:00.226 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T10:47:00.228 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T10:47:00.228 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T10:47:00.268 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T10:47:00.271 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T10:47:00.271 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T10:47:00.328 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-10T10:47:00.329 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-10T10:47:00.329 DEBUG:teuthology.orchestra.run.smithi007:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-10T10:47:00.390 INFO:tasks.workunit.client.0.smithi007.stderr:+ dbench 1 2024-09-10T10:47:00.396 INFO:tasks.workunit.client.0.smithi007.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-10T10:47:00.397 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-09-10T10:47:00.397 INFO:tasks.workunit.client.0.smithi007.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-10T10:47:00.402 INFO:tasks.workunit.client.0.smithi007.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-10T10:47:00.502 INFO:tasks.workunit.client.0.smithi007.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-10T10:47:00.502 INFO:tasks.workunit.client.0.smithi007.stdout:releasing clients 2024-09-10T10:47:01.502 INFO:tasks.workunit.client.0.smithi007.stdout: 1 3843 39.76 MB/sec warmup 1 sec latency 8.063 ms 2024-09-10T10:47:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:01 smithi103 ceph-mon[35160]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:47:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:01 smithi007 ceph-mon[30873]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:47:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:01 smithi133 ceph-mon[35282]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-10T10:47:02.502 INFO:tasks.workunit.client.0.smithi007.stdout: 1 7512 29.59 MB/sec warmup 2 sec latency 11.708 ms 2024-09-10T10:47:03.502 INFO:tasks.workunit.client.0.smithi007.stdout: 1 11038 26.10 MB/sec warmup 3 sec latency 11.503 ms 2024-09-10T10:47:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:03 smithi103 ceph-mon[35160]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:47:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:03 smithi007 ceph-mon[30873]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:47:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:03 smithi133 ceph-mon[35282]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:47:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.039+0000 7f27572d0640 1 -- 172.21.15.7:0/4064023446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2750102480 msgr2=0x7f275010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.039+0000 7f27572d0640 1 --2- 172.21.15.7:0/4064023446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2750102480 0x7f275010e9b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2748009cd0 tx=0x7f274802a8e0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.039+0000 7f27572d0640 1 -- 172.21.15.7:0/4064023446 shutdown_connections 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.039+0000 7f27572d0640 1 --2- 172.21.15.7:0/4064023446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2750102480 0x7f275010e9b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.039+0000 7f27572d0640 1 --2- 172.21.15.7:0/4064023446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750101ae0 0x7f2750101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.039+0000 7f27572d0640 1 --2- 172.21.15.7:0/4064023446 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2750107b80 0x7f2750107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.039+0000 7f27572d0640 1 -- 172.21.15.7:0/4064023446 >> 172.21.15.7:0/4064023446 conn(0x7f27500fd7e0 msgr2=0x7f27500ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.039+0000 7f27572d0640 1 -- 172.21.15.7:0/4064023446 shutdown_connections 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.039+0000 7f27572d0640 1 -- 172.21.15.7:0/4064023446 wait complete. 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f27572d0640 1 Processor -- start 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f27572d0640 1 -- start start 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f27572d0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750101ae0 0x7f2750198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f27572d0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2750102480 0x7f2750198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f27572d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2750107b80 0x7f275019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f27572d0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2750111f80 con 0x7f2750107b80 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f27572d0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2750111e00 con 0x7f2750101ae0 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f27572d0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2750112100 con 0x7f2750102480 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f275586d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2750107b80 0x7f275019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f275506c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750101ae0 0x7f2750198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f275586d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2750107b80 0x7f275019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46728/0 (socket says 172.21.15.7:46728) 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f275586d640 1 -- 172.21.15.7:0/2123688227 learned_addr learned my addr 172.21.15.7:0/2123688227 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.040+0000 7f275486b640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2750102480 0x7f2750198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f275506c640 1 -- 172.21.15.7:0/2123688227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2750102480 msgr2=0x7f2750198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f275506c640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2750102480 0x7f2750198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f275506c640 1 -- 172.21.15.7:0/2123688227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2750107b80 msgr2=0x7f275019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f275506c640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2750107b80 0x7f275019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f275506c640 1 -- 172.21.15.7:0/2123688227 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275019d340 con 0x7f2750101ae0 2024-09-10T10:47:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f275586d640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2750107b80 0x7f275019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f275506c640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750101ae0 0x7f2750198350 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f274c00e9f0 tx=0x7f274c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f27467fc640 1 -- 172.21.15.7:0/2123688227 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c00ccc0 con 0x7f2750101ae0 2024-09-10T10:47:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f27572d0640 1 -- 172.21.15.7:0/2123688227 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275019d630 con 0x7f2750101ae0 2024-09-10T10:47:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f27467fc640 1 -- 172.21.15.7:0/2123688227 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274c005020 con 0x7f2750101ae0 2024-09-10T10:47:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f27467fc640 1 -- 172.21.15.7:0/2123688227 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c0049e0 con 0x7f2750101ae0 2024-09-10T10:47:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.041+0000 7f27572d0640 1 -- 172.21.15.7:0/2123688227 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27501a4ec0 con 0x7f2750101ae0 2024-09-10T10:47:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.044+0000 7f27467fc640 1 -- 172.21.15.7:0/2123688227 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 24) ==== 138065+0+0 (secure 0 0 0) 0x7f274c005560 con 0x7f2750101ae0 2024-09-10T10:47:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.044+0000 7f275a06a740 1 Processor -- start 2024-09-10T10:47:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.044+0000 7f275a06a740 1 -- start start 2024-09-10T10:47:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275a06a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c49ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275a06a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 0x55bf08c54770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275a06a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275a06a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bf08bae2f0 con 0x55bf08c49af0 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275a06a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bf08b56270 con 0x55bf08c4a410 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275a06a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bf08baa2b0 con 0x55bf08c54cb0 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275586d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c49ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275586d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c49ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46738/0 (socket says 172.21.15.7:46738) 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275586d640 1 -- 172.21.15.7:0/1813799049 learned_addr learned my addr 172.21.15.7:0/1813799049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275486b640 1 --2- 172.21.15.7:0/1813799049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275506c640 1 --2- 172.21.15.7:0/1813799049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 0x55bf08c54770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f27467fc640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f273c0a2ec0 0x7f273c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275586d640 1 -- 172.21.15.7:0/1813799049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 msgr2=0x55bf08c570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275586d640 1 --2- 172.21.15.7:0/1813799049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275586d640 1 -- 172.21.15.7:0/1813799049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 msgr2=0x55bf08c54770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275586d640 1 --2- 172.21.15.7:0/1813799049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 0x55bf08c54770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275586d640 1 -- 172.21.15.7:0/1813799049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf08b55340 con 0x55bf08c49af0 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275506c640 1 --2- 172.21.15.7:0/1813799049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 0x55bf08c54770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f27467fc640 1 -- 172.21.15.7:0/2123688227 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f274c0cf5d0 con 0x7f2750101ae0 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275586d640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f273c0a2ec0 0x7f273c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.045+0000 7f275586d640 1 --2- 172.21.15.7:0/1813799049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c49ed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2748009a20 tx=0x7f27480093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96cbb95fd6f6adf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.046+0000 7f273b7fe640 1 -- 172.21.15.7:0/1813799049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748003970 con 0x55bf08c49af0 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.046+0000 7f273b7fe640 1 -- 172.21.15.7:0/1813799049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2748036040 con 0x55bf08c49af0 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.046+0000 7f273b7fe640 1 -- 172.21.15.7:0/1813799049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748030eb0 con 0x55bf08c49af0 2024-09-10T10:47:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.046+0000 7f275a06a740 1 -- 172.21.15.7:0/1813799049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 msgr2=0x55bf08c49ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.046+0000 7f275586d640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f273c0a2ec0 0x7f273c0a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f274804a000 tx=0x7f274804b000 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.046+0000 7f275a06a740 1 --2- 172.21.15.7:0/1813799049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c49ed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2748009a20 tx=0x7f27480093a0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 -- 172.21.15.7:0/1813799049 shutdown_connections 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 --2- 172.21.15.7:0/1813799049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 --2- 172.21.15.7:0/1813799049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 0x55bf08c54770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 --2- 172.21.15.7:0/1813799049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c49ed0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 -- 172.21.15.7:0/1813799049 >> 172.21.15.7:0/1813799049 conn(0x55bf08c46060 msgr2=0x55bf08c46440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 -- 172.21.15.7:0/1813799049 shutdown_connections 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 -- 172.21.15.7:0/1813799049 wait complete. 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 -- start start 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 Processor -- start 2024-09-10T10:47:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c49af0 0x55bf08c51b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c520b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c54cb0 0x55bf08c525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.047+0000 7f275a06a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bf08c57880 con 0x55bf08c4a410 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275a06a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bf08b37290 con 0x55bf08c54cb0 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275a06a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bf08c57a00 con 0x55bf08c49af0 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275a06a740 10 client.0 _ll_register_callbacks cb 0x55bf08b68140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275486b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275506c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c49af0 0x55bf08c51b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275486b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46744/0 (socket says 172.21.15.7:46744) 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275486b640 1 -- 172.21.15.7:0/3280179505 learned_addr learned my addr 172.21.15.7:0/3280179505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275586d640 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c54cb0 0x55bf08c525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275486b640 1 -- 172.21.15.7:0/3280179505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c49af0 msgr2=0x55bf08c51b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275486b640 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c49af0 0x55bf08c51b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275486b640 1 -- 172.21.15.7:0/3280179505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c54cb0 msgr2=0x55bf08c525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275486b640 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c54cb0 0x55bf08c525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275486b640 1 -- 172.21.15.7:0/3280179505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf088c1210 con 0x55bf08c4a410 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275486b640 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c520b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2740012730 tx=0x7f2740012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6551e4f231fe7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f2738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f2738ff9640 1 -- 172.21.15.7:0/3280179505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274000f920 con 0x55bf08c4a410 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f275a06a740 1 -- 172.21.15.7:0/3280179505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2748036040 con 0x55bf08c4a410 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.048+0000 7f2738ff9640 1 -- 172.21.15.7:0/3280179505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f274000fac0 con 0x55bf08c4a410 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f2738ff9640 1 -- 172.21.15.7:0/3280179505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f274000fcc0 con 0x55bf08c4a410 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f275a06a740 10 client.15318 fetch_fsmap learned FSMap version 25 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f275a06a740 1 -- 172.21.15.7:0/3280179505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf0898d620 con 0x55bf08c4a410 2024-09-10T10:47:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f2738ff9640 1 -- 172.21.15.7:0/3280179505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2748036040 con 0x55bf08c4a410 2024-09-10T10:47:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f2738ff9640 1 -- 172.21.15.7:0/3280179505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f274001c420 con 0x55bf08c4a410 2024-09-10T10:47:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f275a06a740 10 client.15318 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:47:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f275a06a740 10 client.15318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f275a06a740 20 client.15318 populate_metadata read hostname 'smithi007' 2024-09-10T10:47:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f275a06a740 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bf08c64cf0 0x55bf08c670e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f275a06a740 4 client.15318 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f275a06a740 1 -- 172.21.15.7:0/3280179505 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf08bbba20 con 0x55bf08c64cf0 2024-09-10T10:47:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.049+0000 7f275506c640 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bf08c64cf0 0x55bf08c670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.050+0000 7f2738ff9640 10 client.15318 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.050+0000 7f275506c640 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bf08c64cf0 0x55bf08c670e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=442af3d02f2f36a5 server_cookie=fe130abc300366be in_seq=0 out_seq=0 2024-09-10T10:47:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f2738ff9640 1 -- 172.21.15.7:0/3280179505 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf08bbba20 con 0x55bf08c64cf0 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f2738ff9640 10 client.15318 handle_command_reply: tid=0 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 client.15318 shutdown 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 -- 172.21.15.7:0/3280179505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 msgr2=0x55bf08c520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c520b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2740012730 tx=0x7f2740012bf0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 -- 172.21.15.7:0/3280179505 shutdown_connections 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bf08c64cf0 0x55bf08c670e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c54cb0 0x55bf08c525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f2738ff9640 0 client.15318 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c520b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 --2- 172.21.15.7:0/3280179505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c49af0 0x55bf08c51b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 -- 172.21.15.7:0/3280179505 >> 172.21.15.7:0/3280179505 conn(0x55bf08c46060 msgr2=0x55bf08c4bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 -- 172.21.15.7:0/3280179505 shutdown_connections 2024-09-10T10:47:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 1 -- 172.21.15.7:0/3280179505 wait complete. 2024-09-10T10:47:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.051+0000 7f275a06a740 20 client.15318 trim_cache size 0 max 16384 2024-09-10T10:47:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275a06a740 1 Processor -- start 2024-09-10T10:47:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275a06a740 1 -- start start 2024-09-10T10:47:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275a06a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c49af0 0x55bf08c4c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275a06a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c52420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275a06a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c6c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275a06a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27501b0070 con 0x55bf08c4a410 2024-09-10T10:47:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275a06a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27501ad940 con 0x55bf08c49af0 2024-09-10T10:47:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275a06a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f27501ab790 con 0x55bf08c54cb0 2024-09-10T10:47:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275486b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c52420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275486b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c52420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46746/0 (socket says 172.21.15.7:46746) 2024-09-10T10:47:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275486b640 1 -- 172.21.15.7:0/3004307661 learned_addr learned my addr 172.21.15.7:0/3004307661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275586d640 1 --2- 172.21.15.7:0/3004307661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c6c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.066+0000 7f275506c640 1 --2- 172.21.15.7:0/3004307661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c49af0 0x55bf08c4c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.067+0000 7f275486b640 1 -- 172.21.15.7:0/3004307661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 msgr2=0x55bf08c6c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.067+0000 7f275486b640 1 --2- 172.21.15.7:0/3004307661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c6c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.067+0000 7f275486b640 1 -- 172.21.15.7:0/3004307661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c49af0 msgr2=0x55bf08c4c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.067+0000 7f275486b640 1 --2- 172.21.15.7:0/3004307661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c49af0 0x55bf08c4c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.067+0000 7f275486b640 1 -- 172.21.15.7:0/3004307661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf08c45e90 con 0x55bf08c4a410 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.067+0000 7f275506c640 1 --2- 172.21.15.7:0/3004307661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c49af0 0x55bf08c4c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.067+0000 7f275486b640 1 --2- 172.21.15.7:0/3004307661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c52420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2740009e00 tx=0x7f27400098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9aed2f922817447 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.067+0000 7f273affd640 1 -- 172.21.15.7:0/3004307661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274000f920 con 0x55bf08c4a410 2024-09-10T10:47:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.067+0000 7f273affd640 1 -- 172.21.15.7:0/3004307661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274000fac0 con 0x55bf08c4a410 2024-09-10T10:47:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.067+0000 7f273affd640 1 -- 172.21.15.7:0/3004307661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27400056d0 con 0x55bf08c4a410 2024-09-10T10:47:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 -- 172.21.15.7:0/3004307661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 msgr2=0x55bf08c52420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 --2- 172.21.15.7:0/3004307661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c52420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2740009e00 tx=0x7f27400098c0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 -- 172.21.15.7:0/3004307661 shutdown_connections 2024-09-10T10:47:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 --2- 172.21.15.7:0/3004307661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c6c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 --2- 172.21.15.7:0/3004307661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c4a410 0x55bf08c52420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 --2- 172.21.15.7:0/3004307661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c49af0 0x55bf08c4c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 -- 172.21.15.7:0/3004307661 >> 172.21.15.7:0/3004307661 conn(0x55bf08c46060 msgr2=0x55bf08c53bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 -- 172.21.15.7:0/3004307661 shutdown_connections 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 -- 172.21.15.7:0/3004307661 wait complete. 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 -- start start 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.068+0000 7f275a06a740 1 Processor -- start 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275a06a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c5d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275a06a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 0x55bf08c5d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275a06a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c58580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275a06a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27501abe00 con 0x55bf08c49af0 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275a06a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27501ad270 con 0x55bf08c4a410 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275a06a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f27501ac4d0 con 0x55bf08c54cb0 2024-09-10T10:47:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275a06a740 10 client.0 _ll_register_callbacks cb 0x7f2740009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275506c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c5d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275486b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 0x55bf08c5d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275506c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c5d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46756/0 (socket says 172.21.15.7:46756) 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275586d640 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275506c640 1 -- 172.21.15.7:0/2449184430 learned_addr learned my addr 172.21.15.7:0/2449184430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275486b640 1 -- 172.21.15.7:0/2449184430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 msgr2=0x55bf08c58580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275486b640 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c58580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275486b640 1 -- 172.21.15.7:0/2449184430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 msgr2=0x55bf08c5d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275486b640 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c5d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275486b640 1 -- 172.21.15.7:0/2449184430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf08c54700 con 0x55bf08c4a410 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275506c640 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c5d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275586d640 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c58580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f275486b640 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 0x55bf08c5d550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55bf08c54c60 tx=0x7f27400123e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8355e29fe9cd72dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f2738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.069+0000 7f2738ff9640 1 -- 172.21.15.7:0/2449184430 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274000f920 con 0x55bf08c4a410 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f275a06a740 1 -- 172.21.15.7:0/2449184430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f274000fac0 con 0x55bf08c4a410 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f2738ff9640 1 -- 172.21.15.7:0/2449184430 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f2740018040 con 0x55bf08c4a410 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f2738ff9640 1 -- 172.21.15.7:0/2449184430 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2740039050 con 0x55bf08c4a410 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f275a06a740 10 client.25016 fetch_fsmap learned FSMap version 25 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f275a06a740 1 -- 172.21.15.7:0/2449184430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bf08c47110 con 0x55bf08c4a410 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f2738ff9640 1 -- 172.21.15.7:0/2449184430 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274000fac0 con 0x55bf08c4a410 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f2738ff9640 1 -- 172.21.15.7:0/2449184430 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2740003c50 con 0x55bf08c4a410 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f275a06a740 10 client.25016 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:47:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f275a06a740 10 client.25016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f275a06a740 20 client.25016 populate_metadata read hostname 'smithi007' 2024-09-10T10:47:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f275a06a740 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bf08cd3c50 0x55bf08cd6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f275a06a740 4 client.25016 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:47:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f275a06a740 1 -- 172.21.15.7:0/2449184430 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf08bb5d70 con 0x55bf08cd3c50 2024-09-10T10:47:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.070+0000 7f275506c640 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bf08cd3c50 0x55bf08cd6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.071+0000 7f2738ff9640 10 client.25016 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.071+0000 7f275506c640 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bf08cd3c50 0x55bf08cd6070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3085be64774ed16 server_cookie=40daf1c54902fb11 in_seq=0 out_seq=0 2024-09-10T10:47:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.160+0000 7f2738ff9640 1 -- 172.21.15.7:0/2449184430 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf08bb5d70 con 0x55bf08cd3c50 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.160+0000 7f2738ff9640 10 client.25016 handle_command_reply: tid=0 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.160+0000 7f275a06a740 1 client.25016 shutdown 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 1 -- 172.21.15.7:0/2449184430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 msgr2=0x55bf08c5d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 0x55bf08c5d550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55bf08c54c60 tx=0x7f27400123e0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 1 -- 172.21.15.7:0/2449184430 shutdown_connections 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55bf08cd3c50 0x55bf08cd6070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bf08c54cb0 0x55bf08c58580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf08c4a410 0x55bf08c5d550 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 1 --2- 172.21.15.7:0/2449184430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf08c49af0 0x55bf08c5d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f2738ff9640 0 client.25016 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 1 -- 172.21.15.7:0/2449184430 >> 172.21.15.7:0/2449184430 conn(0x55bf08c46060 msgr2=0x55bf08c533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 1 -- 172.21.15.7:0/2449184430 shutdown_connections 2024-09-10T10:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 1 -- 172.21.15.7:0/2449184430 wait complete. 2024-09-10T10:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f275a06a740 20 client.25016 trim_cache size 0 max 16384 2024-09-10T10:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:47:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:47:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f27572d0640 1 -- 172.21.15.7:0/2123688227 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f273c0a2ec0 msgr2=0x7f273c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f27572d0640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f273c0a2ec0 0x7f273c0a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f274804a000 tx=0x7f274804b000 comp rx=0 tx=0).stop 2024-09-10T10:47:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f27572d0640 1 -- 172.21.15.7:0/2123688227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750101ae0 msgr2=0x7f2750198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f27572d0640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750101ae0 0x7f2750198350 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f274c00e9f0 tx=0x7f274c00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f27572d0640 1 -- 172.21.15.7:0/2123688227 shutdown_connections 2024-09-10T10:47:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f27572d0640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f273c0a2ec0 0x7f273c0a5380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f27572d0640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2750107b80 0x7f275019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f27572d0640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2750102480 0x7f2750198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.161+0000 7f27572d0640 1 --2- 172.21.15.7:0/2123688227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750101ae0 0x7f2750198350 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.162+0000 7f27572d0640 1 -- 172.21.15.7:0/2123688227 >> 172.21.15.7:0/2123688227 conn(0x7f27500fd7e0 msgr2=0x7f27500fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.162+0000 7f27572d0640 1 -- 172.21.15.7:0/2123688227 shutdown_connections 2024-09-10T10:47:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:04.162+0000 7f27572d0640 1 -- 172.21.15.7:0/2123688227 wait complete. 2024-09-10T10:47:04.502 INFO:tasks.workunit.client.0.smithi007.stdout: 1 14466 24.28 MB/sec warmup 4 sec latency 46.832 ms 2024-09-10T10:47:05.502 INFO:tasks.workunit.client.0.smithi007.stdout: 1 17971 23.25 MB/sec warmup 5 sec latency 16.486 ms 2024-09-10T10:47:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:05 smithi103 ceph-mon[35160]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:47:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:05 smithi007 ceph-mon[30873]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:47:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:05 smithi133 ceph-mon[35282]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-10T10:47:06.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 21550 22.57 MB/sec warmup 6 sec latency 20.518 ms 2024-09-10T10:47:07.502 INFO:tasks.workunit.client.0.smithi007.stdout: 1 24993 21.93 MB/sec warmup 7 sec latency 12.998 ms 2024-09-10T10:47:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:07 smithi103 ceph-mon[35160]: pgmap v39: 97 pgs: 97 active+clean; 9.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 820 KiB/s wr, 195 op/s 2024-09-10T10:47:07.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:07 smithi007 ceph-mon[30873]: pgmap v39: 97 pgs: 97 active+clean; 9.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 820 KiB/s wr, 195 op/s 2024-09-10T10:47:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:07 smithi133 ceph-mon[35282]: pgmap v39: 97 pgs: 97 active+clean; 9.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 820 KiB/s wr, 195 op/s 2024-09-10T10:47:08.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 28361 21.53 MB/sec warmup 8 sec latency 24.010 ms 2024-09-10T10:47:09.502 INFO:tasks.workunit.client.0.smithi007.stdout: 1 31793 20.94 MB/sec warmup 9 sec latency 11.925 ms 2024-09-10T10:47:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:09 smithi103 ceph-mon[35160]: pgmap v40: 97 pgs: 97 active+clean; 9.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 819 KiB/s wr, 192 op/s 2024-09-10T10:47:09.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:09 smithi007 ceph-mon[30873]: pgmap v40: 97 pgs: 97 active+clean; 9.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 819 KiB/s wr, 192 op/s 2024-09-10T10:47:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:09 smithi133 ceph-mon[35282]: pgmap v40: 97 pgs: 97 active+clean; 9.3 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 819 KiB/s wr, 192 op/s 2024-09-10T10:47:10.502 INFO:tasks.workunit.client.0.smithi007.stdout: 1 35506 21.06 MB/sec warmup 10 sec latency 12.976 ms 2024-09-10T10:47:11.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 39019 20.87 MB/sec warmup 11 sec latency 13.532 ms 2024-09-10T10:47:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:11 smithi103 ceph-mon[35160]: pgmap v41: 97 pgs: 97 active+clean; 16 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.6 MiB/s wr, 312 op/s 2024-09-10T10:47:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:11 smithi007 ceph-mon[30873]: pgmap v41: 97 pgs: 97 active+clean; 16 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.6 MiB/s wr, 312 op/s 2024-09-10T10:47:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:11 smithi133 ceph-mon[35282]: pgmap v41: 97 pgs: 97 active+clean; 16 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.6 MiB/s wr, 312 op/s 2024-09-10T10:47:12.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 42077 20.33 MB/sec warmup 12 sec latency 12.583 ms 2024-09-10T10:47:13.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 45088 19.86 MB/sec warmup 13 sec latency 14.766 ms 2024-09-10T10:47:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:13 smithi103 ceph-mon[35160]: pgmap v42: 97 pgs: 97 active+clean; 17 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-09-10T10:47:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:13 smithi007 ceph-mon[30873]: pgmap v42: 97 pgs: 97 active+clean; 17 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-09-10T10:47:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:13 smithi133 ceph-mon[35282]: pgmap v42: 97 pgs: 97 active+clean; 17 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-09-10T10:47:14.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 48530 19.76 MB/sec warmup 14 sec latency 11.499 ms 2024-09-10T10:47:15.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 51382 19.60 MB/sec warmup 15 sec latency 18.295 ms 2024-09-10T10:47:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:15 smithi103 ceph-mon[35160]: pgmap v43: 97 pgs: 97 active+clean; 17 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-09-10T10:47:15.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:15 smithi007 ceph-mon[30873]: pgmap v43: 97 pgs: 97 active+clean; 17 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-09-10T10:47:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:15 smithi133 ceph-mon[35282]: pgmap v43: 97 pgs: 97 active+clean; 17 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-09-10T10:47:16.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 54621 19.55 MB/sec warmup 16 sec latency 11.500 ms 2024-09-10T10:47:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.266+0000 7fbbab58d640 1 -- 172.21.15.7:0/3658257092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbba41009b0 msgr2=0x7fbba4100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.266+0000 7fbbab58d640 1 --2- 172.21.15.7:0/3658257092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbba41009b0 0x7fbba4100e10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009930 tx=0x7fbb9802a420 comp rx=0 tx=0).stop 2024-09-10T10:47:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.266+0000 7fbbab58d640 1 -- 172.21.15.7:0/3658257092 shutdown_connections 2024-09-10T10:47:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.266+0000 7fbbab58d640 1 --2- 172.21.15.7:0/3658257092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbba4101350 0x7fbba410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.266+0000 7fbbab58d640 1 --2- 172.21.15.7:0/3658257092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbba41009b0 0x7fbba4100e10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.266+0000 7fbbab58d640 1 --2- 172.21.15.7:0/3658257092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba41069c0 0x7fbba4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.266+0000 7fbbab58d640 1 -- 172.21.15.7:0/3658257092 >> 172.21.15.7:0/3658257092 conn(0x7fbba4077990 msgr2=0x7fbba40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.266+0000 7fbbab58d640 1 -- 172.21.15.7:0/3658257092 shutdown_connections 2024-09-10T10:47:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.266+0000 7fbbab58d640 1 -- 172.21.15.7:0/3658257092 wait complete. 2024-09-10T10:47:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbbab58d640 1 Processor -- start 2024-09-10T10:47:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbbab58d640 1 -- start start 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbbab58d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbba41009b0 0x7fbba419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbbab58d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbba4101350 0x7fbba419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbbab58d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba41069c0 0x7fbba419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbbab58d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbba4110e00 con 0x7fbba41009b0 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbbab58d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbba4110c80 con 0x7fbba4101350 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbbab58d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbba4110f80 con 0x7fbba41069c0 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbba9329640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbba41009b0 0x7fbba419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbba9329640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbba41009b0 0x7fbba419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58346/0 (socket says 172.21.15.7:58346) 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbba9329640 1 -- 172.21.15.7:0/3019596832 learned_addr learned my addr 172.21.15.7:0/3019596832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbba8b28640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbba4101350 0x7fbba419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbba9b2a640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba41069c0 0x7fbba419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbba9329640 1 -- 172.21.15.7:0/3019596832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba41069c0 msgr2=0x7fbba419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbba9329640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba41069c0 0x7fbba419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbba9329640 1 -- 172.21.15.7:0/3019596832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbba4101350 msgr2=0x7fbba419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbba9329640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbba4101350 0x7fbba419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.267+0000 7fbba9329640 1 -- 172.21.15.7:0/3019596832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba419f5e0 con 0x7fbba41009b0 2024-09-10T10:47:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.268+0000 7fbba9329640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbba41009b0 0x7fbba419a5f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400eb90 tx=0x7fbb9400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.268+0000 7fbba27fc640 1 -- 172.21.15.7:0/3019596832 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94004d20 con 0x7fbba41009b0 2024-09-10T10:47:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.268+0000 7fbba27fc640 1 -- 172.21.15.7:0/3019596832 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb9400ee90 con 0x7fbba41009b0 2024-09-10T10:47:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.268+0000 7fbbab58d640 1 -- 172.21.15.7:0/3019596832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba419f8d0 con 0x7fbba41009b0 2024-09-10T10:47:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.268+0000 7fbba27fc640 1 -- 172.21.15.7:0/3019596832 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94010830 con 0x7fbba41009b0 2024-09-10T10:47:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.268+0000 7fbbab58d640 1 -- 172.21.15.7:0/3019596832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba41a7160 con 0x7fbba41009b0 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.270+0000 7fbba27fc640 1 -- 172.21.15.7:0/3019596832 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 24) ==== 138065+0+0 (secure 0 0 0) 0x7fbb94005020 con 0x7fbba41009b0 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbbae404740 1 Processor -- start 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbbae404740 1 -- start start 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbbae404740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc79af0 0x56524cc79ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbbae404740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 0x56524cc84770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbbae404740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc84cb0 0x56524cc870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbbae404740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56524cbde2f0 con 0x56524cc79af0 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbbae404740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56524cb86270 con 0x56524cc7a410 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbbae404740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56524cbda2b0 con 0x56524cc84cb0 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba9b2a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc79af0 0x56524cc79ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba8b28640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc84cb0 0x56524cc870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba8b28640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc84cb0 0x56524cc870a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:42722/0 (socket says 172.21.15.7:42722) 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba9b2a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc79af0 0x56524cc79ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58352/0 (socket says 172.21.15.7:58352) 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba8b28640 1 -- 172.21.15.7:0/3648778317 learned_addr learned my addr 172.21.15.7:0/3648778317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba9329640 1 --2- 172.21.15.7:0/3648778317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 0x56524cc84770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba8b28640 1 -- 172.21.15.7:0/3648778317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 msgr2=0x56524cc84770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba8b28640 1 --2- 172.21.15.7:0/3648778317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 0x56524cc84770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba8b28640 1 -- 172.21.15.7:0/3648778317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc79af0 msgr2=0x56524cc79ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba8b28640 1 --2- 172.21.15.7:0/3648778317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc79af0 0x56524cc79ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba8b28640 1 -- 172.21.15.7:0/3648778317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524cb85340 con 0x56524cc84cb0 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba9329640 1 --2- 172.21.15.7:0/3648778317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 0x56524cc84770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba27fc640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb8c0a2f40 0x7fbb8c0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.271+0000 7fbba9b2a640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb8c0a2f40 0x7fbb8c0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.272+0000 7fbba8b28640 1 --2- 172.21.15.7:0/3648778317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc84cb0 0x56524cc870a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009930 tx=0x7fbb98005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b06e8b45915890 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.272+0000 7fbba27fc640 1 -- 172.21.15.7:0/3019596832 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fbb940cfc50 con 0x7fbba41009b0 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.272+0000 7fbb8b7fe640 1 -- 172.21.15.7:0/3648778317 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98040070 con 0x56524cc84cb0 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.272+0000 7fbb8b7fe640 1 -- 172.21.15.7:0/3648778317 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb980319f0 con 0x56524cc84cb0 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.272+0000 7fbb8b7fe640 1 -- 172.21.15.7:0/3648778317 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98035040 con 0x56524cc84cb0 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 -- 172.21.15.7:0/3648778317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc84cb0 msgr2=0x56524cc870a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 --2- 172.21.15.7:0/3648778317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc84cb0 0x56524cc870a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009930 tx=0x7fbb98005e80 comp rx=0 tx=0).stop 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbba9b2a640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb8c0a2f40 0x7fbb8c0a5400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c005fd0 tx=0x7fbb9c005f60 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 -- 172.21.15.7:0/3648778317 shutdown_connections 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 --2- 172.21.15.7:0/3648778317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc84cb0 0x56524cc870a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 --2- 172.21.15.7:0/3648778317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 0x56524cc84770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 --2- 172.21.15.7:0/3648778317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc79af0 0x56524cc79ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 -- 172.21.15.7:0/3648778317 >> 172.21.15.7:0/3648778317 conn(0x56524cc76060 msgr2=0x56524cc76440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 -- 172.21.15.7:0/3648778317 shutdown_connections 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 -- 172.21.15.7:0/3648778317 wait complete. 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 -- start start 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.273+0000 7fbbae404740 1 Processor -- start 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbbae404740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc79af0 0x56524cc81af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbbae404740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc7a410 0x56524cc82030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbbae404740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc82570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbbae404740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56524cc87880 con 0x56524cc84cb0 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbbae404740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56524cb67290 con 0x56524cc79af0 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbbae404740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56524cc87a00 con 0x56524cc7a410 2024-09-10T10:47:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9329640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc79af0 0x56524cc81af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbbae404740 10 client.0 _ll_register_callbacks cb 0x56524cb98140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9329640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc79af0 0x56524cc81af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:59540/0 (socket says 172.21.15.7:59540) 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba8b28640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc7a410 0x56524cc82030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9329640 1 -- 172.21.15.7:0/3796261934 learned_addr learned my addr 172.21.15.7:0/3796261934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9b2a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc82570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9329640 1 -- 172.21.15.7:0/3796261934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc7a410 msgr2=0x56524cc82030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9329640 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc7a410 0x56524cc82030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9329640 1 -- 172.21.15.7:0/3796261934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 msgr2=0x56524cc82570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9329640 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc82570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9329640 1 -- 172.21.15.7:0/3796261934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524c8f1210 con 0x56524cc79af0 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9b2a640 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc82570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbba9329640 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc79af0 0x56524cc81af0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb9409beb0 tx=0x7fbb9409bdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f925e9656b68398a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbb88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.274+0000 7fbb88ff9640 1 -- 172.21.15.7:0/3796261934 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb940b7060 con 0x56524cc79af0 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbb88ff9640 1 -- 172.21.15.7:0/3796261934 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fbb940a4700 con 0x56524cc79af0 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbbae404740 1 -- 172.21.15.7:0/3796261934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb980319f0 con 0x56524cc79af0 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbb88ff9640 1 -- 172.21.15.7:0/3796261934 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb9409bf20 con 0x56524cc79af0 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbbae404740 10 client.25022 fetch_fsmap learned FSMap version 25 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbbae404740 1 -- 172.21.15.7:0/3796261934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56524c9bd620 con 0x56524cc79af0 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbb88ff9640 1 -- 172.21.15.7:0/3796261934 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb980319f0 con 0x56524cc79af0 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbb88ff9640 1 -- 172.21.15.7:0/3796261934 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb9409f030 con 0x56524cc79af0 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbbae404740 10 client.25022 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:47:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbbae404740 10 client.25022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbbae404740 20 client.25022 populate_metadata read hostname 'smithi007' 2024-09-10T10:47:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbbae404740 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56524cc94a70 0x56524cc96ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbbae404740 4 client.25022 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbbae404740 1 -- 172.21.15.7:0/3796261934 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56524cbeba20 con 0x56524cc94a70 2024-09-10T10:47:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.275+0000 7fbba8b28640 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56524cc94a70 0x56524cc96ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.276+0000 7fbb88ff9640 10 client.25022 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.276+0000 7fbba8b28640 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56524cc94a70 0x56524cc96ea0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e99cd74bc985a016 server_cookie=37a88949d4f949c8 in_seq=0 out_seq=0 2024-09-10T10:47:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbb88ff9640 1 -- 172.21.15.7:0/3796261934 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56524cbeba20 con 0x56524cc94a70 2024-09-10T10:47:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbb88ff9640 10 client.25022 handle_command_reply: tid=0 2024-09-10T10:47:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 client.25022 shutdown 2024-09-10T10:47:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 -- 172.21.15.7:0/3796261934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc79af0 msgr2=0x56524cc81af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc79af0 0x56524cc81af0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb9409beb0 tx=0x7fbb9409bdb0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 -- 172.21.15.7:0/3796261934 shutdown_connections 2024-09-10T10:47:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56524cc94a70 0x56524cc96ea0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc82570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc7a410 0x56524cc82030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 --2- 172.21.15.7:0/3796261934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc79af0 0x56524cc81af0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbb88ff9640 0 client.25022 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 -- 172.21.15.7:0/3796261934 >> 172.21.15.7:0/3796261934 conn(0x56524cc76060 msgr2=0x56524cc7bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 -- 172.21.15.7:0/3796261934 shutdown_connections 2024-09-10T10:47:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 1 -- 172.21.15.7:0/3796261934 wait complete. 2024-09-10T10:47:17.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.277+0000 7fbbae404740 20 client.25022 trim_cache size 0 max 16384 2024-09-10T10:47:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbbae404740 1 Processor -- start 2024-09-10T10:47:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbbae404740 1 -- start start 2024-09-10T10:47:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbbae404740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 0x56524cc823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbbae404740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc7a410 0x56524cc9c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbbae404740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc84cb0 0x56524cc9ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbbae404740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41b2260 con 0x56524cc7a410 2024-09-10T10:47:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbbae404740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41afb30 con 0x56524cc84cb0 2024-09-10T10:47:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbbae404740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41ad980 con 0x56524cc79af0 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbba8b28640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc7a410 0x56524cc9c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbba8b28640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc7a410 0x56524cc9c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58362/0 (socket says 172.21.15.7:58362) 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbba9329640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 0x56524cc823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbba8b28640 1 -- 172.21.15.7:0/3767521007 learned_addr learned my addr 172.21.15.7:0/3767521007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbba9b2a640 1 --2- 172.21.15.7:0/3767521007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc84cb0 0x56524cc9ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbba8b28640 1 -- 172.21.15.7:0/3767521007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 msgr2=0x56524cc823a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.293+0000 7fbba8b28640 1 --2- 172.21.15.7:0/3767521007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 0x56524cc823a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.294+0000 7fbba8b28640 1 -- 172.21.15.7:0/3767521007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc84cb0 msgr2=0x56524cc9ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.294+0000 7fbba8b28640 1 --2- 172.21.15.7:0/3767521007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc84cb0 0x56524cc9ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.294+0000 7fbba8b28640 1 -- 172.21.15.7:0/3767521007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524cc75e90 con 0x56524cc7a410 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.294+0000 7fbba9329640 1 --2- 172.21.15.7:0/3767521007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 0x56524cc823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.294+0000 7fbba8b28640 1 --2- 172.21.15.7:0/3767521007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc7a410 0x56524cc9c7b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbb98031550 tx=0x7fbb9802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3be0ddd7615c48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.294+0000 7fbb8affd640 1 -- 172.21.15.7:0/3767521007 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98040070 con 0x56524cc7a410 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.294+0000 7fbb8affd640 1 -- 172.21.15.7:0/3767521007 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb98005030 con 0x56524cc7a410 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.294+0000 7fbb8affd640 1 -- 172.21.15.7:0/3767521007 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98035040 con 0x56524cc7a410 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 -- 172.21.15.7:0/3767521007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc7a410 msgr2=0x56524cc9c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 --2- 172.21.15.7:0/3767521007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc7a410 0x56524cc9c7b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbb98031550 tx=0x7fbb9802a420 comp rx=0 tx=0).stop 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 -- 172.21.15.7:0/3767521007 shutdown_connections 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 --2- 172.21.15.7:0/3767521007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc84cb0 0x56524cc9ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 --2- 172.21.15.7:0/3767521007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc7a410 0x56524cc9c7b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 --2- 172.21.15.7:0/3767521007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 0x56524cc823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 -- 172.21.15.7:0/3767521007 >> 172.21.15.7:0/3767521007 conn(0x56524cc76060 msgr2=0x56524cc83b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 -- 172.21.15.7:0/3767521007 shutdown_connections 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 -- 172.21.15.7:0/3767521007 wait complete. 2024-09-10T10:47:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 -- start start 2024-09-10T10:47:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 Processor -- start 2024-09-10T10:47:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 0x56524cc8d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 0x56524cc88690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc88bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.295+0000 7fbbae404740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41adff0 con 0x56524cc84cb0 2024-09-10T10:47:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbbae404740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41af460 con 0x56524cc7a410 2024-09-10T10:47:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbbae404740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbba41ae6c0 con 0x56524cc79af0 2024-09-10T10:47:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbbae404740 10 client.0 _ll_register_callbacks cb 0x7fbb94009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba9b2a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc88bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba9329640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 0x56524cc8d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba8b28640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 0x56524cc88690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba9329640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 0x56524cc8d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:42752/0 (socket says 172.21.15.7:42752) 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba9329640 1 -- 172.21.15.7:0/2751153933 learned_addr learned my addr 172.21.15.7:0/2751153933 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba9b2a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc88bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58376/0 (socket says 172.21.15.7:58376) 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba8b28640 1 -- 172.21.15.7:0/2751153933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 msgr2=0x56524cc8d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba8b28640 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 0x56524cc8d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba8b28640 1 -- 172.21.15.7:0/2751153933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 msgr2=0x56524cc88bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba8b28640 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc88bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba8b28640 1 -- 172.21.15.7:0/2751153933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524cc79920 con 0x56524cc7a410 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba9b2a640 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc88bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbba8b28640 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 0x56524cc88690 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb9802a940 tx=0x7fbb980058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ae7f4de595bf740 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbb88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:47:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbb88ff9640 1 -- 172.21.15.7:0/2751153933 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98035070 con 0x56524cc7a410 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbb88ff9640 1 -- 172.21.15.7:0/2751153933 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fbb98005470 con 0x56524cc7a410 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.296+0000 7fbbae404740 1 -- 172.21.15.7:0/2751153933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb98005030 con 0x56524cc7a410 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbb88ff9640 1 -- 172.21.15.7:0/2751153933 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb98040040 con 0x56524cc7a410 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbbae404740 10 client.25028 fetch_fsmap learned FSMap version 25 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbbae404740 1 -- 172.21.15.7:0/2751153933 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56524ccff560 con 0x56524cc7a410 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbb88ff9640 1 -- 172.21.15.7:0/2751153933 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb98005030 con 0x56524cc7a410 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbb88ff9640 1 -- 172.21.15.7:0/2751153933 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbb98061980 con 0x56524cc7a410 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbbae404740 10 client.25028 fetch_fsmap finished waiting for FSMap version 25 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbbae404740 10 client.25028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbbae404740 20 client.25028 populate_metadata read hostname 'smithi007' 2024-09-10T10:47:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbbae404740 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56524ccff830 0x56524cd01c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbbae404740 4 client.25028 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:47:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbbae404740 1 -- 172.21.15.7:0/2751153933 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56524cbe5d70 con 0x56524ccff830 2024-09-10T10:47:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.297+0000 7fbba9329640 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56524ccff830 0x56524cd01c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.298+0000 7fbb88ff9640 10 client.25028 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.298+0000 7fbba9329640 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56524ccff830 0x56524cd01c20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=140cbb76cf9c6b9e server_cookie=58c609a600301da4 in_seq=0 out_seq=0 2024-09-10T10:47:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.413+0000 7fbb88ff9640 1 -- 172.21.15.7:0/2751153933 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56524cbe5d70 con 0x56524ccff830 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.413+0000 7fbb88ff9640 10 client.25028 handle_command_reply: tid=0 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.413+0000 7fbbae404740 1 client.25028 shutdown 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 1 -- 172.21.15.7:0/2751153933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 msgr2=0x56524cc88690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 0x56524cc88690 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb9802a940 tx=0x7fbb980058d0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 1 -- 172.21.15.7:0/2751153933 shutdown_connections 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56524ccff830 0x56524cd01c20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56524cc84cb0 0x56524cc88bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbb88ff9640 0 client.25028 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56524cc7a410 0x56524cc88690 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 1 --2- 172.21.15.7:0/2751153933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56524cc79af0 0x56524cc8d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 1 -- 172.21.15.7:0/2751153933 >> 172.21.15.7:0/2751153933 conn(0x56524cc76060 msgr2=0x56524cc7b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 1 -- 172.21.15.7:0/2751153933 shutdown_connections 2024-09-10T10:47:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 1 -- 172.21.15.7:0/2751153933 wait complete. 2024-09-10T10:47:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.414+0000 7fbbae404740 20 client.25028 trim_cache size 0 max 16384 2024-09-10T10:47:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:47:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:47:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:47:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:47:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 -- 172.21.15.7:0/3019596832 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb8c0a2f40 msgr2=0x7fbb8c0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb8c0a2f40 0x7fbb8c0a5400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c005fd0 tx=0x7fbb9c005f60 comp rx=0 tx=0).stop 2024-09-10T10:47:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 -- 172.21.15.7:0/3019596832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbba41009b0 msgr2=0x7fbba419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbba41009b0 0x7fbba419a5f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400eb90 tx=0x7fbb9400c6a0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 -- 172.21.15.7:0/3019596832 shutdown_connections 2024-09-10T10:47:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba41069c0 0x7fbba419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbba4101350 0x7fbba419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb8c0a2f40 0x7fbb8c0a5400 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 --2- 172.21.15.7:0/3019596832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbba41009b0 0x7fbba419a5f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 -- 172.21.15.7:0/3019596832 >> 172.21.15.7:0/3019596832 conn(0x7fbba4077990 msgr2=0x7fbba41049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 -- 172.21.15.7:0/3019596832 shutdown_connections 2024-09-10T10:47:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:17.415+0000 7fbbab58d640 1 -- 172.21.15.7:0/3019596832 wait complete. 2024-09-10T10:47:17.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 57387 19.23 MB/sec warmup 17 sec latency 64.610 ms 2024-09-10T10:47:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:17 smithi103 ceph-mon[35160]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 412 op/s 2024-09-10T10:47:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:17 smithi007 ceph-mon[30873]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 412 op/s 2024-09-10T10:47:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:17 smithi133 ceph-mon[35282]: pgmap v44: 97 pgs: 97 active+clean; 25 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 412 op/s 2024-09-10T10:47:18.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 60759 19.22 MB/sec warmup 18 sec latency 13.517 ms 2024-09-10T10:47:19.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 63854 19.13 MB/sec warmup 19 sec latency 99.958 ms 2024-09-10T10:47:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:19 smithi103 ceph-mon[35160]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-10T10:47:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:19 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:47:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:19 smithi007 ceph-mon[30873]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-10T10:47:19.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:19 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:47:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:19 smithi133 ceph-mon[35282]: pgmap v45: 97 pgs: 97 active+clean; 25 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-10T10:47:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:19 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:47:20.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 63854 18.17 MB/sec warmup 20 sec latency 1100.067 ms 2024-09-10T10:47:21.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 63854 17.31 MB/sec warmup 21 sec latency 2100.170 ms 2024-09-10T10:47:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:21 smithi103 ceph-mon[35160]: pgmap v46: 97 pgs: 97 active+clean; 15 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 436 op/s 2024-09-10T10:47:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:21 smithi007 ceph-mon[30873]: pgmap v46: 97 pgs: 97 active+clean; 15 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 436 op/s 2024-09-10T10:47:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:21 smithi133 ceph-mon[35282]: pgmap v46: 97 pgs: 97 active+clean; 15 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 436 op/s 2024-09-10T10:47:22.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 63854 16.52 MB/sec warmup 22 sec latency 3100.246 ms 2024-09-10T10:47:23.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 64030 15.82 MB/sec warmup 23 sec latency 3422.136 ms 2024-09-10T10:47:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:23 smithi103 ceph-mon[35160]: pgmap v47: 97 pgs: 97 active+clean; 15 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 316 op/s 2024-09-10T10:47:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:23 smithi007 ceph-mon[30873]: pgmap v47: 97 pgs: 97 active+clean; 15 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 316 op/s 2024-09-10T10:47:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:23 smithi133 ceph-mon[35282]: pgmap v47: 97 pgs: 97 active+clean; 15 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 316 op/s 2024-09-10T10:47:24.503 INFO:tasks.workunit.client.0.smithi007.stdout: 1 64467 15.22 MB/sec warmup 24 sec latency 955.513 ms 2024-09-10T10:47:25.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 67658 15.36 MB/sec warmup 25 sec latency 14.597 ms 2024-09-10T10:47:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:25 smithi103 ceph-mon[35160]: pgmap v48: 97 pgs: 97 active+clean; 15 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 315 op/s 2024-09-10T10:47:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:25 smithi007 ceph-mon[30873]: pgmap v48: 97 pgs: 97 active+clean; 15 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 315 op/s 2024-09-10T10:47:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:25 smithi133 ceph-mon[35282]: pgmap v48: 97 pgs: 97 active+clean; 15 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 1.6 MiB/s wr, 315 op/s 2024-09-10T10:47:26.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 69868 15.06 MB/sec warmup 26 sec latency 92.107 ms 2024-09-10T10:47:27.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 71025 14.89 MB/sec warmup 27 sec latency 241.636 ms 2024-09-10T10:47:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:27 smithi103 ceph-mon[35160]: pgmap v49: 97 pgs: 97 active+clean; 16 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.7 MiB/s wr, 338 op/s 2024-09-10T10:47:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:27 smithi103 ceph-mon[35160]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-10T10:47:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:27 smithi007 ceph-mon[30873]: pgmap v49: 97 pgs: 97 active+clean; 16 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.7 MiB/s wr, 338 op/s 2024-09-10T10:47:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:27 smithi007 ceph-mon[30873]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-10T10:47:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:27 smithi133 ceph-mon[35282]: pgmap v49: 97 pgs: 97 active+clean; 16 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 1.7 MiB/s wr, 338 op/s 2024-09-10T10:47:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:27 smithi133 ceph-mon[35282]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-10T10:47:28.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 72854 14.60 MB/sec warmup 28 sec latency 49.840 ms 2024-09-10T10:47:29.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 75140 14.57 MB/sec warmup 29 sec latency 13.890 ms 2024-09-10T10:47:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:29 smithi103 ceph-mon[35160]: pgmap v50: 97 pgs: 97 active+clean; 16 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 880 KiB/s wr, 237 op/s 2024-09-10T10:47:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:29 smithi007 ceph-mon[30873]: pgmap v50: 97 pgs: 97 active+clean; 16 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 880 KiB/s wr, 237 op/s 2024-09-10T10:47:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:29 smithi133 ceph-mon[35282]: pgmap v50: 97 pgs: 97 active+clean; 16 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 880 KiB/s wr, 237 op/s 2024-09-10T10:47:30.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 78322 14.71 MB/sec warmup 30 sec latency 11.149 ms 2024-09-10T10:47:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.509+0000 7f633de82640 1 -- 172.21.15.7:0/1808149197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6338101ad0 msgr2=0x7f6338101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.509+0000 7f633de82640 1 --2- 172.21.15.7:0/1808149197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6338101ad0 0x7f6338101f30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f632c009930 tx=0x7f632c02a420 comp rx=0 tx=0).stop 2024-09-10T10:47:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.509+0000 7f633de82640 1 -- 172.21.15.7:0/1808149197 shutdown_connections 2024-09-10T10:47:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.509+0000 7f633de82640 1 --2- 172.21.15.7:0/1808149197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338102470 0x7f633810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.509+0000 7f633de82640 1 --2- 172.21.15.7:0/1808149197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6338101ad0 0x7f6338101f30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.509+0000 7f633de82640 1 --2- 172.21.15.7:0/1808149197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6338107b70 0x7f6338107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.509+0000 7f633de82640 1 -- 172.21.15.7:0/1808149197 >> 172.21.15.7:0/1808149197 conn(0x7f63380fd7d0 msgr2=0x7f63380ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.509+0000 7f633de82640 1 -- 172.21.15.7:0/1808149197 shutdown_connections 2024-09-10T10:47:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.509+0000 7f633de82640 1 -- 172.21.15.7:0/1808149197 wait complete. 2024-09-10T10:47:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.509+0000 7f633de82640 1 Processor -- start 2024-09-10T10:47:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f633de82640 1 -- start start 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f633de82640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6338101ad0 0x7f63380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f633de82640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338102470 0x7f6338074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f633de82640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6338107b70 0x7f6338074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f633de82640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6338111f20 con 0x7f6338107b70 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f633de82640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6338111da0 con 0x7f6338101ad0 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f633de82640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63381120a0 con 0x7f6338102470 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f63377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6338101ad0 0x7f63380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f6337fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6338107b70 0x7f6338074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f63377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6338101ad0 0x7f63380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:43798/0 (socket says 172.21.15.7:43798) 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f63377fe640 1 -- 172.21.15.7:0/519252477 learned_addr learned my addr 172.21.15.7:0/519252477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.510+0000 7f6336ffd640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338102470 0x7f6338074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f63377fe640 1 -- 172.21.15.7:0/519252477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338102470 msgr2=0x7f6338074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f63377fe640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338102470 0x7f6338074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f63377fe640 1 -- 172.21.15.7:0/519252477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6338107b70 msgr2=0x7f6338074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f63377fe640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6338107b70 0x7f6338074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f63377fe640 1 -- 172.21.15.7:0/519252477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63381a67d0 con 0x7f6338101ad0 2024-09-10T10:47:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f63377fe640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6338101ad0 0x7f63380744f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f632800e9f0 tx=0x7f632800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f6334ff9640 1 -- 172.21.15.7:0/519252477 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632800cd80 con 0x7f6338101ad0 2024-09-10T10:47:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f633de82640 1 -- 172.21.15.7:0/519252477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63381a6ac0 con 0x7f6338101ad0 2024-09-10T10:47:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f6334ff9640 1 -- 172.21.15.7:0/519252477 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6328010910 con 0x7f6338101ad0 2024-09-10T10:47:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f6334ff9640 1 -- 172.21.15.7:0/519252477 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280188f0 con 0x7f6338101ad0 2024-09-10T10:47:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.511+0000 7f633de82640 1 -- 172.21.15.7:0/519252477 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63381a7030 con 0x7f6338101ad0 2024-09-10T10:47:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.513+0000 7f6334ff9640 1 -- 172.21.15.7:0/519252477 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6328010ac0 con 0x7f6338101ad0 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6340c21740 1 Processor -- start 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6340c21740 1 -- start start 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6340c21740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d11ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6340c21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 0x5640b4d1c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6340c21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d1f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6340c21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5640b4c762f0 con 0x5640b4d1ccb0 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6340c21740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5640b4c1e270 con 0x5640b4d11af0 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6340c21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5640b4c722b0 con 0x5640b4d12410 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6334ff9640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63200a2da0 0x7f63200a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d1f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f63377fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 0x5640b4d1c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d1f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46740/0 (socket says 172.21.15.7:46740) 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6336ffd640 1 -- 172.21.15.7:0/2408285282 learned_addr learned my addr 172.21.15.7:0/2408285282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6337fff640 1 --2- 172.21.15.7:0/2408285282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d11ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.514+0000 7f6337fff640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63200a2da0 0x7f63200a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f6334ff9640 1 -- 172.21.15.7:0/519252477 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f63280cf790 con 0x7f6338101ad0 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f63377fe640 1 -- 172.21.15.7:0/2408285282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 msgr2=0x5640b4d11ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f63377fe640 1 --2- 172.21.15.7:0/2408285282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d11ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f63377fe640 1 -- 172.21.15.7:0/2408285282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 msgr2=0x5640b4d1f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f63377fe640 1 --2- 172.21.15.7:0/2408285282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d1f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f63377fe640 1 -- 172.21.15.7:0/2408285282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b4c1d340 con 0x5640b4d12410 2024-09-10T10:47:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f6336ffd640 1 --2- 172.21.15.7:0/2408285282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d1f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f6337fff640 1 --2- 172.21.15.7:0/2408285282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d11ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f63377fe640 1 --2- 172.21.15.7:0/2408285282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 0x5640b4d1c770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63280ccee0 tx=0x7f63280bd5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7a134195245898b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f6337fff640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63200a2da0 0x7f63200a5260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6330005c40 tx=0x7f6330009500 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f631dffb640 1 -- 172.21.15.7:0/2408285282 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280a4920 con 0x5640b4d12410 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f631dffb640 1 -- 172.21.15.7:0/2408285282 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63280cd160 con 0x5640b4d12410 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.515+0000 7f631dffb640 1 -- 172.21.15.7:0/2408285282 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328066590 con 0x5640b4d12410 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 -- 172.21.15.7:0/2408285282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 msgr2=0x5640b4d1c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 --2- 172.21.15.7:0/2408285282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 0x5640b4d1c770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63280ccee0 tx=0x7f63280bd5d0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 -- 172.21.15.7:0/2408285282 shutdown_connections 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 --2- 172.21.15.7:0/2408285282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d1f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 --2- 172.21.15.7:0/2408285282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 0x5640b4d1c770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 --2- 172.21.15.7:0/2408285282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d11ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 -- 172.21.15.7:0/2408285282 >> 172.21.15.7:0/2408285282 conn(0x5640b4d0e060 msgr2=0x5640b4d0e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 -- 172.21.15.7:0/2408285282 shutdown_connections 2024-09-10T10:47:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 -- 172.21.15.7:0/2408285282 wait complete. 2024-09-10T10:47:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 -- start start 2024-09-10T10:47:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.516+0000 7f6340c21740 1 Processor -- start 2024-09-10T10:47:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6340c21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d11af0 0x5640b4d19cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6340c21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d1a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6340c21740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d1ccb0 0x5640b4d1a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6340c21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5640b4d1f820 con 0x5640b4d12410 2024-09-10T10:47:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6340c21740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5640b4bff290 con 0x5640b4d1ccb0 2024-09-10T10:47:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6340c21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5640b4d1f9a0 con 0x5640b4d11af0 2024-09-10T10:47:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6340c21740 10 client.0 _ll_register_callbacks cb 0x5640b4c30140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d1a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d1a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46748/0 (socket says 172.21.15.7:46748) 2024-09-10T10:47:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f63377fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d11af0 0x5640b4d19cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6336ffd640 1 -- 172.21.15.7:0/1288529584 learned_addr learned my addr 172.21.15.7:0/1288529584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6337fff640 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d1ccb0 0x5640b4d1a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6336ffd640 1 -- 172.21.15.7:0/1288529584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d11af0 msgr2=0x5640b4d19cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6336ffd640 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d11af0 0x5640b4d19cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6336ffd640 1 -- 172.21.15.7:0/1288529584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d1ccb0 msgr2=0x5640b4d1a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6336ffd640 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d1ccb0 0x5640b4d1a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6336ffd640 1 -- 172.21.15.7:0/1288529584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b4989210 con 0x5640b4d12410 2024-09-10T10:47:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f63377fe640 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d11af0 0x5640b4d19cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6336ffd640 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d1a210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f632c009230 tx=0x7f632c031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee5146007a65b46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6316ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6316ffd640 1 -- 172.21.15.7:0/1288529584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c035040 con 0x5640b4d12410 2024-09-10T10:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6316ffd640 1 -- 172.21.15.7:0/1288529584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f632c004460 con 0x5640b4d12410 2024-09-10T10:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.517+0000 7f6340c21740 1 -- 172.21.15.7:0/1288529584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63280cd160 con 0x5640b4d12410 2024-09-10T10:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6316ffd640 1 -- 172.21.15.7:0/1288529584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f632c005040 con 0x5640b4d12410 2024-09-10T10:47:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6340c21740 10 client.15354 fetch_fsmap learned FSMap version 26 2024-09-10T10:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6340c21740 1 -- 172.21.15.7:0/1288529584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640b4a55620 con 0x5640b4d12410 2024-09-10T10:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6316ffd640 1 -- 172.21.15.7:0/1288529584 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63280cd160 con 0x5640b4d12410 2024-09-10T10:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6316ffd640 1 -- 172.21.15.7:0/1288529584 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f632c0035f0 con 0x5640b4d12410 2024-09-10T10:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6340c21740 10 client.15354 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T10:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6340c21740 10 client.15354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6340c21740 20 client.15354 populate_metadata read hostname 'smithi007' 2024-09-10T10:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6340c21740 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5640b4d2ccd0 0x5640b4d2f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6340c21740 4 client.15354 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f6340c21740 1 -- 172.21.15.7:0/1288529584 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640b4c83a20 con 0x5640b4d2ccd0 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.518+0000 7f63377fe640 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5640b4d2ccd0 0x5640b4d2f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.519+0000 7f63377fe640 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5640b4d2ccd0 0x5640b4d2f130 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a40805758529ed8 server_cookie=7886f9a568c59e37 in_seq=0 out_seq=0 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.519+0000 7f6316ffd640 10 client.15354 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.519+0000 7f6316ffd640 1 -- 172.21.15.7:0/1288529584 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640b4c83a20 con 0x5640b4d2ccd0 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.519+0000 7f6316ffd640 10 client.15354 handle_command_reply: tid=0 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.519+0000 7f6340c21740 1 client.15354 shutdown 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 1 -- 172.21.15.7:0/1288529584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 msgr2=0x5640b4d1a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d1a210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f632c009230 tx=0x7f632c031850 comp rx=0 tx=0).stop 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 1 -- 172.21.15.7:0/1288529584 shutdown_connections 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5640b4d2ccd0 0x5640b4d2f130 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d1ccb0 0x5640b4d1a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d1a210 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6316ffd640 0 client.15354 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 1 --2- 172.21.15.7:0/1288529584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d11af0 0x5640b4d19cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 1 -- 172.21.15.7:0/1288529584 >> 172.21.15.7:0/1288529584 conn(0x5640b4d0e060 msgr2=0x5640b4d13d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 1 -- 172.21.15.7:0/1288529584 shutdown_connections 2024-09-10T10:47:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 1 -- 172.21.15.7:0/1288529584 wait complete. 2024-09-10T10:47:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.520+0000 7f6340c21740 20 client.15354 trim_cache size 0 max 16384 2024-09-10T10:47:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6340c21740 1 Processor -- start 2024-09-10T10:47:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6340c21740 1 -- start start 2024-09-10T10:47:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6340c21740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6340c21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d0d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6340c21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d1ccb0 0x5640b4d34a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6340c21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63381b21b0 con 0x5640b4d12410 2024-09-10T10:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6340c21740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63381afa80 con 0x5640b4d11af0 2024-09-10T10:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6340c21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63381ad8d0 con 0x5640b4d1ccb0 2024-09-10T10:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d0d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d0d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46750/0 (socket says 172.21.15.7:46750) 2024-09-10T10:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f63377fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6336ffd640 1 -- 172.21.15.7:0/4123381016 learned_addr learned my addr 172.21.15.7:0/4123381016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.535+0000 7f6337fff640 1 --2- 172.21.15.7:0/4123381016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d1ccb0 0x5640b4d34a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.536+0000 7f63377fe640 1 -- 172.21.15.7:0/4123381016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d1ccb0 msgr2=0x5640b4d34a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.536+0000 7f63377fe640 1 --2- 172.21.15.7:0/4123381016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d1ccb0 0x5640b4d34a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.536+0000 7f63377fe640 1 -- 172.21.15.7:0/4123381016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 msgr2=0x5640b4d0d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.536+0000 7f63377fe640 1 --2- 172.21.15.7:0/4123381016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d0d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.536+0000 7f63377fe640 1 -- 172.21.15.7:0/4123381016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b4d296a0 con 0x5640b4d11af0 2024-09-10T10:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.536+0000 7f63377fe640 1 --2- 172.21.15.7:0/4123381016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d1a580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5640b4d12ab0 tx=0x7f63280bbee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac83dcc8d81b6b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.536+0000 7f631d7fa640 1 -- 172.21.15.7:0/4123381016 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280a4490 con 0x5640b4d11af0 2024-09-10T10:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.536+0000 7f631d7fa640 1 -- 172.21.15.7:0/4123381016 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f632809a060 con 0x5640b4d11af0 2024-09-10T10:47:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.536+0000 7f631d7fa640 1 -- 172.21.15.7:0/4123381016 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280bd3c0 con 0x5640b4d11af0 2024-09-10T10:47:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.537+0000 7f6340c21740 1 -- 172.21.15.7:0/4123381016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 msgr2=0x5640b4d1a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.537+0000 7f6340c21740 1 --2- 172.21.15.7:0/4123381016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d1a580 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5640b4d12ab0 tx=0x7f63280bbee0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.537+0000 7f6340c21740 1 -- 172.21.15.7:0/4123381016 shutdown_connections 2024-09-10T10:47:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.537+0000 7f6340c21740 1 --2- 172.21.15.7:0/4123381016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d1ccb0 0x5640b4d34a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.537+0000 7f6340c21740 1 --2- 172.21.15.7:0/4123381016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d12410 0x5640b4d0d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.537+0000 7f6340c21740 1 --2- 172.21.15.7:0/4123381016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d1a580 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.537+0000 7f6340c21740 1 -- 172.21.15.7:0/4123381016 >> 172.21.15.7:0/4123381016 conn(0x5640b4d0e060 msgr2=0x5640b4d0c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.537+0000 7f6340c21740 1 -- 172.21.15.7:0/4123381016 shutdown_connections 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.537+0000 7f6340c21740 1 -- 172.21.15.7:0/4123381016 wait complete. 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.537+0000 7f6340c21740 1 -- start start 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6340c21740 1 Processor -- start 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6340c21740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d25110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6340c21740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 0x5640b4d20430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6340c21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d20970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6340c21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63381adf40 con 0x5640b4d1ccb0 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6340c21740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63381af3b0 con 0x5640b4d11af0 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6340c21740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63381ae610 con 0x5640b4d12410 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6340c21740 10 client.0 _ll_register_callbacks cb 0x7f632c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6337fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d20970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 0x5640b4d20430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6336ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 0x5640b4d20430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48518/0 (socket says 172.21.15.7:48518) 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6337fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d20970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46752/0 (socket says 172.21.15.7:46752) 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6336ffd640 1 -- 172.21.15.7:0/2325125396 learned_addr learned my addr 172.21.15.7:0/2325125396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f63377fe640 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d25110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6337fff640 1 -- 172.21.15.7:0/2325125396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 msgr2=0x5640b4d20430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6337fff640 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 0x5640b4d20430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6337fff640 1 -- 172.21.15.7:0/2325125396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 msgr2=0x5640b4d25110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6337fff640 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d25110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.538+0000 7f6337fff640 1 -- 172.21.15.7:0/2325125396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b4d20f30 con 0x5640b4d1ccb0 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6337fff640 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d20970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5640b4d18490 tx=0x7f6330004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4b1bf8e707e745 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6316ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:47:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6316ffd640 1 -- 172.21.15.7:0/2325125396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330004b20 con 0x5640b4d1ccb0 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6316ffd640 1 -- 172.21.15.7:0/2325125396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f6330004cc0 con 0x5640b4d1ccb0 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6340c21740 1 -- 172.21.15.7:0/2325125396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f632809a060 con 0x5640b4d1ccb0 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6316ffd640 1 -- 172.21.15.7:0/2325125396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6330005020 con 0x5640b4d1ccb0 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6340c21740 10 client.15360 fetch_fsmap learned FSMap version 26 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6340c21740 1 -- 172.21.15.7:0/2325125396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640b4d977b0 con 0x5640b4d1ccb0 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6316ffd640 1 -- 172.21.15.7:0/2325125396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f632809a060 con 0x5640b4d1ccb0 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6316ffd640 1 -- 172.21.15.7:0/2325125396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f633001d040 con 0x5640b4d1ccb0 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6340c21740 10 client.15360 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6340c21740 10 client.15360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6340c21740 20 client.15360 populate_metadata read hostname 'smithi007' 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6340c21740 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5640b4d2ccd0 0x5640b4d9a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6340c21740 4 client.15360 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:47:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.539+0000 7f6340c21740 1 -- 172.21.15.7:0/2325125396 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5640b4c7dd70 con 0x5640b4d2ccd0 2024-09-10T10:47:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.540+0000 7f63377fe640 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5640b4d2ccd0 0x5640b4d9a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.540+0000 7f6316ffd640 10 client.15360 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.540+0000 7f63377fe640 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5640b4d2ccd0 0x5640b4d9a050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf9f277af538c003 server_cookie=6c50ae5e188e3885 in_seq=0 out_seq=0 2024-09-10T10:47:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6316ffd640 1 -- 172.21.15.7:0/2325125396 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5640b4c7dd70 con 0x5640b4d2ccd0 2024-09-10T10:47:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6316ffd640 10 client.15360 handle_command_reply: tid=0 2024-09-10T10:47:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6340c21740 1 client.15360 shutdown 2024-09-10T10:47:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6340c21740 1 -- 172.21.15.7:0/2325125396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 msgr2=0x5640b4d20970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6340c21740 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d20970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5640b4d18490 tx=0x7f6330004a40 comp rx=0 tx=0).stop 2024-09-10T10:47:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6340c21740 1 -- 172.21.15.7:0/2325125396 shutdown_connections 2024-09-10T10:47:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6340c21740 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5640b4d2ccd0 0x5640b4d9a050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6316ffd640 0 client.15360 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6340c21740 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640b4d1ccb0 0x5640b4d20970 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6340c21740 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640b4d12410 0x5640b4d20430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6340c21740 1 --2- 172.21.15.7:0/2325125396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5640b4d11af0 0x5640b4d25110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.588+0000 7f6340c21740 1 -- 172.21.15.7:0/2325125396 >> 172.21.15.7:0/2325125396 conn(0x5640b4d0e060 msgr2=0x5640b4d0d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f6340c21740 1 -- 172.21.15.7:0/2325125396 shutdown_connections 2024-09-10T10:47:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f6340c21740 1 -- 172.21.15.7:0/2325125396 wait complete. 2024-09-10T10:47:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f6340c21740 20 client.15360 trim_cache size 0 max 16384 2024-09-10T10:47:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:47:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:47:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:47:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 -- 172.21.15.7:0/519252477 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63200a2da0 msgr2=0x7f63200a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63200a2da0 0x7f63200a5260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6330005c40 tx=0x7f6330009500 comp rx=0 tx=0).stop 2024-09-10T10:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 -- 172.21.15.7:0/519252477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6338101ad0 msgr2=0x7f63380744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6338101ad0 0x7f63380744f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f632800e9f0 tx=0x7f632800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 -- 172.21.15.7:0/519252477 shutdown_connections 2024-09-10T10:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63200a2da0 0x7f63200a5260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6338107b70 0x7f6338074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338102470 0x7f6338074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 --2- 172.21.15.7:0/519252477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6338101ad0 0x7f63380744f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 -- 172.21.15.7:0/519252477 >> 172.21.15.7:0/519252477 conn(0x7f63380fd7d0 msgr2=0x7f63380fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 -- 172.21.15.7:0/519252477 shutdown_connections 2024-09-10T10:47:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:30.589+0000 7f633de82640 1 -- 172.21.15.7:0/519252477 wait complete. 2024-09-10T10:47:31.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 80471 14.47 MB/sec warmup 31 sec latency 45.169 ms 2024-09-10T10:47:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:31 smithi103 ceph-mon[35160]: pgmap v51: 97 pgs: 97 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.6 MiB/s wr, 505 op/s 2024-09-10T10:47:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:31 smithi007 ceph-mon[30873]: pgmap v51: 97 pgs: 97 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.6 MiB/s wr, 505 op/s 2024-09-10T10:47:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:31 smithi133 ceph-mon[35282]: pgmap v51: 97 pgs: 97 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.6 MiB/s wr, 505 op/s 2024-09-10T10:47:32.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 82438 14.42 MB/sec warmup 32 sec latency 12.725 ms 2024-09-10T10:47:33.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 85435 14.53 MB/sec warmup 33 sec latency 37.133 ms 2024-09-10T10:47:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:33 smithi103 ceph-mon[35160]: pgmap v52: 97 pgs: 97 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 6.9 MiB/s wr, 291 op/s 2024-09-10T10:47:33.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:33 smithi007 ceph-mon[30873]: pgmap v52: 97 pgs: 97 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 6.9 MiB/s wr, 291 op/s 2024-09-10T10:47:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:33 smithi133 ceph-mon[35282]: pgmap v52: 97 pgs: 97 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 6.9 MiB/s wr, 291 op/s 2024-09-10T10:47:34.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 87952 14.44 MB/sec warmup 34 sec latency 12.167 ms 2024-09-10T10:47:35.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 89985 14.32 MB/sec warmup 35 sec latency 10.356 ms 2024-09-10T10:47:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:35 smithi103 ceph-mon[35160]: pgmap v53: 97 pgs: 97 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 6.9 MiB/s wr, 291 op/s 2024-09-10T10:47:35.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:35 smithi007 ceph-mon[30873]: pgmap v53: 97 pgs: 97 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 6.9 MiB/s wr, 291 op/s 2024-09-10T10:47:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:35 smithi133 ceph-mon[35282]: pgmap v53: 97 pgs: 97 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 6.9 MiB/s wr, 291 op/s 2024-09-10T10:47:36.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 92666 14.39 MB/sec warmup 36 sec latency 16.758 ms 2024-09-10T10:47:37.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 95476 14.37 MB/sec warmup 37 sec latency 16.706 ms 2024-09-10T10:47:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:37 smithi103 ceph-mon[35160]: pgmap v54: 97 pgs: 97 active+clean; 34 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 445 op/s 2024-09-10T10:47:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:37 smithi007 ceph-mon[30873]: pgmap v54: 97 pgs: 97 active+clean; 34 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 445 op/s 2024-09-10T10:47:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:37 smithi133 ceph-mon[35282]: pgmap v54: 97 pgs: 97 active+clean; 34 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.6 MiB/s wr, 445 op/s 2024-09-10T10:47:38.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 97346 14.27 MB/sec warmup 38 sec latency 28.059 ms 2024-09-10T10:47:39.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 99815 14.27 MB/sec warmup 39 sec latency 38.233 ms 2024-09-10T10:47:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:39 smithi103 ceph-mon[35160]: pgmap v55: 97 pgs: 97 active+clean; 34 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.4 MiB/s wr, 422 op/s 2024-09-10T10:47:39.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:39 smithi007 ceph-mon[30873]: pgmap v55: 97 pgs: 97 active+clean; 34 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.4 MiB/s wr, 422 op/s 2024-09-10T10:47:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:39 smithi133 ceph-mon[35282]: pgmap v55: 97 pgs: 97 active+clean; 34 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.4 MiB/s wr, 422 op/s 2024-09-10T10:47:40.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 102734 14.27 MB/sec warmup 40 sec latency 15.638 ms 2024-09-10T10:47:41.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 104499 14.16 MB/sec warmup 41 sec latency 41.289 ms 2024-09-10T10:47:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:41 smithi103 ceph-mon[35160]: pgmap v56: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.1 MiB/s wr, 537 op/s 2024-09-10T10:47:41.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:41 smithi007 ceph-mon[30873]: pgmap v56: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.1 MiB/s wr, 537 op/s 2024-09-10T10:47:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:41 smithi133 ceph-mon[35282]: pgmap v56: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.1 MiB/s wr, 537 op/s 2024-09-10T10:47:42.504 INFO:tasks.workunit.client.0.smithi007.stdout: 1 106917 14.16 MB/sec warmup 42 sec latency 12.303 ms 2024-09-10T10:47:43.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 109972 14.23 MB/sec warmup 43 sec latency 19.100 ms 2024-09-10T10:47:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.682+0000 7fc17a00e640 1 -- 172.21.15.7:0/1990245703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc174101350 msgr2=0x7fc17410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.682+0000 7fc17a00e640 1 --2- 172.21.15.7:0/1990245703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc174101350 0x7fc17410d880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009930 tx=0x7fc16c02a420 comp rx=0 tx=0).stop 2024-09-10T10:47:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.682+0000 7fc17a00e640 1 -- 172.21.15.7:0/1990245703 shutdown_connections 2024-09-10T10:47:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.682+0000 7fc17a00e640 1 --2- 172.21.15.7:0/1990245703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc174101350 0x7fc17410d880 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.682+0000 7fc17a00e640 1 --2- 172.21.15.7:0/1990245703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1741009b0 0x7fc174100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.682+0000 7fc17a00e640 1 --2- 172.21.15.7:0/1990245703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1741069c0 0x7fc174106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.682+0000 7fc17a00e640 1 -- 172.21.15.7:0/1990245703 >> 172.21.15.7:0/1990245703 conn(0x7fc174077990 msgr2=0x7fc1740fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.682+0000 7fc17a00e640 1 -- 172.21.15.7:0/1990245703 shutdown_connections 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.682+0000 7fc17a00e640 1 -- 172.21.15.7:0/1990245703 wait complete. 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.683+0000 7fc17a00e640 1 Processor -- start 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.683+0000 7fc17a00e640 1 -- start start 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.683+0000 7fc17a00e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1741009b0 0x7fc174199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.683+0000 7fc17a00e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc174101350 0x7fc1741996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.683+0000 7fc17a00e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1741069c0 0x7fc1741a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.683+0000 7fc17a00e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc174110e00 con 0x7fc174101350 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.683+0000 7fc17a00e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc174110c80 con 0x7fc1741009b0 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.683+0000 7fc17a00e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc174110f80 con 0x7fc1741069c0 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc172ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc174101350 0x7fc1741996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc1737fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1741009b0 0x7fc174199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc172ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc174101350 0x7fc1741996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32794/0 (socket says 172.21.15.7:32794) 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc172ffd640 1 -- 172.21.15.7:0/753631183 learned_addr learned my addr 172.21.15.7:0/753631183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc173fff640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1741069c0 0x7fc1741a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc172ffd640 1 -- 172.21.15.7:0/753631183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1741069c0 msgr2=0x7fc1741a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc172ffd640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1741069c0 0x7fc1741a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc172ffd640 1 -- 172.21.15.7:0/753631183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1741009b0 msgr2=0x7fc174199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc172ffd640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1741009b0 0x7fc174199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc172ffd640 1 -- 172.21.15.7:0/753631183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1741a4730 con 0x7fc174101350 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc1737fe640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1741009b0 0x7fc174199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc172ffd640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc174101350 0x7fc1741996b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc16800e9f0 tx=0x7fc16800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc170ff9640 1 -- 172.21.15.7:0/753631183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16800cd80 con 0x7fc174101350 2024-09-10T10:47:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc17a00e640 1 -- 172.21.15.7:0/753631183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1741a4900 con 0x7fc174101350 2024-09-10T10:47:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc170ff9640 1 -- 172.21.15.7:0/753631183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc168005500 con 0x7fc174101350 2024-09-10T10:47:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc170ff9640 1 -- 172.21.15.7:0/753631183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc168010690 con 0x7fc174101350 2024-09-10T10:47:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.684+0000 7fc17a00e640 1 -- 172.21.15.7:0/753631183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1741a4e20 con 0x7fc174101350 2024-09-10T10:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc170ff9640 1 -- 172.21.15.7:0/753631183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc1680046c0 con 0x7fc174101350 2024-09-10T10:47:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc17ce87740 1 Processor -- start 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc17ce87740 1 -- start start 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ec2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 0x55d660ed1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ed44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc17ce87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d660e272f0 con 0x55d660ec29b0 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc17ce87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d660dcf270 con 0x55d660ed20c0 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc17ce87740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d660e232b0 con 0x55d660ec32d0 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc173fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ec2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc173fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ec2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32796/0 (socket says 172.21.15.7:32796) 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc173fff640 1 -- 172.21.15.7:0/668451382 learned_addr learned my addr 172.21.15.7:0/668451382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.687+0000 7fc1737fe640 1 --2- 172.21.15.7:0/668451382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 0x55d660ed1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc172ffd640 1 --2- 172.21.15.7:0/668451382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ed44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc173fff640 1 -- 172.21.15.7:0/668451382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 msgr2=0x55d660ed1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc173fff640 1 --2- 172.21.15.7:0/668451382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 0x55d660ed1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc173fff640 1 -- 172.21.15.7:0/668451382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 msgr2=0x55d660ed44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc173fff640 1 --2- 172.21.15.7:0/668451382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ed44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc170ff9640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc15c0a2da0 0x7fc15c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc173fff640 1 -- 172.21.15.7:0/668451382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d660dce340 con 0x55d660ec29b0 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc172ffd640 1 --2- 172.21.15.7:0/668451382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ed44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc1737fe640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc15c0a2da0 0x7fc15c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc170ff9640 1 -- 172.21.15.7:0/753631183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fc1680cf870 con 0x7fc174101350 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc173fff640 1 --2- 172.21.15.7:0/668451382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ec2d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009930 tx=0x7fc16c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da69da9ccd7af8e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc159ffb640 1 -- 172.21.15.7:0/668451382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c040070 con 0x55d660ec29b0 2024-09-10T10:47:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc159ffb640 1 -- 172.21.15.7:0/668451382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc16c0317b0 con 0x55d660ec29b0 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.688+0000 7fc159ffb640 1 -- 172.21.15.7:0/668451382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16c035040 con 0x55d660ec29b0 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.689+0000 7fc17ce87740 1 -- 172.21.15.7:0/668451382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 msgr2=0x55d660ec2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.689+0000 7fc1737fe640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc15c0a2da0 0x7fc15c0a5260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc164009fd0 tx=0x7fc164009500 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.689+0000 7fc17ce87740 1 --2- 172.21.15.7:0/668451382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ec2d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009930 tx=0x7fc16c005d50 comp rx=0 tx=0).stop 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.689+0000 7fc17ce87740 1 -- 172.21.15.7:0/668451382 shutdown_connections 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.689+0000 7fc17ce87740 1 --2- 172.21.15.7:0/668451382 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ed44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.689+0000 7fc17ce87740 1 --2- 172.21.15.7:0/668451382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 0x55d660ed1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.689+0000 7fc17ce87740 1 --2- 172.21.15.7:0/668451382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ec2d90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.689+0000 7fc17ce87740 1 -- 172.21.15.7:0/668451382 >> 172.21.15.7:0/668451382 conn(0x55d660ebf060 msgr2=0x55d660ebf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.689+0000 7fc17ce87740 1 -- 172.21.15.7:0/668451382 shutdown_connections 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.689+0000 7fc17ce87740 1 -- 172.21.15.7:0/668451382 wait complete. 2024-09-10T10:47:43.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc17ce87740 1 -- start start 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc17ce87740 1 Processor -- start 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ecf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 0x55d660ecf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ecfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc17ce87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d660ed4c30 con 0x55d660ec29b0 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc17ce87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d660db0290 con 0x55d660ed20c0 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc17ce87740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d660ed4db0 con 0x55d660ec32d0 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc17ce87740 10 client.0 _ll_register_callbacks cb 0x55d660de1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc1737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ecf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc1737fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ecf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32808/0 (socket says 172.21.15.7:32808) 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc1737fe640 1 -- 172.21.15.7:0/669525312 learned_addr learned my addr 172.21.15.7:0/669525312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:43.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc173fff640 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ecfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc172ffd640 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 0x55d660ecf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.690+0000 7fc1737fe640 1 -- 172.21.15.7:0/669525312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 msgr2=0x55d660ecf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc1737fe640 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 0x55d660ecf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc1737fe640 1 -- 172.21.15.7:0/669525312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 msgr2=0x55d660ecfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc1737fe640 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ecfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc1737fe640 1 -- 172.21.15.7:0/669525312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d660b3a210 con 0x55d660ec29b0 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc172ffd640 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 0x55d660ecf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc173fff640 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ecfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc1737fe640 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ecf050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc16400fbd0 tx=0x7fc164004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a32abdec0c5a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc152ffd640 1 -- 172.21.15.7:0/669525312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164011ca0 con 0x55d660ec29b0 2024-09-10T10:47:43.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc152ffd640 1 -- 172.21.15.7:0/669525312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fc164003970 con 0x55d660ec29b0 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc17ce87740 1 -- 172.21.15.7:0/669525312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc16c0317b0 con 0x55d660ec29b0 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc152ffd640 1 -- 172.21.15.7:0/669525312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc164011e40 con 0x55d660ec29b0 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc17ce87740 10 client.15372 fetch_fsmap learned FSMap version 26 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc17ce87740 1 -- 172.21.15.7:0/669525312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d660c06620 con 0x55d660ec29b0 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc152ffd640 1 -- 172.21.15.7:0/669525312 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc16c0317b0 con 0x55d660ec29b0 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc152ffd640 1 -- 172.21.15.7:0/669525312 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc164027020 con 0x55d660ec29b0 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc17ce87740 10 client.15372 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.691+0000 7fc17ce87740 10 client.15372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.692+0000 7fc17ce87740 20 client.15372 populate_metadata read hostname 'smithi007' 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.692+0000 7fc17ce87740 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d660ee2050 0x55d660ee4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.692+0000 7fc17ce87740 4 client.15372 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.692+0000 7fc17ce87740 1 -- 172.21.15.7:0/669525312 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d660e34a20 con 0x55d660ee2050 2024-09-10T10:47:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.692+0000 7fc172ffd640 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d660ee2050 0x55d660ee4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.692+0000 7fc152ffd640 10 client.15372 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.692+0000 7fc172ffd640 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d660ee2050 0x55d660ee4440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a3d63619f9b53e7 server_cookie=ae60162d932271b5 in_seq=0 out_seq=0 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc152ffd640 1 -- 172.21.15.7:0/669525312 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d660e34a20 con 0x55d660ee2050 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc152ffd640 10 client.15372 handle_command_reply: tid=0 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 client.15372 shutdown 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 -- 172.21.15.7:0/669525312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 msgr2=0x55d660ecf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ecf050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc16400fbd0 tx=0x7fc164004ab0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 -- 172.21.15.7:0/669525312 shutdown_connections 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d660ee2050 0x55d660ee4440 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ecfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec32d0 0x55d660ecf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc152ffd640 0 client.15372 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 --2- 172.21.15.7:0/669525312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec29b0 0x55d660ecf050 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 -- 172.21.15.7:0/669525312 >> 172.21.15.7:0/669525312 conn(0x55d660ebf060 msgr2=0x55d660ec4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 -- 172.21.15.7:0/669525312 shutdown_connections 2024-09-10T10:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 1 -- 172.21.15.7:0/669525312 wait complete. 2024-09-10T10:47:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.693+0000 7fc17ce87740 20 client.15372 trim_cache size 0 max 16384 2024-09-10T10:47:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc17ce87740 1 Processor -- start 2024-09-10T10:47:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc17ce87740 1 -- start start 2024-09-10T10:47:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ec29b0 0x55d660ed0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ee9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ed20c0 0x55d660eea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc17ce87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741ade20 con 0x55d660ec32d0 2024-09-10T10:47:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc17ce87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741ab6f0 con 0x55d660ec29b0 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc17ce87740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741a9540 con 0x55d660ed20c0 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc172ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ee9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc172ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ee9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32812/0 (socket says 172.21.15.7:32812) 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc172ffd640 1 -- 172.21.15.7:0/3249176346 learned_addr learned my addr 172.21.15.7:0/3249176346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc1737fe640 1 --2- 172.21.15.7:0/3249176346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ec29b0 0x55d660ed0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.709+0000 7fc173fff640 1 --2- 172.21.15.7:0/3249176346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ed20c0 0x55d660eea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.710+0000 7fc172ffd640 1 -- 172.21.15.7:0/3249176346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ed20c0 msgr2=0x55d660eea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.710+0000 7fc172ffd640 1 --2- 172.21.15.7:0/3249176346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ed20c0 0x55d660eea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.710+0000 7fc172ffd640 1 -- 172.21.15.7:0/3249176346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ec29b0 msgr2=0x55d660ed0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.710+0000 7fc172ffd640 1 --2- 172.21.15.7:0/3249176346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ec29b0 0x55d660ed0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.710+0000 7fc172ffd640 1 -- 172.21.15.7:0/3249176346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d660ec9e90 con 0x55d660ec32d0 2024-09-10T10:47:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.710+0000 7fc1737fe640 1 --2- 172.21.15.7:0/3249176346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ec29b0 0x55d660ed0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.710+0000 7fc172ffd640 1 --2- 172.21.15.7:0/3249176346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ee9d50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1680cd020 tx=0x7fc168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a233ec5074f1f2b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.710+0000 7fc1597fa640 1 -- 172.21.15.7:0/3249176346 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16809c490 con 0x55d660ec32d0 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.710+0000 7fc1597fa640 1 -- 172.21.15.7:0/3249176346 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc16809bd90 con 0x55d660ec32d0 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.710+0000 7fc1597fa640 1 -- 172.21.15.7:0/3249176346 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680a4570 con 0x55d660ec32d0 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 -- 172.21.15.7:0/3249176346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 msgr2=0x55d660ee9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 --2- 172.21.15.7:0/3249176346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ee9d50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1680cd020 tx=0x7fc168004a40 comp rx=0 tx=0).stop 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 -- 172.21.15.7:0/3249176346 shutdown_connections 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 --2- 172.21.15.7:0/3249176346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ed20c0 0x55d660eea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 --2- 172.21.15.7:0/3249176346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ee9d50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 --2- 172.21.15.7:0/3249176346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ec29b0 0x55d660ed0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 -- 172.21.15.7:0/3249176346 >> 172.21.15.7:0/3249176346 conn(0x55d660ebf060 msgr2=0x55d660ed08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 -- 172.21.15.7:0/3249176346 shutdown_connections 2024-09-10T10:47:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 -- 172.21.15.7:0/3249176346 wait complete. 2024-09-10T10:47:43.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 -- start start 2024-09-10T10:47:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.711+0000 7fc17ce87740 1 Processor -- start 2024-09-10T10:47:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec29b0 0x55d660eda590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ecbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc17ce87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ecc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc17ce87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741a9bb0 con 0x55d660ec32d0 2024-09-10T10:47:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc17ce87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741ab020 con 0x55d660ed20c0 2024-09-10T10:47:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc17ce87740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc1741aa280 con 0x55d660ec29b0 2024-09-10T10:47:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc17ce87740 10 client.0 _ll_register_callbacks cb 0x7fc1640125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc172ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ecbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc172ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ecbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32814/0 (socket says 172.21.15.7:32814) 2024-09-10T10:47:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc172ffd640 1 -- 172.21.15.7:0/3170012748 learned_addr learned my addr 172.21.15.7:0/3170012748 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc1737fe640 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec29b0 0x55d660eda590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc173fff640 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ecc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc172ffd640 1 -- 172.21.15.7:0/3170012748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec29b0 msgr2=0x55d660eda590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc172ffd640 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec29b0 0x55d660eda590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc172ffd640 1 -- 172.21.15.7:0/3170012748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 msgr2=0x55d660ecc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc172ffd640 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ecc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc172ffd640 1 -- 172.21.15.7:0/3170012748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d660ecc820 con 0x55d660ec32d0 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc172ffd640 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ecbc70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc168098760 tx=0x7fc1680a0140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4c34fad7261e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc152ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc152ffd640 1 -- 172.21.15.7:0/3170012748 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1680b9060 con 0x55d660ec32d0 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc152ffd640 1 -- 172.21.15.7:0/3170012748 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fc168098470 con 0x55d660ec32d0 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.712+0000 7fc17ce87740 1 -- 172.21.15.7:0/3170012748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc16809bd90 con 0x55d660ec32d0 2024-09-10T10:47:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc152ffd640 1 -- 172.21.15.7:0/3170012748 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc16809c490 con 0x55d660ec32d0 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc17ce87740 10 client.15378 fetch_fsmap learned FSMap version 26 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc17ce87740 1 -- 172.21.15.7:0/3170012748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d660f4cbc0 con 0x55d660ec32d0 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc152ffd640 1 -- 172.21.15.7:0/3170012748 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc16809bd90 con 0x55d660ec32d0 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc152ffd640 1 -- 172.21.15.7:0/3170012748 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc16809ac50 con 0x55d660ec32d0 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc17ce87740 10 client.15378 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc17ce87740 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc17ce87740 20 client.15378 populate_metadata read hostname 'smithi007' 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc17ce87740 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d660f4d3a0 0x55d660f4f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc17ce87740 4 client.15378 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc17ce87740 1 -- 172.21.15.7:0/3170012748 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d660e2ed70 con 0x55d660f4d3a0 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.713+0000 7fc1737fe640 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d660f4d3a0 0x55d660f4f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.714+0000 7fc1737fe640 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d660f4d3a0 0x55d660f4f790 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39081a38a1c37d47 server_cookie=cf80db6d6b8bdb7d in_seq=0 out_seq=0 2024-09-10T10:47:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.714+0000 7fc152ffd640 10 client.15378 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.778+0000 7fc152ffd640 1 -- 172.21.15.7:0/3170012748 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d660e2ed70 con 0x55d660f4d3a0 2024-09-10T10:47:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.778+0000 7fc152ffd640 10 client.15378 handle_command_reply: tid=0 2024-09-10T10:47:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.778+0000 7fc17ce87740 1 client.15378 shutdown 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.778+0000 7fc17ce87740 1 -- 172.21.15.7:0/3170012748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 msgr2=0x55d660ecbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.778+0000 7fc17ce87740 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ecbc70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc168098760 tx=0x7fc1680a0140 comp rx=0 tx=0).stop 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.779+0000 7fc17ce87740 1 -- 172.21.15.7:0/3170012748 shutdown_connections 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.779+0000 7fc17ce87740 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d660f4d3a0 0x55d660f4f790 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.779+0000 7fc17ce87740 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d660ed20c0 0x55d660ecc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.779+0000 7fc152ffd640 0 client.15378 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.779+0000 7fc17ce87740 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d660ec32d0 0x55d660ecbc70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.779+0000 7fc17ce87740 1 --2- 172.21.15.7:0/3170012748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d660ec29b0 0x55d660eda590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.779+0000 7fc17ce87740 1 -- 172.21.15.7:0/3170012748 >> 172.21.15.7:0/3170012748 conn(0x55d660ebf060 msgr2=0x55d660ebe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.779+0000 7fc17ce87740 1 -- 172.21.15.7:0/3170012748 shutdown_connections 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.779+0000 7fc17ce87740 1 -- 172.21.15.7:0/3170012748 wait complete. 2024-09-10T10:47:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.779+0000 7fc17ce87740 20 client.15378 trim_cache size 0 max 16384 2024-09-10T10:47:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:47:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:47:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:47:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 -- 172.21.15.7:0/753631183 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc15c0a2da0 msgr2=0x7fc15c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc15c0a2da0 0x7fc15c0a5260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc164009fd0 tx=0x7fc164009500 comp rx=0 tx=0).stop 2024-09-10T10:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 -- 172.21.15.7:0/753631183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc174101350 msgr2=0x7fc1741996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc174101350 0x7fc1741996b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc16800e9f0 tx=0x7fc16800edf0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 -- 172.21.15.7:0/753631183 shutdown_connections 2024-09-10T10:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1741069c0 0x7fc1741a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc15c0a2da0 0x7fc15c0a5260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc174101350 0x7fc1741996b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 --2- 172.21.15.7:0/753631183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1741009b0 0x7fc174199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 -- 172.21.15.7:0/753631183 >> 172.21.15.7:0/753631183 conn(0x7fc174077990 msgr2=0x7fc174103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 -- 172.21.15.7:0/753631183 shutdown_connections 2024-09-10T10:47:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:43.780+0000 7fc17a00e640 1 -- 172.21.15.7:0/753631183 wait complete. 2024-09-10T10:47:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:43 smithi103 ceph-mon[35160]: pgmap v57: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 269 op/s 2024-09-10T10:47:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:43 smithi007 ceph-mon[30873]: pgmap v57: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 269 op/s 2024-09-10T10:47:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:43 smithi133 ceph-mon[35282]: pgmap v57: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 269 op/s 2024-09-10T10:47:44.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 111558 14.07 MB/sec warmup 44 sec latency 33.621 ms 2024-09-10T10:47:45.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 113979 14.06 MB/sec warmup 45 sec latency 13.332 ms 2024-09-10T10:47:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:45 smithi103 ceph-mon[35160]: pgmap v58: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 269 op/s 2024-09-10T10:47:45.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:45 smithi007 ceph-mon[30873]: pgmap v58: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 269 op/s 2024-09-10T10:47:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:45 smithi133 ceph-mon[35282]: pgmap v58: 97 pgs: 97 active+clean; 41 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 269 op/s 2024-09-10T10:47:46.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 116891 14.07 MB/sec warmup 46 sec latency 16.467 ms 2024-09-10T10:47:47.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 118728 13.98 MB/sec warmup 47 sec latency 12.592 ms 2024-09-10T10:47:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:47 smithi103 ceph-mon[35160]: pgmap v59: 97 pgs: 97 active+clean; 39 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 434 op/s 2024-09-10T10:47:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:47 smithi007 ceph-mon[30873]: pgmap v59: 97 pgs: 97 active+clean; 39 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 434 op/s 2024-09-10T10:47:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:47 smithi133 ceph-mon[35282]: pgmap v59: 97 pgs: 97 active+clean; 39 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.0 MiB/s wr, 434 op/s 2024-09-10T10:47:48.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 121099 13.98 MB/sec warmup 48 sec latency 11.865 ms 2024-09-10T10:47:49.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 124240 14.06 MB/sec warmup 49 sec latency 12.489 ms 2024-09-10T10:47:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:49 smithi103 ceph-mon[35160]: pgmap v60: 97 pgs: 97 active+clean; 39 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 280 op/s 2024-09-10T10:47:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:49 smithi007 ceph-mon[30873]: pgmap v60: 97 pgs: 97 active+clean; 39 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 280 op/s 2024-09-10T10:47:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:49 smithi133 ceph-mon[35282]: pgmap v60: 97 pgs: 97 active+clean; 39 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 280 op/s 2024-09-10T10:47:50.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 126222 13.92 MB/sec warmup 50 sec latency 10.886 ms 2024-09-10T10:47:51.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 128315 13.91 MB/sec warmup 51 sec latency 15.118 ms 2024-09-10T10:47:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:51 smithi103 ceph-mon[35160]: pgmap v61: 97 pgs: 97 active+clean; 44 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 396 op/s 2024-09-10T10:47:51.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:51 smithi007 ceph-mon[30873]: pgmap v61: 97 pgs: 97 active+clean; 44 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 396 op/s 2024-09-10T10:47:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:51 smithi133 ceph-mon[35282]: pgmap v61: 97 pgs: 97 active+clean; 44 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 396 op/s 2024-09-10T10:47:52.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 131485 14.00 MB/sec warmup 52 sec latency 13.741 ms 2024-09-10T10:47:53.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 133667 13.88 MB/sec warmup 53 sec latency 14.562 ms 2024-09-10T10:47:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:53 smithi103 ceph-mon[35160]: pgmap v62: 97 pgs: 97 active+clean; 44 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-09-10T10:47:53.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:53 smithi007 ceph-mon[30873]: pgmap v62: 97 pgs: 97 active+clean; 44 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-09-10T10:47:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:53 smithi133 ceph-mon[35282]: pgmap v62: 97 pgs: 97 active+clean; 44 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-09-10T10:47:54.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 135531 13.85 MB/sec warmup 54 sec latency 16.961 ms 2024-09-10T10:47:55.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 138588 13.94 MB/sec warmup 55 sec latency 12.760 ms 2024-09-10T10:47:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:55 smithi103 ceph-mon[35160]: pgmap v63: 97 pgs: 97 active+clean; 44 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-09-10T10:47:55.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:55 smithi007 ceph-mon[30873]: pgmap v63: 97 pgs: 97 active+clean; 44 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-09-10T10:47:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:55 smithi133 ceph-mon[35282]: pgmap v63: 97 pgs: 97 active+clean; 44 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.3 MiB/s wr, 281 op/s 2024-09-10T10:47:56.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 140805 13.82 MB/sec warmup 56 sec latency 15.151 ms 2024-09-10T10:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.872+0000 7fda5ece4640 1 -- 172.21.15.7:0/247061106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58102ce0 msgr2=0x7fda581030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.872+0000 7fda5ece4640 1 --2- 172.21.15.7:0/247061106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58102ce0 0x7fda581030c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009900 tx=0x7fda4c02a480 comp rx=0 tx=0).stop 2024-09-10T10:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.872+0000 7fda5ece4640 1 -- 172.21.15.7:0/247061106 shutdown_connections 2024-09-10T10:47:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.872+0000 7fda5ece4640 1 --2- 172.21.15.7:0/247061106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda5810c0b0 0x7fda5810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.872+0000 7fda5ece4640 1 --2- 172.21.15.7:0/247061106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda58103600 0x7fda5810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.872+0000 7fda5ece4640 1 --2- 172.21.15.7:0/247061106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda58102ce0 0x7fda581030c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.872+0000 7fda5ece4640 1 -- 172.21.15.7:0/247061106 >> 172.21.15.7:0/247061106 conn(0x7fda58077990 msgr2=0x7fda580fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.872+0000 7fda5ece4640 1 -- 172.21.15.7:0/247061106 shutdown_connections 2024-09-10T10:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.872+0000 7fda5ece4640 1 -- 172.21.15.7:0/247061106 wait complete. 2024-09-10T10:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5ece4640 1 Processor -- start 2024-09-10T10:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5ece4640 1 -- start start 2024-09-10T10:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5ece4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda58102ce0 0x7fda58198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5ece4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda58103600 0x7fda58198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5ece4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda5810c0b0 0x7fda5819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5ece4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda58111eb0 con 0x7fda5810c0b0 2024-09-10T10:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5ece4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda58111d30 con 0x7fda58102ce0 2024-09-10T10:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5ece4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda58112030 con 0x7fda58103600 2024-09-10T10:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5d281640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda5810c0b0 0x7fda5819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda57fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda58103600 0x7fda58198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5d281640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda5810c0b0 0x7fda5819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33682/0 (socket says 172.21.15.7:33682) 2024-09-10T10:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5d281640 1 -- 172.21.15.7:0/3914786778 learned_addr learned my addr 172.21.15.7:0/3914786778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.873+0000 7fda5ca80640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda58102ce0 0x7fda58198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda5d281640 1 -- 172.21.15.7:0/3914786778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda58103600 msgr2=0x7fda58198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda5d281640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda58103600 0x7fda58198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda5d281640 1 -- 172.21.15.7:0/3914786778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda58102ce0 msgr2=0x7fda58198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda5d281640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda58102ce0 0x7fda58198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda5d281640 1 -- 172.21.15.7:0/3914786778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5819d2f0 con 0x7fda5810c0b0 2024-09-10T10:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda5d281640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda5810c0b0 0x7fda5819cbc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fda5000eb10 tx=0x7fda5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda55ffb640 1 -- 172.21.15.7:0/3914786778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50004b20 con 0x7fda5810c0b0 2024-09-10T10:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda55ffb640 1 -- 172.21.15.7:0/3914786778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda50004cc0 con 0x7fda5810c0b0 2024-09-10T10:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda5ece4640 1 -- 172.21.15.7:0/3914786778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5819d5e0 con 0x7fda5810c0b0 2024-09-10T10:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda55ffb640 1 -- 172.21.15.7:0/3914786778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50010760 con 0x7fda5810c0b0 2024-09-10T10:47:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.874+0000 7fda5ece4640 1 -- 172.21.15.7:0/3914786778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda581a4e70 con 0x7fda5810c0b0 2024-09-10T10:47:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.876+0000 7fda55ffb640 1 -- 172.21.15.7:0/3914786778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fda50005020 con 0x7fda5810c0b0 2024-09-10T10:47:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda61a85740 1 Processor -- start 2024-09-10T10:47:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda61a85740 1 -- start start 2024-09-10T10:47:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda55ffb640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fda480a2e20 0x7fda480a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda61a85740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda61a85740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a6410 0x564cd40b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda61a85740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda57fff640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fda480a2e20 0x7fda480a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda61a85740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564cd400a2f0 con 0x564cd40a5af0 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda55ffb640 1 -- 172.21.15.7:0/3914786778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fda500cfa20 con 0x7fda5810c0b0 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda61a85740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564cd3fb2270 con 0x564cd40a6410 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.877+0000 7fda61a85740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564cd40062b0 con 0x564cd40b0cb0 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda5ca80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda5ca80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33688/0 (socket says 172.21.15.7:33688) 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda5ca80640 1 -- 172.21.15.7:0/447651529 learned_addr learned my addr 172.21.15.7:0/447651529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda57fff640 1 --2- 172.21.15.7:0/447651529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a6410 0x564cd40b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda5d281640 1 --2- 172.21.15.7:0/447651529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda5ca80640 1 -- 172.21.15.7:0/447651529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 msgr2=0x564cd40b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda5ca80640 1 --2- 172.21.15.7:0/447651529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda5ca80640 1 -- 172.21.15.7:0/447651529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a6410 msgr2=0x564cd40b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda5ca80640 1 --2- 172.21.15.7:0/447651529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a6410 0x564cd40b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda5ca80640 1 -- 172.21.15.7:0/447651529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cd3fb1340 con 0x564cd40a5af0 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda5ca80640 1 --2- 172.21.15.7:0/447651529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a5ed0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009900 tx=0x7fda4c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd11fecb5c989874 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda57fff640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fda480a2e20 0x7fda480a52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda580ffe50 tx=0x7fda44009ad0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda42ffd640 1 -- 172.21.15.7:0/447651529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c040070 con 0x564cd40a5af0 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda42ffd640 1 -- 172.21.15.7:0/447651529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4c031de0 con 0x564cd40a5af0 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.878+0000 7fda42ffd640 1 -- 172.21.15.7:0/447651529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c035040 con 0x564cd40a5af0 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.879+0000 7fda61a85740 1 -- 172.21.15.7:0/447651529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 msgr2=0x564cd40a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.879+0000 7fda61a85740 1 --2- 172.21.15.7:0/447651529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a5ed0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009900 tx=0x7fda4c005be0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.879+0000 7fda61a85740 1 -- 172.21.15.7:0/447651529 shutdown_connections 2024-09-10T10:47:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.879+0000 7fda61a85740 1 --2- 172.21.15.7:0/447651529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.879+0000 7fda61a85740 1 --2- 172.21.15.7:0/447651529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a6410 0x564cd40b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.879+0000 7fda61a85740 1 --2- 172.21.15.7:0/447651529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a5ed0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.879+0000 7fda61a85740 1 -- 172.21.15.7:0/447651529 >> 172.21.15.7:0/447651529 conn(0x564cd40a2060 msgr2=0x564cd40a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.879+0000 7fda61a85740 1 -- 172.21.15.7:0/447651529 shutdown_connections 2024-09-10T10:47:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.879+0000 7fda61a85740 1 -- 172.21.15.7:0/447651529 wait complete. 2024-09-10T10:47:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.879+0000 7fda61a85740 1 -- start start 2024-09-10T10:47:56.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda61a85740 1 Processor -- start 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda61a85740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a5af0 0x564cd40adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda61a85740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40a6410 0x564cd40ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda61a85740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40b0cb0 0x564cd40ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda61a85740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564cd40b7c90 con 0x564cd40b0cb0 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda61a85740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564cd3f93290 con 0x564cd40a5af0 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda61a85740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564cd40b7e10 con 0x564cd40a6410 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda61a85740 10 client.0 _ll_register_callbacks cb 0x564cd3fc4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda5d281640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40b0cb0 0x564cd40ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda5d281640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40b0cb0 0x564cd40ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33698/0 (socket says 172.21.15.7:33698) 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda5d281640 1 -- 172.21.15.7:0/97794138 learned_addr learned my addr 172.21.15.7:0/97794138 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:56.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda57fff640 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40a6410 0x564cd40ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda5ca80640 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a5af0 0x564cd40adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda5d281640 1 -- 172.21.15.7:0/97794138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40a6410 msgr2=0x564cd40ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda5d281640 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40a6410 0x564cd40ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda5d281640 1 -- 172.21.15.7:0/97794138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a5af0 msgr2=0x564cd40adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda5d281640 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a5af0 0x564cd40adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.880+0000 7fda5d281640 1 -- 172.21.15.7:0/97794138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cd3d1d210 con 0x564cd40b0cb0 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda5d281640 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40b0cb0 0x564cd40ae5c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fda500049d0 tx=0x7fda5009bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d71eae28e20d0f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda37fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda37fff640 1 -- 172.21.15.7:0/97794138 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda500b4030 con 0x564cd40b0cb0 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda61a85740 1 -- 172.21.15.7:0/97794138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda4c031de0 con 0x564cd40b0cb0 2024-09-10T10:47:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda37fff640 1 -- 172.21.15.7:0/97794138 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fda500bad80 con 0x564cd40b0cb0 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda37fff640 1 -- 172.21.15.7:0/97794138 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda5007f050 con 0x564cd40b0cb0 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda61a85740 10 client.15390 fetch_fsmap learned FSMap version 26 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda61a85740 1 -- 172.21.15.7:0/97794138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cd3de9620 con 0x564cd40b0cb0 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda37fff640 1 -- 172.21.15.7:0/97794138 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4c031de0 con 0x564cd40b0cb0 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda37fff640 1 -- 172.21.15.7:0/97794138 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fda50005310 con 0x564cd40b0cb0 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda61a85740 10 client.15390 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda61a85740 10 client.15390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda61a85740 20 client.15390 populate_metadata read hostname 'smithi007' 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda61a85740 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564cd40c50d0 0x564cd40c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda61a85740 4 client.15390 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.881+0000 7fda61a85740 1 -- 172.21.15.7:0/97794138 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cd4017a20 con 0x564cd40c50d0 2024-09-10T10:47:56.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.882+0000 7fda5ca80640 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564cd40c50d0 0x564cd40c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.882+0000 7fda5ca80640 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564cd40c50d0 0x564cd40c74e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4ed4d4cfb5f811 server_cookie=55c5d709083b222e in_seq=0 out_seq=0 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.882+0000 7fda37fff640 10 client.15390 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda37fff640 1 -- 172.21.15.7:0/97794138 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564cd4017a20 con 0x564cd40c50d0 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda37fff640 10 client.15390 handle_command_reply: tid=0 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 client.15390 shutdown 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 -- 172.21.15.7:0/97794138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40b0cb0 msgr2=0x564cd40ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40b0cb0 0x564cd40ae5c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fda500049d0 tx=0x7fda5009bde0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 -- 172.21.15.7:0/97794138 shutdown_connections 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564cd40c50d0 0x564cd40c74e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40b0cb0 0x564cd40ae5c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40a6410 0x564cd40ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda37fff640 0 client.15390 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 --2- 172.21.15.7:0/97794138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a5af0 0x564cd40adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 -- 172.21.15.7:0/97794138 >> 172.21.15.7:0/97794138 conn(0x564cd40a2060 msgr2=0x564cd40a7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 -- 172.21.15.7:0/97794138 shutdown_connections 2024-09-10T10:47:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 1 -- 172.21.15.7:0/97794138 wait complete. 2024-09-10T10:47:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.883+0000 7fda61a85740 20 client.15390 trim_cache size 0 max 16384 2024-09-10T10:47:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda61a85740 1 Processor -- start 2024-09-10T10:47:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda61a85740 1 -- start start 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda61a85740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda61a85740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a6410 0x564cd40afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda61a85740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda61a85740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda581ade00 con 0x564cd40a5af0 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda61a85740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda581ab6d0 con 0x564cd40a6410 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda61a85740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda581a9540 con 0x564cd40b0cb0 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda5ca80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda5ca80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33706/0 (socket says 172.21.15.7:33706) 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda5ca80640 1 -- 172.21.15.7:0/2854878407 learned_addr learned my addr 172.21.15.7:0/2854878407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda57fff640 1 --2- 172.21.15.7:0/2854878407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a6410 0x564cd40afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda5d281640 1 --2- 172.21.15.7:0/2854878407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda5ca80640 1 -- 172.21.15.7:0/2854878407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 msgr2=0x564cd40ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda5ca80640 1 --2- 172.21.15.7:0/2854878407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda5ca80640 1 -- 172.21.15.7:0/2854878407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a6410 msgr2=0x564cd40afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.899+0000 7fda5ca80640 1 --2- 172.21.15.7:0/2854878407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a6410 0x564cd40afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.900+0000 7fda5ca80640 1 -- 172.21.15.7:0/2854878407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cd40a1e90 con 0x564cd40a5af0 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.900+0000 7fda5ca80640 1 --2- 172.21.15.7:0/2854878407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a85b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009900 tx=0x7fda4c031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6906d354db889d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.900+0000 7fda427fc640 1 -- 172.21.15.7:0/2854878407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c035040 con 0x564cd40a5af0 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.900+0000 7fda427fc640 1 -- 172.21.15.7:0/2854878407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4c005030 con 0x564cd40a5af0 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.900+0000 7fda427fc640 1 -- 172.21.15.7:0/2854878407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c040040 con 0x564cd40a5af0 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 -- 172.21.15.7:0/2854878407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 msgr2=0x564cd40a85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 --2- 172.21.15.7:0/2854878407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a85b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009900 tx=0x7fda4c031f90 comp rx=0 tx=0).stop 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 -- 172.21.15.7:0/2854878407 shutdown_connections 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 --2- 172.21.15.7:0/2854878407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 --2- 172.21.15.7:0/2854878407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a6410 0x564cd40afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 --2- 172.21.15.7:0/2854878407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a5af0 0x564cd40a85b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 -- 172.21.15.7:0/2854878407 >> 172.21.15.7:0/2854878407 conn(0x564cd40a2060 msgr2=0x564cd40af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 -- 172.21.15.7:0/2854878407 shutdown_connections 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 -- 172.21.15.7:0/2854878407 wait complete. 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 -- start start 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.901+0000 7fda61a85740 1 Processor -- start 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda61a85740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a5af0 0x564cd40bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda61a85740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a6410 0x564cd40bd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda61a85740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40aa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda61a85740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda581a9b90 con 0x564cd40a6410 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda61a85740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda581ab000 con 0x564cd40a5af0 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda61a85740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda581aa260 con 0x564cd40b0cb0 2024-09-10T10:47:56.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda61a85740 10 client.0 _ll_register_callbacks cb 0x7fda50004e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda57fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a6410 0x564cd40bd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda5d281640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda57fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a6410 0x564cd40bd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33708/0 (socket says 172.21.15.7:33708) 2024-09-10T10:47:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda57fff640 1 -- 172.21.15.7:0/80890479 learned_addr learned my addr 172.21.15.7:0/80890479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:47:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda5ca80640 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a5af0 0x564cd40bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda57fff640 1 -- 172.21.15.7:0/80890479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 msgr2=0x564cd40aa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda57fff640 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40aa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda57fff640 1 -- 172.21.15.7:0/80890479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a5af0 msgr2=0x564cd40bd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:56.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda57fff640 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a5af0 0x564cd40bd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda57fff640 1 -- 172.21.15.7:0/80890479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cd40aaab0 con 0x564cd40a6410 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda5d281640 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda57fff640 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a6410 0x564cd40bd7d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda44009250 tx=0x7fda44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff6e530b4a7cc9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.902+0000 7fda37fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda37fff640 1 -- 172.21.15.7:0/80890479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4400fe10 con 0x564cd40a6410 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda37fff640 1 -- 172.21.15.7:0/80890479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7fda44009030 con 0x564cd40a6410 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda61a85740 1 -- 172.21.15.7:0/80890479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda4c005030 con 0x564cd40a6410 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda37fff640 1 -- 172.21.15.7:0/80890479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda44005020 con 0x564cd40a6410 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda61a85740 10 client.15396 fetch_fsmap learned FSMap version 26 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda61a85740 1 -- 172.21.15.7:0/80890479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cd412f790 con 0x564cd40a6410 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda37fff640 1 -- 172.21.15.7:0/80890479 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4c005030 con 0x564cd40a6410 2024-09-10T10:47:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda37fff640 1 -- 172.21.15.7:0/80890479 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fda44020050 con 0x564cd40a6410 2024-09-10T10:47:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda61a85740 10 client.15396 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T10:47:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda61a85740 10 client.15396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:47:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda61a85740 20 client.15396 populate_metadata read hostname 'smithi007' 2024-09-10T10:47:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda61a85740 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564cd412ff50 0x564cd4132370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda61a85740 4 client.15396 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:47:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.903+0000 7fda61a85740 1 -- 172.21.15.7:0/80890479 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564cd4011d70 con 0x564cd412ff50 2024-09-10T10:47:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.904+0000 7fda5ca80640 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564cd412ff50 0x564cd4132370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.904+0000 7fda5ca80640 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564cd412ff50 0x564cd4132370 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=759b36cdebeb026d server_cookie=7adb2244a5e2fd36 in_seq=0 out_seq=0 2024-09-10T10:47:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.904+0000 7fda37fff640 10 client.15396 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.966+0000 7fda37fff640 1 -- 172.21.15.7:0/80890479 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564cd4011d70 con 0x564cd412ff50 2024-09-10T10:47:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.966+0000 7fda37fff640 10 client.15396 handle_command_reply: tid=0 2024-09-10T10:47:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.966+0000 7fda61a85740 1 client.15396 shutdown 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.966+0000 7fda61a85740 1 -- 172.21.15.7:0/80890479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a6410 msgr2=0x564cd40bd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.966+0000 7fda61a85740 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a6410 0x564cd40bd7d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda44009250 tx=0x7fda44004730 comp rx=0 tx=0).stop 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.966+0000 7fda61a85740 1 -- 172.21.15.7:0/80890479 shutdown_connections 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.966+0000 7fda61a85740 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564cd412ff50 0x564cd4132370 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda61a85740 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd40b0cb0 0x564cd40aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda61a85740 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd40a6410 0x564cd40bd7d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda37fff640 0 client.15396 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda61a85740 1 --2- 172.21.15.7:0/80890479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564cd40a5af0 0x564cd40bd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda61a85740 1 -- 172.21.15.7:0/80890479 >> 172.21.15.7:0/80890479 conn(0x564cd40a2060 msgr2=0x564cd40af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda61a85740 1 -- 172.21.15.7:0/80890479 shutdown_connections 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda61a85740 1 -- 172.21.15.7:0/80890479 wait complete. 2024-09-10T10:47:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda61a85740 20 client.15396 trim_cache size 0 max 16384 2024-09-10T10:47:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:47:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:47:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:47:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:47:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda5ece4640 1 -- 172.21.15.7:0/3914786778 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fda480a2e20 msgr2=0x7fda480a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda5ece4640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fda480a2e20 0x7fda480a52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda580ffe50 tx=0x7fda44009ad0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda5ece4640 1 -- 172.21.15.7:0/3914786778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda5810c0b0 msgr2=0x7fda5819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.967+0000 7fda5ece4640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda5810c0b0 0x7fda5819cbc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fda5000eb10 tx=0x7fda5000c2d0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.968+0000 7fda5ece4640 1 -- 172.21.15.7:0/3914786778 shutdown_connections 2024-09-10T10:47:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.968+0000 7fda5ece4640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fda480a2e20 0x7fda480a52e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.968+0000 7fda5ece4640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda5810c0b0 0x7fda5819cbc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.968+0000 7fda5ece4640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda58103600 0x7fda58198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.968+0000 7fda5ece4640 1 --2- 172.21.15.7:0/3914786778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda58102ce0 0x7fda58198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.968+0000 7fda5ece4640 1 -- 172.21.15.7:0/3914786778 >> 172.21.15.7:0/3914786778 conn(0x7fda58077990 msgr2=0x7fda580fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.968+0000 7fda5ece4640 1 -- 172.21.15.7:0/3914786778 shutdown_connections 2024-09-10T10:47:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:47:56.968+0000 7fda5ece4640 1 -- 172.21.15.7:0/3914786778 wait complete. 2024-09-10T10:47:57.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 142578 13.79 MB/sec warmup 57 sec latency 38.081 ms 2024-09-10T10:47:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:57 smithi103 ceph-mon[35160]: pgmap v64: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-09-10T10:47:57.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:57 smithi007 ceph-mon[30873]: pgmap v64: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-09-10T10:47:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:57 smithi133 ceph-mon[35282]: pgmap v64: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-09-10T10:47:58.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 145703 13.87 MB/sec warmup 58 sec latency 10.862 ms 2024-09-10T10:47:58.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:58 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:47:58.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:58 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:47:58.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:58 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:47:59.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 148182 13.83 MB/sec warmup 59 sec latency 17.538 ms 2024-09-10T10:47:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:59 smithi103 ceph-mon[35160]: pgmap v65: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-09-10T10:47:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:59 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:59 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:59 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:59 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:59 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:59 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:59 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:47:59.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:59 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:47:59.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:47:59 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:59 smithi007 ceph-mon[30873]: pgmap v65: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-09-10T10:47:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:59 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:59 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:59 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:59 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:59 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:59 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:59 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:47:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:59 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:47:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:47:59 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:59 smithi133 ceph-mon[35282]: pgmap v65: 97 pgs: 97 active+clean; 50 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-09-10T10:47:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:59 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:59 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:59 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:59 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:59 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:59 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:47:59.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:59 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:47:59.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:59 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:47:59.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:47:59 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:48:00.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 150136 13.77 MB/sec warmup 60 sec latency 11.334 ms 2024-09-10T10:48:01.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 152610 13.80 MB/sec warmup 61 sec latency 24.241 ms 2024-09-10T10:48:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:01 smithi103 ceph-mon[35160]: pgmap v66: 97 pgs: 97 active+clean; 52 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.1 MiB/s wr, 399 op/s 2024-09-10T10:48:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:01 smithi007 ceph-mon[30873]: pgmap v66: 97 pgs: 97 active+clean; 52 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.1 MiB/s wr, 399 op/s 2024-09-10T10:48:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:01 smithi133 ceph-mon[35282]: pgmap v66: 97 pgs: 97 active+clean; 52 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.1 MiB/s wr, 399 op/s 2024-09-10T10:48:02.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 154396 13.69 MB/sec warmup 62 sec latency 33.470 ms 2024-09-10T10:48:03.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 156684 13.69 MB/sec warmup 63 sec latency 17.713 ms 2024-09-10T10:48:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:03 smithi103 ceph-mon[35160]: pgmap v67: 97 pgs: 97 active+clean; 52 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-09-10T10:48:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:03 smithi007 ceph-mon[30873]: pgmap v67: 97 pgs: 97 active+clean; 52 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-09-10T10:48:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:03 smithi133 ceph-mon[35282]: pgmap v67: 97 pgs: 97 active+clean; 52 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-09-10T10:48:04.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 159777 13.75 MB/sec warmup 64 sec latency 11.715 ms 2024-09-10T10:48:05.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 161618 13.65 MB/sec warmup 65 sec latency 24.948 ms 2024-09-10T10:48:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:05 smithi103 ceph-mon[35160]: pgmap v68: 97 pgs: 97 active+clean; 52 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-09-10T10:48:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:05 smithi007 ceph-mon[30873]: pgmap v68: 97 pgs: 97 active+clean; 52 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-09-10T10:48:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:05 smithi133 ceph-mon[35282]: pgmap v68: 97 pgs: 97 active+clean; 52 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-09-10T10:48:06.505 INFO:tasks.workunit.client.0.smithi007.stdout: 1 163850 13.65 MB/sec warmup 66 sec latency 11.487 ms 2024-09-10T10:48:07.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 166692 13.70 MB/sec warmup 67 sec latency 42.035 ms 2024-09-10T10:48:07.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:07 smithi007 ceph-mon[30873]: pgmap v69: 97 pgs: 97 active+clean; 59 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 455 op/s 2024-09-10T10:48:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:07 smithi133 ceph-mon[35282]: pgmap v69: 97 pgs: 97 active+clean; 59 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 455 op/s 2024-09-10T10:48:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:07 smithi103 ceph-mon[35160]: pgmap v69: 97 pgs: 97 active+clean; 59 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 455 op/s 2024-09-10T10:48:08.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 168508 13.60 MB/sec warmup 68 sec latency 12.171 ms 2024-09-10T10:48:09.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 170759 13.61 MB/sec warmup 69 sec latency 16.134 ms 2024-09-10T10:48:09.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:09 smithi007 ceph-mon[30873]: pgmap v70: 97 pgs: 97 active+clean; 59 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 341 op/s 2024-09-10T10:48:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:09 smithi133 ceph-mon[35282]: pgmap v70: 97 pgs: 97 active+clean; 59 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 341 op/s 2024-09-10T10:48:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.059+0000 7f2e96b96640 1 -- 172.21.15.7:0/1238141039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e9010c0c0 msgr2=0x7f2e9010e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.059+0000 7f2e96b96640 1 --2- 172.21.15.7:0/1238141039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e9010c0c0 0x7f2e9010e580 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009900 tx=0x7f2e8802a480 comp rx=0 tx=0).stop 2024-09-10T10:48:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.059+0000 7f2e96b96640 1 -- 172.21.15.7:0/1238141039 shutdown_connections 2024-09-10T10:48:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.059+0000 7f2e96b96640 1 --2- 172.21.15.7:0/1238141039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e9010c0c0 0x7f2e9010e580 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.059+0000 7f2e96b96640 1 --2- 172.21.15.7:0/1238141039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e901035c0 0x7f2e9010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.059+0000 7f2e96b96640 1 --2- 172.21.15.7:0/1238141039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e90102ca0 0x7f2e90103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.059+0000 7f2e96b96640 1 -- 172.21.15.7:0/1238141039 >> 172.21.15.7:0/1238141039 conn(0x7f2e900fc6f0 msgr2=0x7f2e900feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.059+0000 7f2e96b96640 1 -- 172.21.15.7:0/1238141039 shutdown_connections 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.059+0000 7f2e96b96640 1 -- 172.21.15.7:0/1238141039 wait complete. 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e96b96640 1 Processor -- start 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e96b96640 1 -- start start 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e96b96640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e90102ca0 0x7f2e90198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e96b96640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e901035c0 0x7f2e901987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e96b96640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010c0c0 0x7f2e9019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e96b96640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e90111ec0 con 0x7f2e901035c0 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e96b96640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e90111d40 con 0x7f2e90102ca0 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e96b96640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e90112040 con 0x7f2e9010c0c0 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e901035c0 0x7f2e901987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e901035c0 0x7f2e901987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54106/0 (socket says 172.21.15.7:54106) 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e94932640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e90102ca0 0x7f2e90198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e8ffff640 1 -- 172.21.15.7:0/4044043098 learned_addr learned my addr 172.21.15.7:0/4044043098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e95133640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010c0c0 0x7f2e9019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e8ffff640 1 -- 172.21.15.7:0/4044043098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010c0c0 msgr2=0x7f2e9019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010c0c0 0x7f2e9019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e8ffff640 1 -- 172.21.15.7:0/4044043098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e90102ca0 msgr2=0x7f2e90198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e90102ca0 0x7f2e90198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.060+0000 7f2e8ffff640 1 -- 172.21.15.7:0/4044043098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9019d280 con 0x7f2e901035c0 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.061+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e901035c0 0x7f2e901987d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e9f0 tx=0x7f2e7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.061+0000 7f2e8dffb640 1 -- 172.21.15.7:0/4044043098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800cd80 con 0x7f2e901035c0 2024-09-10T10:48:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.061+0000 7f2e96b96640 1 -- 172.21.15.7:0/4044043098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9019d570 con 0x7f2e901035c0 2024-09-10T10:48:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.061+0000 7f2e8dffb640 1 -- 172.21.15.7:0/4044043098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e78005500 con 0x7f2e901035c0 2024-09-10T10:48:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.061+0000 7f2e8dffb640 1 -- 172.21.15.7:0/4044043098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78010690 con 0x7f2e901035c0 2024-09-10T10:48:10.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.061+0000 7f2e96b96640 1 -- 172.21.15.7:0/4044043098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e901a4e00 con 0x7f2e901035c0 2024-09-10T10:48:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.063+0000 7f2e8dffb640 1 -- 172.21.15.7:0/4044043098 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2e780046c0 con 0x7f2e901035c0 2024-09-10T10:48:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e99a08740 1 Processor -- start 2024-09-10T10:48:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e99a08740 1 -- start start 2024-09-10T10:48:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e8dffb640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2e7c0a2da0 0x7f2e7c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8b410 0x55ad6db95770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db95cb0 0x55ad6db980a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e99a08740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ad6daef2f0 con 0x55ad6db8aaf0 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e99a08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ad6da97270 con 0x55ad6db95cb0 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e99a08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad6daeb2b0 con 0x55ad6db8b410 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e94932640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e8dffb640 1 -- 172.21.15.7:0/4044043098 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f2e780cf800 con 0x7f2e901035c0 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e95133640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2e7c0a2da0 0x7f2e7c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8b410 0x55ad6db95770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e94932640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54118/0 (socket says 172.21.15.7:54118) 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e94932640 1 -- 172.21.15.7:0/2561532962 learned_addr learned my addr 172.21.15.7:0/2561532962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.064+0000 7f2e95133640 1 --2- 172.21.15.7:0/2561532962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db95cb0 0x55ad6db980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e8ffff640 1 -- 172.21.15.7:0/2561532962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db95cb0 msgr2=0x55ad6db980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/2561532962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db95cb0 0x55ad6db980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e8ffff640 1 -- 172.21.15.7:0/2561532962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8aaf0 msgr2=0x55ad6db8aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/2561532962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e8ffff640 1 -- 172.21.15.7:0/2561532962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6da96340 con 0x55ad6db8b410 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e94932640 1 --2- 172.21.15.7:0/2561532962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e95133640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2e7c0a2da0 0x7f2e7c0a5260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2e88031550 tx=0x7f2e88031480 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/2561532962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8b410 0x55ad6db95770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e780ccf50 tx=0x7f2e7809b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ecc265dbebc54c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e86ffd640 1 -- 172.21.15.7:0/2561532962 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e780b4f20 con 0x55ad6db8b410 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e86ffd640 1 -- 172.21.15.7:0/2561532962 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7809c420 con 0x55ad6db8b410 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.065+0000 7f2e86ffd640 1 -- 172.21.15.7:0/2561532962 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e780b3170 con 0x55ad6db8b410 2024-09-10T10:48:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 -- 172.21.15.7:0/2561532962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8b410 msgr2=0x55ad6db95770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 --2- 172.21.15.7:0/2561532962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8b410 0x55ad6db95770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e780ccf50 tx=0x7f2e7809b860 comp rx=0 tx=0).stop 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 -- 172.21.15.7:0/2561532962 shutdown_connections 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 --2- 172.21.15.7:0/2561532962 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db95cb0 0x55ad6db980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 --2- 172.21.15.7:0/2561532962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8b410 0x55ad6db95770 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 --2- 172.21.15.7:0/2561532962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 -- 172.21.15.7:0/2561532962 >> 172.21.15.7:0/2561532962 conn(0x55ad6db87060 msgr2=0x55ad6db87440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 -- 172.21.15.7:0/2561532962 shutdown_connections 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 -- 172.21.15.7:0/2561532962 wait complete. 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 -- start start 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.066+0000 7f2e99a08740 1 Processor -- start 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db92bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 0x55ad6db93130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db93670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e99a08740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ad6db98880 con 0x55ad6db95cb0 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e99a08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ad6da78290 con 0x55ad6db8b410 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e99a08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad6db98a00 con 0x55ad6db8aaf0 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e99a08740 10 client.0 _ll_register_callbacks cb 0x55ad6daa9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e95133640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db93670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e94932640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db92bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e95133640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db93670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54132/0 (socket says 172.21.15.7:54132) 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 0x55ad6db93130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e95133640 1 -- 172.21.15.7:0/3289034144 learned_addr learned my addr 172.21.15.7:0/3289034144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e95133640 1 -- 172.21.15.7:0/3289034144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 msgr2=0x55ad6db92bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e95133640 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db92bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e95133640 1 -- 172.21.15.7:0/3289034144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 msgr2=0x55ad6db93130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e95133640 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 0x55ad6db93130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e95133640 1 -- 172.21.15.7:0/3289034144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6d802210 con 0x55ad6db95cb0 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e95133640 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db93670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2e88033000 tx=0x7f2e8802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13b83d5afae25cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.067+0000 7f2e6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3289034144 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88050020 con 0x55ad6db95cb0 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e99a08740 1 -- 172.21.15.7:0/3289034144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7809c420 con 0x55ad6db95cb0 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3289034144 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f2e880037f0 con 0x55ad6db95cb0 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3289034144 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2e8805b050 con 0x55ad6db95cb0 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e99a08740 10 client.15408 fetch_fsmap learned FSMap version 26 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e99a08740 1 -- 172.21.15.7:0/3289034144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ad6d8ce620 con 0x55ad6db95cb0 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3289034144 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7809c420 con 0x55ad6db95cb0 2024-09-10T10:48:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3289034144 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2e8804b9f0 con 0x55ad6db95cb0 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e99a08740 10 client.15408 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e99a08740 10 client.15408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e99a08740 20 client.15408 populate_metadata read hostname 'smithi007' 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad6dba5ca0 0x55ad6dba80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e99a08740 4 client.15408 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.068+0000 7f2e99a08740 1 -- 172.21.15.7:0/3289034144 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad6dafca20 con 0x55ad6dba5ca0 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.069+0000 7f2e94932640 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad6dba5ca0 0x55ad6dba80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.069+0000 7f2e6ffff640 10 client.15408 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.069+0000 7f2e94932640 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad6dba5ca0 0x55ad6dba80b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8760514f6cf298 server_cookie=57ec2707c23dc119 in_seq=0 out_seq=0 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3289034144 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad6dafca20 con 0x55ad6dba5ca0 2024-09-10T10:48:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e6ffff640 10 client.15408 handle_command_reply: tid=0 2024-09-10T10:48:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 client.15408 shutdown 2024-09-10T10:48:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 -- 172.21.15.7:0/3289034144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 msgr2=0x55ad6db93670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db93670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2e88033000 tx=0x7f2e8802a480 comp rx=0 tx=0).stop 2024-09-10T10:48:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 -- 172.21.15.7:0/3289034144 shutdown_connections 2024-09-10T10:48:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad6dba5ca0 0x55ad6dba80b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db93670 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 0x55ad6db93130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e6ffff640 0 client.15408 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3289034144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db92bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 -- 172.21.15.7:0/3289034144 >> 172.21.15.7:0/3289034144 conn(0x55ad6db87060 msgr2=0x55ad6db8cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 -- 172.21.15.7:0/3289034144 shutdown_connections 2024-09-10T10:48:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 1 -- 172.21.15.7:0/3289034144 wait complete. 2024-09-10T10:48:10.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.070+0000 7f2e99a08740 20 client.15408 trim_cache size 0 max 16384 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e99a08740 1 Processor -- start 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e99a08740 1 -- start start 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8b410 0x55ad6db934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db95cb0 0x55ad6dbad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e99a08740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901affc0 con 0x55ad6db8b410 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e99a08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901ad890 con 0x55ad6db95cb0 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e99a08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901ab6e0 con 0x55ad6db8aaf0 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8b410 0x55ad6db934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8b410 0x55ad6db934a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54148/0 (socket says 172.21.15.7:54148) 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e8ffff640 1 -- 172.21.15.7:0/3861923046 learned_addr learned my addr 172.21.15.7:0/3861923046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e95133640 1 --2- 172.21.15.7:0/3861923046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db95cb0 0x55ad6dbad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.085+0000 7f2e94932640 1 --2- 172.21.15.7:0/3861923046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.086+0000 7f2e8ffff640 1 -- 172.21.15.7:0/3861923046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 msgr2=0x55ad6db8d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.086+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/3861923046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.086+0000 7f2e8ffff640 1 -- 172.21.15.7:0/3861923046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db95cb0 msgr2=0x55ad6dbad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.086+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/3861923046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db95cb0 0x55ad6dbad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.086+0000 7f2e8ffff640 1 -- 172.21.15.7:0/3861923046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6dba8730 con 0x55ad6db8b410 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.086+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/3861923046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8b410 0x55ad6db934a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2e78099fd0 tx=0x7f2e7809b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c7fe1f84dc582e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.086+0000 7f2e867fc640 1 -- 172.21.15.7:0/3861923046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e780b4020 con 0x55ad6db8b410 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.086+0000 7f2e867fc640 1 -- 172.21.15.7:0/3861923046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7809bd20 con 0x55ad6db8b410 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.086+0000 7f2e867fc640 1 -- 172.21.15.7:0/3861923046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e780b9030 con 0x55ad6db8b410 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 -- 172.21.15.7:0/3861923046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8b410 msgr2=0x55ad6db934a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3861923046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8b410 0x55ad6db934a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2e78099fd0 tx=0x7f2e7809b990 comp rx=0 tx=0).stop 2024-09-10T10:48:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 -- 172.21.15.7:0/3861923046 shutdown_connections 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3861923046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db95cb0 0x55ad6dbad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3861923046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db8b410 0x55ad6db934a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3861923046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 -- 172.21.15.7:0/3861923046 >> 172.21.15.7:0/3861923046 conn(0x55ad6db87060 msgr2=0x55ad6db94c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 -- 172.21.15.7:0/3861923046 shutdown_connections 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 -- 172.21.15.7:0/3861923046 wait complete. 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 -- start start 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 Processor -- start 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 0x55ad6db901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db90700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901abd50 con 0x55ad6db95cb0 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901ad1c0 con 0x55ad6db8b410 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.087+0000 7f2e99a08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901ac420 con 0x55ad6db8aaf0 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e99a08740 10 client.0 _ll_register_callbacks cb 0x7f2e88032d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e95133640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db90700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e94932640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 0x55ad6db901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e95133640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db90700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54160/0 (socket says 172.21.15.7:54160) 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e95133640 1 -- 172.21.15.7:0/3098459932 learned_addr learned my addr 172.21.15.7:0/3098459932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e94932640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:51552/0 (socket says 172.21.15.7:51552) 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e8ffff640 1 -- 172.21.15.7:0/3098459932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 msgr2=0x55ad6db8fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e8ffff640 1 -- 172.21.15.7:0/3098459932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 msgr2=0x55ad6db90700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db90700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e8ffff640 1 -- 172.21.15.7:0/3098459932 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6db87cf0 con 0x55ad6db8b410 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e95133640 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db90700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e8ffff640 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 0x55ad6db901c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e7809bee0 tx=0x7f2e78005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8a3194055151f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3098459932 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e780b9070 con 0x55ad6db8b410 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3098459932 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f2e780a4df0 con 0x55ad6db8b410 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e99a08740 1 -- 172.21.15.7:0/3098459932 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ad6daf6d70 con 0x55ad6db8b410 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3098459932 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2e780b4030 con 0x55ad6db8b410 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e99a08740 10 client.25100 fetch_fsmap learned FSMap version 26 2024-09-10T10:48:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.088+0000 7f2e99a08740 1 -- 172.21.15.7:0/3098459932 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ad6dc10b60 con 0x55ad6db8b410 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3098459932 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ad6daf6d70 con 0x55ad6db8b410 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3098459932 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2e7809b170 con 0x55ad6db8b410 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e99a08740 10 client.25100 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e99a08740 10 client.25100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e99a08740 20 client.25100 populate_metadata read hostname 'smithi007' 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad6dc10de0 0x55ad6dc131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e99a08740 4 client.25100 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e99a08740 1 -- 172.21.15.7:0/3098459932 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad6da73900 con 0x55ad6dc10de0 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e94932640 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad6dc10de0 0x55ad6dc131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e94932640 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad6dc10de0 0x55ad6dc131d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5239fb80f077113b server_cookie=5bfd171919fc1ed4 in_seq=0 out_seq=0 2024-09-10T10:48:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.089+0000 7f2e6ffff640 10 client.25100 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e6ffff640 1 -- 172.21.15.7:0/3098459932 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad6da73900 con 0x55ad6dc10de0 2024-09-10T10:48:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e6ffff640 10 client.25100 handle_command_reply: tid=0 2024-09-10T10:48:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 client.25100 shutdown 2024-09-10T10:48:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 -- 172.21.15.7:0/3098459932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 msgr2=0x55ad6db901c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 0x55ad6db901c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e7809bee0 tx=0x7f2e78005720 comp rx=0 tx=0).stop 2024-09-10T10:48:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 -- 172.21.15.7:0/3098459932 shutdown_connections 2024-09-10T10:48:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad6dc10de0 0x55ad6dc131d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad6db95cb0 0x55ad6db90700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad6db8b410 0x55ad6db901c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 --2- 172.21.15.7:0/3098459932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad6db8aaf0 0x55ad6db8fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 -- 172.21.15.7:0/3098459932 >> 172.21.15.7:0/3098459932 conn(0x55ad6db87060 msgr2=0x55ad6db859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e6ffff640 0 client.25100 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 -- 172.21.15.7:0/3098459932 shutdown_connections 2024-09-10T10:48:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.164+0000 7f2e99a08740 1 -- 172.21.15.7:0/3098459932 wait complete. 2024-09-10T10:48:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e99a08740 20 client.25100 trim_cache size 0 max 16384 2024-09-10T10:48:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:48:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:48:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:48:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:48:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 -- 172.21.15.7:0/4044043098 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2e7c0a2da0 msgr2=0x7f2e7c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2e7c0a2da0 0x7f2e7c0a5260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2e88031550 tx=0x7f2e88031480 comp rx=0 tx=0).stop 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 -- 172.21.15.7:0/4044043098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e901035c0 msgr2=0x7f2e901987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e901035c0 0x7f2e901987d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e9f0 tx=0x7f2e7800edf0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 -- 172.21.15.7:0/4044043098 shutdown_connections 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010c0c0 0x7f2e9019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2e7c0a2da0 0x7f2e7c0a5260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e901035c0 0x7f2e901987d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 --2- 172.21.15.7:0/4044043098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e90102ca0 0x7f2e90198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 -- 172.21.15.7:0/4044043098 >> 172.21.15.7:0/4044043098 conn(0x7f2e900fc6f0 msgr2=0x7f2e900fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.165+0000 7f2e96b96640 1 -- 172.21.15.7:0/4044043098 shutdown_connections 2024-09-10T10:48:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:10.166+0000 7f2e96b96640 1 -- 172.21.15.7:0/4044043098 wait complete. 2024-09-10T10:48:10.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:09 smithi103 ceph-mon[35160]: pgmap v70: 97 pgs: 97 active+clean; 59 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 341 op/s 2024-09-10T10:48:10.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 173644 13.62 MB/sec warmup 70 sec latency 22.941 ms 2024-09-10T10:48:11.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 175490 13.57 MB/sec warmup 71 sec latency 13.201 ms 2024-09-10T10:48:11.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:11 smithi007 ceph-mon[30873]: pgmap v71: 97 pgs: 97 active+clean; 64 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 487 op/s 2024-09-10T10:48:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:11 smithi133 ceph-mon[35282]: pgmap v71: 97 pgs: 97 active+clean; 64 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 487 op/s 2024-09-10T10:48:12.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:11 smithi103 ceph-mon[35160]: pgmap v71: 97 pgs: 97 active+clean; 64 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 487 op/s 2024-09-10T10:48:12.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 177817 13.57 MB/sec warmup 72 sec latency 27.535 ms 2024-09-10T10:48:13.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 180571 13.57 MB/sec warmup 73 sec latency 47.011 ms 2024-09-10T10:48:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:13 smithi007 ceph-mon[30873]: pgmap v72: 97 pgs: 97 active+clean; 64 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 319 op/s 2024-09-10T10:48:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:13 smithi133 ceph-mon[35282]: pgmap v72: 97 pgs: 97 active+clean; 64 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 319 op/s 2024-09-10T10:48:14.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:13 smithi103 ceph-mon[35160]: pgmap v72: 97 pgs: 97 active+clean; 64 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 319 op/s 2024-09-10T10:48:14.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 182418 13.53 MB/sec warmup 74 sec latency 12.340 ms 2024-09-10T10:48:15.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 184857 13.54 MB/sec warmup 75 sec latency 18.601 ms 2024-09-10T10:48:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:15 smithi133 ceph-mon[35282]: pgmap v73: 97 pgs: 97 active+clean; 64 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 319 op/s 2024-09-10T10:48:16.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:15 smithi103 ceph-mon[35160]: pgmap v73: 97 pgs: 97 active+clean; 64 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 319 op/s 2024-09-10T10:48:16.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:15 smithi007 ceph-mon[30873]: pgmap v73: 97 pgs: 97 active+clean; 64 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 319 op/s 2024-09-10T10:48:16.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 187622 13.53 MB/sec warmup 76 sec latency 12.959 ms 2024-09-10T10:48:16.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:16 smithi133 ceph-mon[35282]: pgmap v74: 97 pgs: 97 active+clean; 61 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 512 op/s 2024-09-10T10:48:17.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:16 smithi103 ceph-mon[35160]: pgmap v74: 97 pgs: 97 active+clean; 61 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 512 op/s 2024-09-10T10:48:17.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:16 smithi007 ceph-mon[30873]: pgmap v74: 97 pgs: 97 active+clean; 61 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 512 op/s 2024-09-10T10:48:17.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 189512 13.50 MB/sec warmup 77 sec latency 11.696 ms 2024-09-10T10:48:18.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 191918 13.51 MB/sec warmup 78 sec latency 61.294 ms 2024-09-10T10:48:19.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:18 smithi103 ceph-mon[35160]: pgmap v75: 97 pgs: 97 active+clean; 61 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-09-10T10:48:19.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:18 smithi007 ceph-mon[30873]: pgmap v75: 97 pgs: 97 active+clean; 61 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-09-10T10:48:19.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:18 smithi133 ceph-mon[35282]: pgmap v75: 97 pgs: 97 active+clean; 61 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 339 op/s 2024-09-10T10:48:19.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 194477 13.49 MB/sec warmup 79 sec latency 15.302 ms 2024-09-10T10:48:20.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 196328 13.44 MB/sec warmup 80 sec latency 11.719 ms 2024-09-10T10:48:21.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:20 smithi103 ceph-mon[35160]: pgmap v76: 97 pgs: 97 active+clean; 66 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 483 op/s 2024-09-10T10:48:21.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:20 smithi007 ceph-mon[30873]: pgmap v76: 97 pgs: 97 active+clean; 66 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 483 op/s 2024-09-10T10:48:21.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:20 smithi133 ceph-mon[35282]: pgmap v76: 97 pgs: 97 active+clean; 66 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.5 MiB/s wr, 483 op/s 2024-09-10T10:48:21.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 199012 13.48 MB/sec warmup 81 sec latency 13.843 ms 2024-09-10T10:48:22.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 201649 13.47 MB/sec warmup 82 sec latency 11.366 ms 2024-09-10T10:48:23.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:22 smithi103 ceph-mon[35160]: pgmap v77: 97 pgs: 97 active+clean; 66 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-09-10T10:48:23.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:22 smithi007 ceph-mon[30873]: pgmap v77: 97 pgs: 97 active+clean; 66 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-09-10T10:48:23.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:22 smithi133 ceph-mon[35282]: pgmap v77: 97 pgs: 97 active+clean; 66 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-09-10T10:48:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.261+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1096518746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bc010c0e0 msgr2=0x7f0bc010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.261+0000 7f0bc66c0640 1 --2- 172.21.15.7:0/1096518746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bc010c0e0 0x7f0bc010e550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009ca0 tx=0x7f0bb802a970 comp rx=0 tx=0).stop 2024-09-10T10:48:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.261+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1096518746 shutdown_connections 2024-09-10T10:48:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.261+0000 7f0bc66c0640 1 --2- 172.21.15.7:0/1096518746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bc010c0e0 0x7f0bc010e550 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.261+0000 7f0bc66c0640 1 --2- 172.21.15.7:0/1096518746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bc01035e0 0x7f0bc010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.261+0000 7f0bc66c0640 1 --2- 172.21.15.7:0/1096518746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc0102cc0 0x7f0bc01030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.261+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1096518746 >> 172.21.15.7:0/1096518746 conn(0x7f0bc0077990 msgr2=0x7f0bc00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.261+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1096518746 shutdown_connections 2024-09-10T10:48:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.261+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1096518746 wait complete. 2024-09-10T10:48:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.262+0000 7f0bc66c0640 1 Processor -- start 2024-09-10T10:48:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.262+0000 7f0bc66c0640 1 -- start start 2024-09-10T10:48:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.262+0000 7f0bc66c0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bc0102cc0 0x7f0bc019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.262+0000 7f0bc66c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc01035e0 0x7f0bc019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.262+0000 7f0bc66c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bc010c0e0 0x7f0bc019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.262+0000 7f0bc66c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0111ee0 con 0x7f0bc01035e0 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.262+0000 7f0bc66c0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0111d60 con 0x7f0bc0102cc0 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.262+0000 7f0bc66c0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0112060 con 0x7f0bc010c0e0 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bbf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc01035e0 0x7f0bc019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bbf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc01035e0 0x7f0bc019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58368/0 (socket says 172.21.15.7:58368) 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bc0102cc0 0x7f0bc019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bbf7fe640 1 -- 172.21.15.7:0/1162550192 learned_addr learned my addr 172.21.15.7:0/1162550192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bc4c5d640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bc010c0e0 0x7f0bc019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bbf7fe640 1 -- 172.21.15.7:0/1162550192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bc010c0e0 msgr2=0x7f0bc019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bbf7fe640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bc010c0e0 0x7f0bc019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bbf7fe640 1 -- 172.21.15.7:0/1162550192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bc0102cc0 msgr2=0x7f0bc019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bbf7fe640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bc0102cc0 0x7f0bc019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bbf7fe640 1 -- 172.21.15.7:0/1162550192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc019f520 con 0x7f0bc01035e0 2024-09-10T10:48:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.263+0000 7f0bc4c5d640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bc010c0e0 0x7f0bc019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:48:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.264+0000 7f0bbf7fe640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc01035e0 0x7f0bc019aa70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400d770 tx=0x7f0bb400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.264+0000 7f0bbd7fa640 1 -- 172.21.15.7:0/1162550192 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4017070 con 0x7f0bc01035e0 2024-09-10T10:48:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.264+0000 7f0bbd7fa640 1 -- 172.21.15.7:0/1162550192 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb4005500 con 0x7f0bc01035e0 2024-09-10T10:48:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.264+0000 7f0bbd7fa640 1 -- 172.21.15.7:0/1162550192 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4004e00 con 0x7f0bc01035e0 2024-09-10T10:48:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.264+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1162550192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc019f810 con 0x7f0bc01035e0 2024-09-10T10:48:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.264+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1162550192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc01a70a0 con 0x7f0bc01035e0 2024-09-10T10:48:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.268+0000 7f0bc9454740 1 Processor -- start 2024-09-10T10:48:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.268+0000 7f0bbd7fa640 1 -- 172.21.15.7:0/1162550192 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0bb4004fa0 con 0x7f0bc01035e0 2024-09-10T10:48:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.268+0000 7f0bc9454740 1 -- start start 2024-09-10T10:48:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.268+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 0x562cc7aa4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.268+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7aa5330 0x562cc7ab7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.268+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7ab84c0 0x562cc7aba8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.268+0000 7f0bc9454740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562cc7a092f0 con 0x562cc7aa5330 2024-09-10T10:48:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.268+0000 7f0bc9454740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562cc79b1270 con 0x562cc7ab84c0 2024-09-10T10:48:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.268+0000 7f0bc9454740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562cc7a052b0 con 0x562cc7aa4a10 2024-09-10T10:48:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7aa5330 0x562cc7ab7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7aa5330 0x562cc7ab7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58382/0 (socket says 172.21.15.7:58382) 2024-09-10T10:48:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbffff640 1 -- 172.21.15.7:0/4145880028 learned_addr learned my addr 172.21.15.7:0/4145880028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbf7fe640 1 --2- 172.21.15.7:0/4145880028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7ab84c0 0x562cc7aba8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbffff640 1 -- 172.21.15.7:0/4145880028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 msgr2=0x562cc7aa4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbffff640 1 --2- 172.21.15.7:0/4145880028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 0x562cc7aa4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbffff640 1 -- 172.21.15.7:0/4145880028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7ab84c0 msgr2=0x562cc7aba8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbffff640 1 --2- 172.21.15.7:0/4145880028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7ab84c0 0x562cc7aba8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbffff640 1 -- 172.21.15.7:0/4145880028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cc79b0340 con 0x562cc7aa5330 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbd7fa640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0ba80a2e20 0x7f0ba80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbffff640 1 --2- 172.21.15.7:0/4145880028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7aa5330 0x562cc7ab7f80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009cf0 tx=0x7f0bb0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cfda6d56def861 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bc4c5d640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0ba80a2e20 0x7f0ba80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.269+0000 7f0bbd7fa640 1 -- 172.21.15.7:0/1162550192 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f0bb40cedb0 con 0x7f0bc01035e0 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.270+0000 7f0ba67fc640 1 -- 172.21.15.7:0/4145880028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0004230 con 0x562cc7aa5330 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.270+0000 7f0ba67fc640 1 -- 172.21.15.7:0/4145880028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb00043d0 con 0x562cc7aa5330 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.271+0000 7f0ba67fc640 1 -- 172.21.15.7:0/4145880028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0019960 con 0x562cc7aa5330 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.271+0000 7f0bc9454740 1 -- 172.21.15.7:0/4145880028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7aa5330 msgr2=0x562cc7ab7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.271+0000 7f0bc4c5d640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0ba80a2e20 0x7f0ba80a52e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8005e30 tx=0x7f0bb8005d80 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.271+0000 7f0bc9454740 1 --2- 172.21.15.7:0/4145880028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7aa5330 0x562cc7ab7f80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009cf0 tx=0x7f0bb0011810 comp rx=0 tx=0).stop 2024-09-10T10:48:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.272+0000 7f0bc9454740 1 -- 172.21.15.7:0/4145880028 shutdown_connections 2024-09-10T10:48:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.272+0000 7f0bc9454740 1 --2- 172.21.15.7:0/4145880028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7ab84c0 0x562cc7aba8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.272+0000 7f0bc9454740 1 --2- 172.21.15.7:0/4145880028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7aa5330 0x562cc7ab7f80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.272+0000 7f0bc9454740 1 --2- 172.21.15.7:0/4145880028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 0x562cc7aa4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.272+0000 7f0bc9454740 1 -- 172.21.15.7:0/4145880028 >> 172.21.15.7:0/4145880028 conn(0x562cc7aa1060 msgr2=0x562cc7aa1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.272+0000 7f0bc9454740 1 -- 172.21.15.7:0/4145880028 shutdown_connections 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.272+0000 7f0bc9454740 1 -- 172.21.15.7:0/4145880028 wait complete. 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.273+0000 7f0bc9454740 1 -- start start 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.273+0000 7f0bc9454740 1 Processor -- start 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.273+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 0x562cc7ab5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.273+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 0x562cc7ab5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.273+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7ab5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.273+0000 7f0bc9454740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562cc7abb0e0 con 0x562cc7ab84c0 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.273+0000 7f0bc9454740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562cc7992290 con 0x562cc7aa4a10 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.273+0000 7f0bc9454740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562cc7abb260 con 0x562cc7aa5330 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.273+0000 7f0bc9454740 10 client.0 _ll_register_callbacks cb 0x562cc79c3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bbf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 0x562cc7ab5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bbf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 0x562cc7ab5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:53556/0 (socket says 172.21.15.7:53556) 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bbf7fe640 1 -- 172.21.15.7:0/679618644 learned_addr learned my addr 172.21.15.7:0/679618644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bbffff640 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 0x562cc7ab5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bc4c5d640 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7ab5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bbf7fe640 1 -- 172.21.15.7:0/679618644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 msgr2=0x562cc7ab5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bbf7fe640 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 0x562cc7ab5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bbf7fe640 1 -- 172.21.15.7:0/679618644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 msgr2=0x562cc7ab5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bbf7fe640 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7ab5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bbf7fe640 1 -- 172.21.15.7:0/679618644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cc771c210 con 0x562cc7aa5330 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bc4c5d640 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7ab5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0bbf7fe640 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 0x562cc7ab5870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40cc560 tx=0x7f0bb40d24f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b79cedf6612db00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0b9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:48:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.274+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/679618644 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4020070 con 0x562cc7aa5330 2024-09-10T10:48:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.275+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/679618644 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f0bb40cc780 con 0x562cc7aa5330 2024-09-10T10:48:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.275+0000 7f0bc9454740 1 -- 172.21.15.7:0/679618644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb00043d0 con 0x562cc7aa5330 2024-09-10T10:48:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.275+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/679618644 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bb409ae10 con 0x562cc7aa5330 2024-09-10T10:48:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.275+0000 7f0bc9454740 10 client.25111 fetch_fsmap learned FSMap version 26 2024-09-10T10:48:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.275+0000 7f0bc9454740 1 -- 172.21.15.7:0/679618644 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562cc77e8620 con 0x562cc7aa5330 2024-09-10T10:48:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.275+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/679618644 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb00043d0 con 0x562cc7aa5330 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.276+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/679618644 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0bb4097940 con 0x562cc7aa5330 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.276+0000 7f0bc9454740 10 client.25111 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.276+0000 7f0bc9454740 10 client.25111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.276+0000 7f0bc9454740 20 client.25111 populate_metadata read hostname 'smithi007' 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.276+0000 7f0bc9454740 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562cc7ac84c0 0x562cc7aca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.276+0000 7f0bc9454740 4 client.25111 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.276+0000 7f0bc9454740 1 -- 172.21.15.7:0/679618644 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cc7a16a20 con 0x562cc7ac84c0 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.276+0000 7f0bbffff640 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562cc7ac84c0 0x562cc7aca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.277+0000 7f0bbffff640 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562cc7ac84c0 0x562cc7aca8d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=227d1e197e92668c server_cookie=1f3961b2e395386c in_seq=0 out_seq=0 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.277+0000 7f0b9f7fe640 10 client.25111 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/679618644 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cc7a16a20 con 0x562cc7ac84c0 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0b9f7fe640 10 client.25111 handle_command_reply: tid=0 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 client.25111 shutdown 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 -- 172.21.15.7:0/679618644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 msgr2=0x562cc7ab5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 0x562cc7ab5870 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40cc560 tx=0x7f0bb40d24f0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 -- 172.21.15.7:0/679618644 shutdown_connections 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562cc7ac84c0 0x562cc7aca8d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7ab5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0b9f7fe640 0 client.25111 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 0x562cc7ab5870 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 --2- 172.21.15.7:0/679618644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 0x562cc7ab5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 -- 172.21.15.7:0/679618644 >> 172.21.15.7:0/679618644 conn(0x562cc7aa1060 msgr2=0x562cc7aa6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 -- 172.21.15.7:0/679618644 shutdown_connections 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 1 -- 172.21.15.7:0/679618644 wait complete. 2024-09-10T10:48:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.278+0000 7f0bc9454740 20 client.25111 trim_cache size 0 max 16384 2024-09-10T10:48:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.306+0000 7f0bc9454740 1 Processor -- start 2024-09-10T10:48:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.306+0000 7f0bc9454740 1 -- start start 2024-09-10T10:48:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.306+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 0x562cc7ac4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.306+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa5330 0x562cc7ad01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7ad0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 0x562cc7ac4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 0x562cc7ac4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:53560/0 (socket says 172.21.15.7:53560) 2024-09-10T10:48:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bbf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa5330 0x562cc7ad01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bbf7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa5330 0x562cc7ad01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:54982/0 (socket says 172.21.15.7:54982) 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bbffff640 1 -- 172.21.15.7:0/3583986330 learned_addr learned my addr 172.21.15.7:0/3583986330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bc9454740 1 -- 172.21.15.7:0/3583986330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01adf60 con 0x562cc7ab84c0 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bc9454740 1 -- 172.21.15.7:0/3583986330 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01ab830 con 0x562cc7aa5330 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bc9454740 1 -- 172.21.15.7:0/3583986330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01a96c0 con 0x562cc7aa4a10 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bc4c5d640 1 --2- 172.21.15.7:0/3583986330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7ad0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bbffff640 1 -- 172.21.15.7:0/3583986330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa5330 msgr2=0x562cc7ad01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bbffff640 1 --2- 172.21.15.7:0/3583986330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa5330 0x562cc7ad01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bbffff640 1 -- 172.21.15.7:0/3583986330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 msgr2=0x562cc7ad0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bbffff640 1 --2- 172.21.15.7:0/3583986330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7ad0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bbffff640 1 -- 172.21.15.7:0/3583986330 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cc7ab42c0 con 0x562cc7aa4a10 2024-09-10T10:48:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.307+0000 7f0bc4c5d640 1 --2- 172.21.15.7:0/3583986330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7ad0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:48:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.308+0000 7f0bbffff640 1 --2- 172.21.15.7:0/3583986330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 0x562cc7ac4aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0014040 tx=0x7f0bb0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f8e4d177096b7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.308+0000 7f0ba5ffb640 1 -- 172.21.15.7:0/3583986330 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb002f020 con 0x562cc7aa4a10 2024-09-10T10:48:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.308+0000 7f0ba5ffb640 1 -- 172.21.15.7:0/3583986330 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0005520 con 0x562cc7aa4a10 2024-09-10T10:48:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.308+0000 7f0ba5ffb640 1 -- 172.21.15.7:0/3583986330 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0003a10 con 0x562cc7aa4a10 2024-09-10T10:48:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.309+0000 7f0bc9454740 1 -- 172.21.15.7:0/3583986330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 msgr2=0x562cc7ac4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.309+0000 7f0bc9454740 1 --2- 172.21.15.7:0/3583986330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 0x562cc7ac4aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0014040 tx=0x7f0bb0011810 comp rx=0 tx=0).stop 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.309+0000 7f0bc9454740 1 -- 172.21.15.7:0/3583986330 shutdown_connections 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.309+0000 7f0bc9454740 1 --2- 172.21.15.7:0/3583986330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7ad0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.309+0000 7f0bc9454740 1 --2- 172.21.15.7:0/3583986330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa5330 0x562cc7ad01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.309+0000 7f0bc9454740 1 --2- 172.21.15.7:0/3583986330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa4a10 0x562cc7ac4aa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.309+0000 7f0bc9454740 1 -- 172.21.15.7:0/3583986330 >> 172.21.15.7:0/3583986330 conn(0x562cc7aa1060 msgr2=0x562cc7aa01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.309+0000 7f0bc9454740 1 -- 172.21.15.7:0/3583986330 shutdown_connections 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.309+0000 7f0bc9454740 1 -- 172.21.15.7:0/3583986330 wait complete. 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.309+0000 7f0bc9454740 1 -- start start 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bc9454740 1 Processor -- start 2024-09-10T10:48:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 0x562cc7ac0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 0x562cc7ac0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bc9454740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7abb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bc9454740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01a9d10 con 0x562cc7ab84c0 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bc9454740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01ab160 con 0x562cc7aa4a10 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bc9454740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc01aa3c0 con 0x562cc7aa5330 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bc9454740 10 client.0 _ll_register_callbacks cb 0x7f0bb40d25f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 0x562cc7ac0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bbffff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 0x562cc7ac0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:54988/0 (socket says 172.21.15.7:54988) 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bbffff640 1 -- 172.21.15.7:0/715128014 learned_addr learned my addr 172.21.15.7:0/715128014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bc4c5d640 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7abb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bbf7fe640 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 0x562cc7ac0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bbffff640 1 -- 172.21.15.7:0/715128014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 msgr2=0x562cc7ac0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bbffff640 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 0x562cc7ac0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bbffff640 1 -- 172.21.15.7:0/715128014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 msgr2=0x562cc7abb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bbffff640 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7abb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.310+0000 7f0bbffff640 1 -- 172.21.15.7:0/715128014 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cc7ab31e0 con 0x562cc7aa4a10 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0bbffff640 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 0x562cc7ac0580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0005aa0 tx=0x7f0bb0005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9947c9f60b51447 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0b9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/715128014 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb002f030 con 0x562cc7aa4a10 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/715128014 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8520+0+0 (secure 0 0 0) 0x7f0bb0003690 con 0x562cc7aa4a10 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0bc9454740 1 -- 172.21.15.7:0/715128014 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb0005520 con 0x562cc7aa4a10 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/715128014 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bb0046050 con 0x562cc7aa4a10 2024-09-10T10:48:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0bc9454740 10 client.25118 fetch_fsmap learned FSMap version 26 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0bc9454740 1 -- 172.21.15.7:0/715128014 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562cc7b33640 con 0x562cc7aa4a10 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/715128014 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0005520 con 0x562cc7aa4a10 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/715128014 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0bb0034420 con 0x562cc7aa4a10 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0bc9454740 10 client.25118 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0bc9454740 10 client.25118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0bc9454740 20 client.25118 populate_metadata read hostname 'smithi007' 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0bc9454740 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562cc7b33b70 0x562cc7b35f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0bc9454740 4 client.25118 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.311+0000 7f0bc9454740 1 -- 172.21.15.7:0/715128014 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cc7a10d70 con 0x562cc7b33b70 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.312+0000 7f0bbf7fe640 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562cc7b33b70 0x562cc7b35f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.312+0000 7f0bbf7fe640 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562cc7b33b70 0x562cc7b35f90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=339f6f408204e265 server_cookie=4430c29fa9b12d26 in_seq=0 out_seq=0 2024-09-10T10:48:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.312+0000 7f0b9f7fe640 10 client.25118 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0b9f7fe640 1 -- 172.21.15.7:0/715128014 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562cc7a10d70 con 0x562cc7b33b70 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0b9f7fe640 10 client.25118 handle_command_reply: tid=0 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 client.25118 shutdown 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 -- 172.21.15.7:0/715128014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 msgr2=0x562cc7ac0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 0x562cc7ac0580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0005aa0 tx=0x7f0bb0005ad0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 -- 172.21.15.7:0/715128014 shutdown_connections 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562cc7b33b70 0x562cc7b35f90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562cc7ab84c0 0x562cc7abb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562cc7aa5330 0x562cc7ac0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0b9f7fe640 0 client.25118 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 --2- 172.21.15.7:0/715128014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562cc7aa4a10 0x562cc7ac0580 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 -- 172.21.15.7:0/715128014 >> 172.21.15.7:0/715128014 conn(0x562cc7aa1060 msgr2=0x562cc7ab6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 -- 172.21.15.7:0/715128014 shutdown_connections 2024-09-10T10:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 1 -- 172.21.15.7:0/715128014 wait complete. 2024-09-10T10:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.391+0000 7f0bc9454740 20 client.25118 trim_cache size 0 max 16384 2024-09-10T10:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:48:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:48:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1162550192 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0ba80a2e20 msgr2=0x7f0ba80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0ba80a2e20 0x7f0ba80a52e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8005e30 tx=0x7f0bb8005d80 comp rx=0 tx=0).stop 2024-09-10T10:48:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1162550192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc01035e0 msgr2=0x7f0bc019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc01035e0 0x7f0bc019aa70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400d770 tx=0x7f0bb400dc30 comp rx=0 tx=0).stop 2024-09-10T10:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1162550192 shutdown_connections 2024-09-10T10:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bc010c0e0 0x7f0bc019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0ba80a2e20 0x7f0ba80a52e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc01035e0 0x7f0bc019aa70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 --2- 172.21.15.7:0/1162550192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bc0102cc0 0x7f0bc019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1162550192 >> 172.21.15.7:0/1162550192 conn(0x7f0bc0077990 msgr2=0x7f0bc00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1162550192 shutdown_connections 2024-09-10T10:48:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:23.392+0000 7f0bc66c0640 1 -- 172.21.15.7:0/1162550192 wait complete. 2024-09-10T10:48:23.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 203164 13.40 MB/sec warmup 83 sec latency 46.100 ms 2024-09-10T10:48:23.894 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:48:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: ignoring --setuser ceph since I am not root 2024-09-10T10:48:23.894 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:48:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: ignoring --setgroup ceph since I am not root 2024-09-10T10:48:23.894 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:48:23 smithi133 ceph-mds[65473]: -- 172.21.15.133:0/3434916493 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e31f5ddc20 con 0x55e31f5bb000 2024-09-10T10:48:23.894 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:48:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: starting mds.f at 2024-09-10T10:48:24.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:23 smithi103 ceph-mon[35160]: daemon mds.f restarted 2024-09-10T10:48:24.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:23 smithi007 ceph-mon[30873]: daemon mds.f restarted 2024-09-10T10:48:24.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:23 smithi133 ceph-mon[35282]: daemon mds.f restarted 2024-09-10T10:48:24.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 206017 13.45 MB/sec warmup 84 sec latency 16.260 ms 2024-09-10T10:48:25.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:24 smithi103 ceph-mon[35160]: pgmap v78: 97 pgs: 97 active+clean; 66 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-09-10T10:48:25.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:24 smithi103 ceph-mon[35160]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T10:48:25.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:24 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6834/203237970,v1:172.21.15.133:6835/203237970] up:boot 2024-09-10T10:48:25.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:24 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T10:48:25.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:24 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:48:25.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:24 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:48:25.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:24 smithi007 ceph-mon[30873]: pgmap v78: 97 pgs: 97 active+clean; 66 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-09-10T10:48:25.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:24 smithi007 ceph-mon[30873]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T10:48:25.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:24 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6834/203237970,v1:172.21.15.133:6835/203237970] up:boot 2024-09-10T10:48:25.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:24 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T10:48:25.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:24 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:48:25.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:24 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:48:25.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:24 smithi133 ceph-mon[35282]: pgmap v78: 97 pgs: 97 active+clean; 66 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-09-10T10:48:25.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:24 smithi133 ceph-mon[35282]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T10:48:25.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:24 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6834/203237970,v1:172.21.15.133:6835/203237970] up:boot 2024-09-10T10:48:25.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:24 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T10:48:25.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:24 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:48:25.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:24 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:48:25.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 208487 13.42 MB/sec warmup 85 sec latency 16.728 ms 2024-09-10T10:48:26.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 210462 13.39 MB/sec warmup 86 sec latency 10.677 ms 2024-09-10T10:48:27.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:26 smithi103 ceph-mon[35160]: pgmap v80: 97 pgs: 97 active+clean; 59 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-10T10:48:27.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:26 smithi007 ceph-mon[30873]: pgmap v80: 97 pgs: 97 active+clean; 59 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-10T10:48:27.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:26 smithi133 ceph-mon[35282]: pgmap v80: 97 pgs: 97 active+clean; 59 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-10T10:48:27.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 213211 13.43 MB/sec warmup 87 sec latency 15.265 ms 2024-09-10T10:48:28.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 215643 13.40 MB/sec warmup 88 sec latency 52.248 ms 2024-09-10T10:48:29.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:28 smithi103 ceph-mon[35160]: pgmap v81: 97 pgs: 97 active+clean; 59 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-10T10:48:29.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:28 smithi007 ceph-mon[30873]: pgmap v81: 97 pgs: 97 active+clean; 59 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-10T10:48:29.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:28 smithi133 ceph-mon[35282]: pgmap v81: 97 pgs: 97 active+clean; 59 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-10T10:48:29.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 217610 13.37 MB/sec warmup 89 sec latency 10.459 ms 2024-09-10T10:48:30.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 220276 13.40 MB/sec warmup 90 sec latency 12.895 ms 2024-09-10T10:48:31.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:30 smithi103 ceph-mon[35160]: pgmap v82: 97 pgs: 97 active+clean; 65 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-09-10T10:48:31.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:30 smithi007 ceph-mon[30873]: pgmap v82: 97 pgs: 97 active+clean; 65 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-09-10T10:48:31.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:30 smithi133 ceph-mon[35282]: pgmap v82: 97 pgs: 97 active+clean; 65 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-09-10T10:48:31.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 222858 13.39 MB/sec warmup 91 sec latency 12.497 ms 2024-09-10T10:48:32.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1 224898 13.38 MB/sec warmup 92 sec latency 16.339 ms 2024-09-10T10:48:33.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:32 smithi103 ceph-mon[35160]: pgmap v83: 97 pgs: 97 active+clean; 65 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-09-10T10:48:33.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:32 smithi007 ceph-mon[30873]: pgmap v83: 97 pgs: 97 active+clean; 65 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-09-10T10:48:33.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:32 smithi133 ceph-mon[35282]: pgmap v83: 97 pgs: 97 active+clean; 65 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-09-10T10:48:33.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 227383 13.38 MB/sec warmup 93 sec latency 47.024 ms 2024-09-10T10:48:34.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 229933 13.36 MB/sec warmup 94 sec latency 13.735 ms 2024-09-10T10:48:35.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 231791 13.33 MB/sec warmup 95 sec latency 16.581 ms 2024-09-10T10:48:35.874 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:35 smithi103 ceph-mon[35160]: pgmap v84: 97 pgs: 97 active+clean; 65 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-09-10T10:48:35.885 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:35 smithi007 ceph-mon[30873]: pgmap v84: 97 pgs: 97 active+clean; 65 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-09-10T10:48:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:35 smithi133 ceph-mon[35282]: pgmap v84: 97 pgs: 97 active+clean; 65 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-09-10T10:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.487+0000 7fe300e77640 1 -- 172.21.15.7:0/1431105347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2fc1035e0 msgr2=0x7fe2fc10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.487+0000 7fe300e77640 1 --2- 172.21.15.7:0/1431105347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2fc1035e0 0x7fe2fc10bac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009900 tx=0x7fe2f002a480 comp rx=0 tx=0).stop 2024-09-10T10:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.487+0000 7fe300e77640 1 -- 172.21.15.7:0/1431105347 shutdown_connections 2024-09-10T10:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.487+0000 7fe300e77640 1 --2- 172.21.15.7:0/1431105347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2fc10c110 0x7fe2fc10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.487+0000 7fe300e77640 1 --2- 172.21.15.7:0/1431105347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2fc1035e0 0x7fe2fc10bac0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.487+0000 7fe300e77640 1 --2- 172.21.15.7:0/1431105347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2fc102cc0 0x7fe2fc1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.487+0000 7fe300e77640 1 -- 172.21.15.7:0/1431105347 >> 172.21.15.7:0/1431105347 conn(0x7fe2fc0fc6b0 msgr2=0x7fe2fc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:36.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.487+0000 7fe300e77640 1 -- 172.21.15.7:0/1431105347 shutdown_connections 2024-09-10T10:48:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.487+0000 7fe300e77640 1 -- 172.21.15.7:0/1431105347 wait complete. 2024-09-10T10:48:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.488+0000 7fe300e77640 1 Processor -- start 2024-09-10T10:48:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.488+0000 7fe300e77640 1 -- start start 2024-09-10T10:48:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.488+0000 7fe300e77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2fc102cc0 0x7fe2fc1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.488+0000 7fe300e77640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2fc1035e0 0x7fe2fc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.488+0000 7fe300e77640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2fc10c110 0x7fe2fc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe300e77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc111f40 con 0x7fe2fc102cc0 2024-09-10T10:48:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe300e77640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc111dc0 con 0x7fe2fc10c110 2024-09-10T10:48:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe300e77640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1120c0 con 0x7fe2fc1035e0 2024-09-10T10:48:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2fc102cc0 0x7fe2fc1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2fc102cc0 0x7fe2fc1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35852/0 (socket says 172.21.15.7:35852) 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fa59c640 1 -- 172.21.15.7:0/1520917698 learned_addr learned my addr 172.21.15.7:0/1520917698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2fc1035e0 0x7fe2fc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2fc10c110 0x7fe2fc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fa59c640 1 -- 172.21.15.7:0/1520917698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2fc1035e0 msgr2=0x7fe2fc1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fa59c640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2fc1035e0 0x7fe2fc1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fa59c640 1 -- 172.21.15.7:0/1520917698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2fc10c110 msgr2=0x7fe2fc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fa59c640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2fc10c110 0x7fe2fc19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fa59c640 1 -- 172.21.15.7:0/1520917698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc19d3a0 con 0x7fe2fc102cc0 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2fc10c110 0x7fe2fc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2fa59c640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2fc102cc0 0x7fe2fc1983b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00e9f0 tx=0x7fe2ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2eb7fe640 1 -- 172.21.15.7:0/1520917698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00cd80 con 0x7fe2fc102cc0 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2eb7fe640 1 -- 172.21.15.7:0/1520917698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ec005500 con 0x7fe2fc102cc0 2024-09-10T10:48:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe300e77640 1 -- 172.21.15.7:0/1520917698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2fc19d690 con 0x7fe2fc102cc0 2024-09-10T10:48:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe2eb7fe640 1 -- 172.21.15.7:0/1520917698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec010690 con 0x7fe2fc102cc0 2024-09-10T10:48:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.489+0000 7fe300e77640 1 -- 172.21.15.7:0/1520917698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2fc1a4f20 con 0x7fe2fc102cc0 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.492+0000 7fe2eb7fe640 1 -- 172.21.15.7:0/1520917698 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe2ec0046c0 con 0x7fe2fc102cc0 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.492+0000 7fe303c12740 1 Processor -- start 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.492+0000 7fe303c12740 1 -- start start 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe303c12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1caf0 0x558e53e1ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe303c12740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e27770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe303c12740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e27cb0 0x558e53e2a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe303c12740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558e53d812f0 con 0x558e53e1d410 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe303c12740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e53d29270 con 0x558e53e27cb0 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe303c12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558e53d7d2b0 con 0x558e53e1caf0 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e27770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1caf0 0x558e53e1ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e27770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35864/0 (socket says 172.21.15.7:35864) 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fa59c640 1 -- 172.21.15.7:0/1275409216 learned_addr learned my addr 172.21.15.7:0/1275409216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/1275409216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e27cb0 0x558e53e2a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fad9d640 1 -- 172.21.15.7:0/1275409216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e27cb0 msgr2=0x558e53e2a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/1275409216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e27cb0 0x558e53e2a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fad9d640 1 -- 172.21.15.7:0/1275409216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 msgr2=0x558e53e27770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/1275409216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e27770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fad9d640 1 -- 172.21.15.7:0/1275409216 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e53d28340 con 0x558e53e1caf0 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2eb7fe640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe2e40a2e20 0x7fe2e40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/1275409216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1caf0 0x558e53e1ced0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4005fd0 tx=0x7fe2f4011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d1523493940367 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.493+0000 7fe2fa59c640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe2e40a2e20 0x7fe2e40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.494+0000 7fe2eb7fe640 1 -- 172.21.15.7:0/1520917698 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7fe2ec0cf7d0 con 0x7fe2fc102cc0 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.494+0000 7fe2e8ff9640 1 -- 172.21.15.7:0/1275409216 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4004d20 con 0x558e53e1caf0 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.494+0000 7fe2e8ff9640 1 -- 172.21.15.7:0/1275409216 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f400f9a0 con 0x558e53e1caf0 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.494+0000 7fe2e8ff9640 1 -- 172.21.15.7:0/1275409216 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40045b0 con 0x558e53e1caf0 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.494+0000 7fe303c12740 1 -- 172.21.15.7:0/1275409216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1caf0 msgr2=0x558e53e1ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.494+0000 7fe303c12740 1 --2- 172.21.15.7:0/1275409216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1caf0 0x558e53e1ced0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4005fd0 tx=0x7fe2f4011bf0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.494+0000 7fe2fa59c640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe2e40a2e20 0x7fe2e40a52e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00e9c0 tx=0x7fe2ec0d5550 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 -- 172.21.15.7:0/1275409216 shutdown_connections 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 --2- 172.21.15.7:0/1275409216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e27cb0 0x558e53e2a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 --2- 172.21.15.7:0/1275409216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e27770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 --2- 172.21.15.7:0/1275409216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1caf0 0x558e53e1ced0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 -- 172.21.15.7:0/1275409216 >> 172.21.15.7:0/1275409216 conn(0x558e53e19060 msgr2=0x558e53e19440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 -- 172.21.15.7:0/1275409216 shutdown_connections 2024-09-10T10:48:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 -- 172.21.15.7:0/1275409216 wait complete. 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 -- start start 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 Processor -- start 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1caf0 0x558e53e24b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 0x558e53e250c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e27cb0 0x558e53e25600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558e53e2a880 con 0x558e53e1caf0 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e53d0a290 con 0x558e53e27cb0 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558e53e2aa00 con 0x558e53e1d410 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe303c12740 10 client.0 _ll_register_callbacks cb 0x558e53d3b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe2fad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1caf0 0x558e53e24b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe2fad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1caf0 0x558e53e24b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35872/0 (socket says 172.21.15.7:35872) 2024-09-10T10:48:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.495+0000 7fe2fad9d640 1 -- 172.21.15.7:0/2737045615 learned_addr learned my addr 172.21.15.7:0/2737045615 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 0x558e53e250c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e27cb0 0x558e53e25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2fad9d640 1 -- 172.21.15.7:0/2737045615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 msgr2=0x558e53e250c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 0x558e53e250c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2fad9d640 1 -- 172.21.15.7:0/2737045615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e27cb0 msgr2=0x558e53e25600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e27cb0 0x558e53e25600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2fad9d640 1 -- 172.21.15.7:0/2737045615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e53a94210 con 0x558e53e1caf0 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1caf0 0x558e53e24b80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400f790 tx=0x7fe2f4004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27c3cdf8b12aaf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/2737045615 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f401e040 con 0x558e53e1caf0 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/2737045615 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7fe2f4004a40 con 0x558e53e1caf0 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe303c12740 1 -- 172.21.15.7:0/2737045615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2f400f9a0 con 0x558e53e1caf0 2024-09-10T10:48:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/2737045615 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe2f4004c40 con 0x558e53e1caf0 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe303c12740 10 client.15450 fetch_fsmap learned FSMap version 28 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.496+0000 7fe303c12740 1 -- 172.21.15.7:0/2737045615 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e53b60620 con 0x558e53e1caf0 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/2737045615 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f400f9a0 con 0x558e53e1caf0 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/2737045615 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe2f4018070 con 0x558e53e1caf0 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe303c12740 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe303c12740 10 client.15450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe303c12740 20 client.15450 populate_metadata read hostname 'smithi007' 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe303c12740 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e53e37ba0 0x558e53e39f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe303c12740 4 client.15450 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe303c12740 1 -- 172.21.15.7:0/2737045615 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e53d8ea20 con 0x558e53e37ba0 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e53e37ba0 0x558e53e39f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe2e1ffb640 10 client.15450 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.497+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e53e37ba0 0x558e53e39f90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=789143d72fb6a6b6 server_cookie=751268a6c2e1759b in_seq=0 out_seq=0 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.498+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/2737045615 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e53d8ea20 con 0x558e53e37ba0 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.498+0000 7fe2e1ffb640 10 client.15450 handle_command_reply: tid=0 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.498+0000 7fe303c12740 1 client.15450 shutdown 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.498+0000 7fe303c12740 1 -- 172.21.15.7:0/2737045615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1caf0 msgr2=0x558e53e24b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.498+0000 7fe303c12740 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1caf0 0x558e53e24b80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400f790 tx=0x7fe2f4004230 comp rx=0 tx=0).stop 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.499+0000 7fe303c12740 1 -- 172.21.15.7:0/2737045615 shutdown_connections 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.499+0000 7fe303c12740 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e53e37ba0 0x558e53e39f90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.499+0000 7fe303c12740 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e27cb0 0x558e53e25600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.499+0000 7fe2e1ffb640 0 client.15450 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.499+0000 7fe303c12740 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 0x558e53e250c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.499+0000 7fe303c12740 1 --2- 172.21.15.7:0/2737045615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1caf0 0x558e53e24b80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.499+0000 7fe303c12740 1 -- 172.21.15.7:0/2737045615 >> 172.21.15.7:0/2737045615 conn(0x558e53e19060 msgr2=0x558e53e1edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.499+0000 7fe303c12740 1 -- 172.21.15.7:0/2737045615 shutdown_connections 2024-09-10T10:48:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.499+0000 7fe303c12740 1 -- 172.21.15.7:0/2737045615 wait complete. 2024-09-10T10:48:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.499+0000 7fe303c12740 20 client.15450 trim_cache size 0 max 16384 2024-09-10T10:48:36.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 234469 13.36 MB/sec warmup 96 sec latency 19.587 ms 2024-09-10T10:48:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe303c12740 1 Processor -- start 2024-09-10T10:48:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe303c12740 1 -- start start 2024-09-10T10:48:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe303c12740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e1caf0 0x558e53e25430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe303c12740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e3f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe303c12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e27cb0 0x558e53e3fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe303c12740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1b0100 con 0x558e53e1d410 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe303c12740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1ad970 con 0x558e53e1caf0 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe303c12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1ab7c0 con 0x558e53e27cb0 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe2f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e3f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe2fad9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e1caf0 0x558e53e25430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe2f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e3f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35886/0 (socket says 172.21.15.7:35886) 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe2f9d9b640 1 -- 172.21.15.7:0/1636718794 learned_addr learned my addr 172.21.15.7:0/1636718794 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.514+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/1636718794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e27cb0 0x558e53e3fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.515+0000 7fe2f9d9b640 1 -- 172.21.15.7:0/1636718794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e27cb0 msgr2=0x558e53e3fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.515+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/1636718794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e27cb0 0x558e53e3fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.515+0000 7fe2f9d9b640 1 -- 172.21.15.7:0/1636718794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e1caf0 msgr2=0x558e53e25430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.515+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/1636718794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e1caf0 0x558e53e25430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.515+0000 7fe2f9d9b640 1 -- 172.21.15.7:0/1636718794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e53e18e90 con 0x558e53e1d410 2024-09-10T10:48:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.515+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/1636718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e3f700 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009230 tx=0x7fe2f0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5f532e711f9724 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.515+0000 7fe2e3fff640 1 -- 172.21.15.7:0/1636718794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0035040 con 0x558e53e1d410 2024-09-10T10:48:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.515+0000 7fe2e3fff640 1 -- 172.21.15.7:0/1636718794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f004b070 con 0x558e53e1d410 2024-09-10T10:48:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.515+0000 7fe2e3fff640 1 -- 172.21.15.7:0/1636718794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00315b0 con 0x558e53e1d410 2024-09-10T10:48:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 -- 172.21.15.7:0/1636718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 msgr2=0x558e53e3f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 --2- 172.21.15.7:0/1636718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e3f700 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009230 tx=0x7fe2f0005b50 comp rx=0 tx=0).stop 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 -- 172.21.15.7:0/1636718794 shutdown_connections 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 --2- 172.21.15.7:0/1636718794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e27cb0 0x558e53e3fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 --2- 172.21.15.7:0/1636718794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e1d410 0x558e53e3f700 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 --2- 172.21.15.7:0/1636718794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e1caf0 0x558e53e25430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 -- 172.21.15.7:0/1636718794 >> 172.21.15.7:0/1636718794 conn(0x558e53e19060 msgr2=0x558e53e26c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 -- 172.21.15.7:0/1636718794 shutdown_connections 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 -- 172.21.15.7:0/1636718794 wait complete. 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 -- start start 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 Processor -- start 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e1caf0 0x558e53e30270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 0x558e53e2b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e27cb0 0x558e53e2bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1abe30 con 0x558e53e27cb0 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1ad2a0 con 0x558e53e1caf0 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.516+0000 7fe303c12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1ac500 con 0x558e53e1d410 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe303c12740 10 client.0 _ll_register_callbacks cb 0x7fe2ec0d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2fad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e27cb0 0x558e53e2bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2f9d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 0x558e53e2b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2fad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e27cb0 0x558e53e2bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35888/0 (socket says 172.21.15.7:35888) 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2fad9d640 1 -- 172.21.15.7:0/4139126333 learned_addr learned my addr 172.21.15.7:0/4139126333 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e1caf0 0x558e53e30270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2fad9d640 1 -- 172.21.15.7:0/4139126333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 msgr2=0x558e53e2b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 0x558e53e2b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2fad9d640 1 -- 172.21.15.7:0/4139126333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e1caf0 msgr2=0x558e53e30270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e1caf0 0x558e53e30270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2fad9d640 1 -- 172.21.15.7:0/4139126333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e53e142d0 con 0x558e53e27cb0 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 0x558e53e2b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2fad9d640 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e27cb0 0x558e53e2bb20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4011730 tx=0x7fe2f4005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7716a01f00a6386 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/4139126333 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4037040 con 0x558e53e27cb0 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/4139126333 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7fe2f4004de0 con 0x558e53e27cb0 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.517+0000 7fe303c12740 1 -- 172.21.15.7:0/4139126333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2f004b070 con 0x558e53e27cb0 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/4139126333 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe2f4003f40 con 0x558e53e27cb0 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe303c12740 10 client.15456 fetch_fsmap learned FSMap version 28 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe303c12740 1 -- 172.21.15.7:0/4139126333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e53ea23a0 con 0x558e53e27cb0 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/4139126333 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f004b070 con 0x558e53e27cb0 2024-09-10T10:48:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/4139126333 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe2f4018070 con 0x558e53e27cb0 2024-09-10T10:48:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe303c12740 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:48:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe303c12740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe303c12740 20 client.15456 populate_metadata read hostname 'smithi007' 2024-09-10T10:48:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe303c12740 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e53ea2950 0x558e53ea4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe303c12740 4 client.15456 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:48:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe303c12740 1 -- 172.21.15.7:0/4139126333 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e53d88d70 con 0x558e53ea2950 2024-09-10T10:48:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.518+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e53ea2950 0x558e53ea4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.519+0000 7fe2e1ffb640 10 client.15456 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.519+0000 7fe2f9d9b640 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e53ea2950 0x558e53ea4d70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6576bc1f194772 server_cookie=148ce6958ab97e72 in_seq=0 out_seq=0 2024-09-10T10:48:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.583+0000 7fe2e1ffb640 1 -- 172.21.15.7:0/4139126333 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e53d88d70 con 0x558e53ea2950 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe2e1ffb640 10 client.15456 handle_command_reply: tid=0 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 client.15456 shutdown 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 -- 172.21.15.7:0/4139126333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e27cb0 msgr2=0x558e53e2bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e27cb0 0x558e53e2bb20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4011730 tx=0x7fe2f4005ab0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 -- 172.21.15.7:0/4139126333 shutdown_connections 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e53ea2950 0x558e53ea4d70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe2e1ffb640 0 client.15456 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e53e27cb0 0x558e53e2bb20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e53e1d410 0x558e53e2b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 --2- 172.21.15.7:0/4139126333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e53e1caf0 0x558e53e30270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 -- 172.21.15.7:0/4139126333 >> 172.21.15.7:0/4139126333 conn(0x558e53e19060 msgr2=0x558e53e1e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 -- 172.21.15.7:0/4139126333 shutdown_connections 2024-09-10T10:48:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 1 -- 172.21.15.7:0/4139126333 wait complete. 2024-09-10T10:48:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.584+0000 7fe303c12740 20 client.15456 trim_cache size 0 max 16384 2024-09-10T10:48:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:48:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:48:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:48:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 -- 172.21.15.7:0/1520917698 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe2e40a2e20 msgr2=0x7fe2e40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe2e40a2e20 0x7fe2e40a52e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00e9c0 tx=0x7fe2ec0d5550 comp rx=0 tx=0).stop 2024-09-10T10:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 -- 172.21.15.7:0/1520917698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2fc102cc0 msgr2=0x7fe2fc1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2fc102cc0 0x7fe2fc1983b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec00e9f0 tx=0x7fe2ec00edf0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 -- 172.21.15.7:0/1520917698 shutdown_connections 2024-09-10T10:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2fc10c110 0x7fe2fc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2fc1035e0 0x7fe2fc1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe2e40a2e20 0x7fe2e40a52e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 --2- 172.21.15.7:0/1520917698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2fc102cc0 0x7fe2fc1983b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 -- 172.21.15.7:0/1520917698 >> 172.21.15.7:0/1520917698 conn(0x7fe2fc0fc6b0 msgr2=0x7fe2fc0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 -- 172.21.15.7:0/1520917698 shutdown_connections 2024-09-10T10:48:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:36.585+0000 7fe300e77640 1 -- 172.21.15.7:0/1520917698 wait complete. 2024-09-10T10:48:37.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 236729 13.31 MB/sec warmup 97 sec latency 51.156 ms 2024-09-10T10:48:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:37 smithi103 ceph-mon[35160]: pgmap v85: 97 pgs: 97 active+clean; 70 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.8 MiB/s wr, 472 op/s 2024-09-10T10:48:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:37 smithi007 ceph-mon[30873]: pgmap v85: 97 pgs: 97 active+clean; 70 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.8 MiB/s wr, 472 op/s 2024-09-10T10:48:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:37 smithi133 ceph-mon[35282]: pgmap v85: 97 pgs: 97 active+clean; 70 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.8 MiB/s wr, 472 op/s 2024-09-10T10:48:38.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 238509 13.29 MB/sec warmup 98 sec latency 43.860 ms 2024-09-10T10:48:39.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 241544 13.34 MB/sec warmup 99 sec latency 17.964 ms 2024-09-10T10:48:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:39 smithi103 ceph-mon[35160]: pgmap v86: 97 pgs: 97 active+clean; 70 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-09-10T10:48:39.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:39 smithi007 ceph-mon[30873]: pgmap v86: 97 pgs: 97 active+clean; 70 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-09-10T10:48:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:39 smithi133 ceph-mon[35282]: pgmap v86: 97 pgs: 97 active+clean; 70 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2024-09-10T10:48:40.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 244116 13.33 MB/sec warmup 100 sec latency 15.908 ms 2024-09-10T10:48:41.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 246188 13.32 MB/sec warmup 101 sec latency 13.083 ms 2024-09-10T10:48:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:41 smithi103 ceph-mon[35160]: pgmap v87: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 472 op/s 2024-09-10T10:48:41.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:41 smithi007 ceph-mon[30873]: pgmap v87: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 472 op/s 2024-09-10T10:48:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:41 smithi133 ceph-mon[35282]: pgmap v87: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 472 op/s 2024-09-10T10:48:42.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 248617 13.33 MB/sec warmup 102 sec latency 57.767 ms 2024-09-10T10:48:43.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 250770 13.27 MB/sec warmup 103 sec latency 48.052 ms 2024-09-10T10:48:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:43 smithi103 ceph-mon[35160]: pgmap v88: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-09-10T10:48:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:43 smithi007 ceph-mon[30873]: pgmap v88: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-09-10T10:48:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:43 smithi133 ceph-mon[35282]: pgmap v88: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-09-10T10:48:44.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 252621 13.26 MB/sec warmup 104 sec latency 17.420 ms 2024-09-10T10:48:45.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 255640 13.31 MB/sec warmup 105 sec latency 13.308 ms 2024-09-10T10:48:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:45 smithi103 ceph-mon[35160]: pgmap v89: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-09-10T10:48:45.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:45 smithi007 ceph-mon[30873]: pgmap v89: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-09-10T10:48:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:45 smithi133 ceph-mon[35282]: pgmap v89: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 328 op/s 2024-09-10T10:48:46.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 257621 13.24 MB/sec warmup 106 sec latency 12.998 ms 2024-09-10T10:48:47.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 259533 13.24 MB/sec warmup 107 sec latency 16.775 ms 2024-09-10T10:48:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:47 smithi103 ceph-mon[35160]: pgmap v90: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 466 op/s 2024-09-10T10:48:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:47 smithi007 ceph-mon[30873]: pgmap v90: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 466 op/s 2024-09-10T10:48:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:47 smithi133 ceph-mon[35282]: pgmap v90: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 466 op/s 2024-09-10T10:48:48.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 262171 13.24 MB/sec warmup 108 sec latency 55.222 ms 2024-09-10T10:48:49.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 264018 13.22 MB/sec warmup 109 sec latency 21.361 ms 2024-09-10T10:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.677+0000 7f5b43482640 1 -- 172.21.15.7:0/2576364117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3c101350 msgr2=0x7f5b3c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.677+0000 7f5b43482640 1 --2- 172.21.15.7:0/2576364117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3c101350 0x7f5b3c10d880 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009cd0 tx=0x7f5b3402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.677+0000 7f5b43482640 1 -- 172.21.15.7:0/2576364117 shutdown_connections 2024-09-10T10:48:49.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.677+0000 7f5b43482640 1 --2- 172.21.15.7:0/2576364117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3c101350 0x7f5b3c10d880 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.677+0000 7f5b43482640 1 --2- 172.21.15.7:0/2576364117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c1009b0 0x7f5b3c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.677+0000 7f5b43482640 1 --2- 172.21.15.7:0/2576364117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3c1069c0 0x7f5b3c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.677+0000 7f5b43482640 1 -- 172.21.15.7:0/2576364117 >> 172.21.15.7:0/2576364117 conn(0x7f5b3c077990 msgr2=0x7f5b3c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.677+0000 7f5b43482640 1 -- 172.21.15.7:0/2576364117 shutdown_connections 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.677+0000 7f5b43482640 1 -- 172.21.15.7:0/2576364117 wait complete. 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b43482640 1 Processor -- start 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b43482640 1 -- start start 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b43482640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c1009b0 0x7f5b3c0fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b43482640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3c101350 0x7f5b3c0fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b43482640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3c1069c0 0x7f5b3c0fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b43482640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c110e00 con 0x7f5b3c101350 2024-09-10T10:48:49.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b43482640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c110c80 con 0x7f5b3c1009b0 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b43482640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c110f80 con 0x7f5b3c1069c0 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b40a1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3c101350 0x7f5b3c0fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b40a1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3c101350 0x7f5b3c0fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56254/0 (socket says 172.21.15.7:56254) 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b40a1d640 1 -- 172.21.15.7:0/1582011483 learned_addr learned my addr 172.21.15.7:0/1582011483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b4121e640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c1009b0 0x7f5b3c0fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b41a1f640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3c1069c0 0x7f5b3c0fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b40a1d640 1 -- 172.21.15.7:0/1582011483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3c1069c0 msgr2=0x7f5b3c0fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b40a1d640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3c1069c0 0x7f5b3c0fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b40a1d640 1 -- 172.21.15.7:0/1582011483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c1009b0 msgr2=0x7f5b3c0fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b40a1d640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c1009b0 0x7f5b3c0fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.678+0000 7f5b40a1d640 1 -- 172.21.15.7:0/1582011483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c1a67f0 con 0x7f5b3c101350 2024-09-10T10:48:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.679+0000 7f5b40a1d640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3c101350 0x7f5b3c0fd320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00e9f0 tx=0x7f5b2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.679+0000 7f5b327fc640 1 -- 172.21.15.7:0/1582011483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c00cd80 con 0x7f5b3c101350 2024-09-10T10:48:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.679+0000 7f5b327fc640 1 -- 172.21.15.7:0/1582011483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2c010910 con 0x7f5b3c101350 2024-09-10T10:48:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.679+0000 7f5b43482640 1 -- 172.21.15.7:0/1582011483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c1a6ae0 con 0x7f5b3c101350 2024-09-10T10:48:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.679+0000 7f5b327fc640 1 -- 172.21.15.7:0/1582011483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c018990 con 0x7f5b3c101350 2024-09-10T10:48:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.679+0000 7f5b43482640 1 -- 172.21.15.7:0/1582011483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c1a7030 con 0x7f5b3c101350 2024-09-10T10:48:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b46222740 1 Processor -- start 2024-09-10T10:48:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b327fc640 1 -- 172.21.15.7:0/1582011483 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f5b2c010ac0 con 0x7f5b3c101350 2024-09-10T10:48:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b46222740 1 -- start start 2024-09-10T10:48:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b46222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b46222740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 0x55af7f9ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b46222740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 0x55af7f9cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b46222740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55af7f9242f0 con 0x55af7f9cacb0 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b46222740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55af7f8cc270 con 0x55af7f9c0410 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b46222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55af7f9202b0 con 0x55af7f9bfaf0 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b41a1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b40a1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 0x55af7f9cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b41a1f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48954/0 (socket says 172.21.15.7:48954) 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b41a1f640 1 -- 172.21.15.7:0/3698537035 learned_addr learned my addr 172.21.15.7:0/3698537035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b4121e640 1 --2- 172.21.15.7:0/3698537035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 0x55af7f9ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b41a1f640 1 -- 172.21.15.7:0/3698537035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 msgr2=0x55af7f9ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b41a1f640 1 --2- 172.21.15.7:0/3698537035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 0x55af7f9ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b41a1f640 1 -- 172.21.15.7:0/3698537035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 msgr2=0x55af7f9cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b41a1f640 1 --2- 172.21.15.7:0/3698537035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 0x55af7f9cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b41a1f640 1 -- 172.21.15.7:0/3698537035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7f8cb340 con 0x55af7f9bfaf0 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.682+0000 7f5b327fc640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5b280a2da0 0x7f5b280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.683+0000 7f5b4121e640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5b280a2da0 0x7f5b280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.683+0000 7f5b41a1f640 1 --2- 172.21.15.7:0/3698537035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bfed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009cd0 tx=0x7f5b34009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c568b7f2a6699e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.683+0000 7f5b327fc640 1 -- 172.21.15.7:0/1582011483 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f5b2c0cf740 con 0x7f5b3c101350 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.683+0000 7f5b277fe640 1 -- 172.21.15.7:0/3698537035 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34048070 con 0x55af7f9bfaf0 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.683+0000 7f5b277fe640 1 -- 172.21.15.7:0/3698537035 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b34036070 con 0x55af7f9bfaf0 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.683+0000 7f5b277fe640 1 -- 172.21.15.7:0/3698537035 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34004400 con 0x55af7f9bfaf0 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 -- 172.21.15.7:0/3698537035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 msgr2=0x55af7f9bfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b4121e640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5b280a2da0 0x7f5b280a5260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55af7f9bcec0 tx=0x7f5b38005f90 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 --2- 172.21.15.7:0/3698537035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bfed0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009cd0 tx=0x7f5b34009a20 comp rx=0 tx=0).stop 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 -- 172.21.15.7:0/3698537035 shutdown_connections 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 --2- 172.21.15.7:0/3698537035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 0x55af7f9cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 --2- 172.21.15.7:0/3698537035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 0x55af7f9ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 --2- 172.21.15.7:0/3698537035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bfed0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 -- 172.21.15.7:0/3698537035 >> 172.21.15.7:0/3698537035 conn(0x55af7f9bc060 msgr2=0x55af7f9bc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 -- 172.21.15.7:0/3698537035 shutdown_connections 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 -- 172.21.15.7:0/3698537035 wait complete. 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 -- start start 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 Processor -- start 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.684+0000 7f5b46222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9c0410 0x55af7f9c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b46222740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9cacb0 0x55af7f9c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b46222740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55af7f9cd880 con 0x55af7f9bfaf0 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b46222740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55af7f8ad290 con 0x55af7f9cacb0 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b46222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55af7f9cda00 con 0x55af7f9c0410 2024-09-10T10:48:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b46222740 10 client.0 _ll_register_callbacks cb 0x55af7f8de140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b4121e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b4121e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56276/0 (socket says 172.21.15.7:56276) 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b4121e640 1 -- 172.21.15.7:0/2059472458 learned_addr learned my addr 172.21.15.7:0/2059472458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b40a1d640 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9c0410 0x55af7f9c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b41a1f640 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9cacb0 0x55af7f9c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b4121e640 1 -- 172.21.15.7:0/2059472458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9c0410 msgr2=0x55af7f9c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b4121e640 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9c0410 0x55af7f9c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b4121e640 1 -- 172.21.15.7:0/2059472458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9cacb0 msgr2=0x55af7f9c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b4121e640 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9cacb0 0x55af7f9c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b4121e640 1 -- 172.21.15.7:0/2059472458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7f637210 con 0x55af7f9bfaf0 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b4121e640 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9c7c00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5b380045f0 tx=0x7f5b38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4f7b2dc708264 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:49.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b24ff9640 1 -- 172.21.15.7:0/2059472458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3801f640 con 0x55af7f9bfaf0 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b24ff9640 1 -- 172.21.15.7:0/2059472458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f5b3801f7e0 con 0x55af7f9bfaf0 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.685+0000 7f5b46222740 1 -- 172.21.15.7:0/2059472458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b34036070 con 0x55af7f9bfaf0 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b24ff9640 1 -- 172.21.15.7:0/2059472458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b3801f9e0 con 0x55af7f9bfaf0 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b46222740 10 client.15468 fetch_fsmap learned FSMap version 28 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b46222740 1 -- 172.21.15.7:0/2059472458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af7f703620 con 0x55af7f9bfaf0 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b24ff9640 1 -- 172.21.15.7:0/2059472458 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b34036070 con 0x55af7f9bfaf0 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b24ff9640 1 -- 172.21.15.7:0/2059472458 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5b38027070 con 0x55af7f9bfaf0 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b46222740 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b46222740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:49.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b46222740 20 client.15468 populate_metadata read hostname 'smithi007' 2024-09-10T10:48:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b46222740 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55af7f9dad70 0x55af7f9dd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b46222740 4 client.15468 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:48:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b46222740 1 -- 172.21.15.7:0/2059472458 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af7f931a20 con 0x55af7f9dad70 2024-09-10T10:48:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.686+0000 7f5b40a1d640 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55af7f9dad70 0x55af7f9dd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.687+0000 7f5b24ff9640 10 client.15468 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:49.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.687+0000 7f5b40a1d640 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55af7f9dad70 0x55af7f9dd1a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e2d519509e0f69 server_cookie=fb1d6740a259ef82 in_seq=0 out_seq=0 2024-09-10T10:48:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.687+0000 7f5b24ff9640 1 -- 172.21.15.7:0/2059472458 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af7f931a20 con 0x55af7f9dad70 2024-09-10T10:48:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.687+0000 7f5b24ff9640 10 client.15468 handle_command_reply: tid=0 2024-09-10T10:48:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.687+0000 7f5b46222740 1 client.15468 shutdown 2024-09-10T10:48:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.687+0000 7f5b46222740 1 -- 172.21.15.7:0/2059472458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 msgr2=0x55af7f9c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b46222740 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9c7c00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5b380045f0 tx=0x7f5b38004730 comp rx=0 tx=0).stop 2024-09-10T10:48:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b46222740 1 -- 172.21.15.7:0/2059472458 shutdown_connections 2024-09-10T10:48:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b46222740 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55af7f9dad70 0x55af7f9dd1a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b46222740 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9cacb0 0x55af7f9c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b46222740 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9c0410 0x55af7f9c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b24ff9640 0 client.15468 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b46222740 1 --2- 172.21.15.7:0/2059472458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9c7c00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b46222740 1 -- 172.21.15.7:0/2059472458 >> 172.21.15.7:0/2059472458 conn(0x55af7f9bc060 msgr2=0x55af7f9bb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b46222740 1 -- 172.21.15.7:0/2059472458 shutdown_connections 2024-09-10T10:48:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b46222740 1 -- 172.21.15.7:0/2059472458 wait complete. 2024-09-10T10:48:49.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.688+0000 7f5b46222740 20 client.15468 trim_cache size 0 max 16384 2024-09-10T10:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b46222740 1 Processor -- start 2024-09-10T10:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b46222740 1 -- start start 2024-09-10T10:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b46222740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b46222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9c0410 0x55af7f9c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b46222740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9cacb0 0x55af7f9e2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b46222740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b2230 con 0x55af7f9bfaf0 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b46222740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1afaa0 con 0x55af7f9cacb0 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b46222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1ad8f0 con 0x55af7f9c0410 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b4121e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b4121e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56288/0 (socket says 172.21.15.7:56288) 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b4121e640 1 -- 172.21.15.7:0/4106141360 learned_addr learned my addr 172.21.15.7:0/4106141360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b41a1f640 1 --2- 172.21.15.7:0/4106141360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9cacb0 0x55af7f9e2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b40a1d640 1 --2- 172.21.15.7:0/4106141360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9c0410 0x55af7f9c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b4121e640 1 -- 172.21.15.7:0/4106141360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9c0410 msgr2=0x55af7f9c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b4121e640 1 --2- 172.21.15.7:0/4106141360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9c0410 0x55af7f9c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b4121e640 1 -- 172.21.15.7:0/4106141360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9cacb0 msgr2=0x55af7f9e2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b4121e640 1 --2- 172.21.15.7:0/4106141360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9cacb0 0x55af7f9e2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.703+0000 7f5b4121e640 1 -- 172.21.15.7:0/4106141360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7f9c6c10 con 0x55af7f9bfaf0 2024-09-10T10:48:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.704+0000 7f5b4121e640 1 --2- 172.21.15.7:0/4106141360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bc440 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5b38029e30 tx=0x7f5b38005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4037d43721bcc55e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.704+0000 7f5b26ffd640 1 -- 172.21.15.7:0/4106141360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3801f640 con 0x55af7f9bfaf0 2024-09-10T10:48:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.704+0000 7f5b26ffd640 1 -- 172.21.15.7:0/4106141360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b3801f7e0 con 0x55af7f9bfaf0 2024-09-10T10:48:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.704+0000 7f5b26ffd640 1 -- 172.21.15.7:0/4106141360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b380296a0 con 0x55af7f9bfaf0 2024-09-10T10:48:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.704+0000 7f5b46222740 1 -- 172.21.15.7:0/4106141360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 msgr2=0x55af7f9bc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.704+0000 7f5b46222740 1 --2- 172.21.15.7:0/4106141360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bc440 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5b38029e30 tx=0x7f5b38005b90 comp rx=0 tx=0).stop 2024-09-10T10:48:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 -- 172.21.15.7:0/4106141360 shutdown_connections 2024-09-10T10:48:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 --2- 172.21.15.7:0/4106141360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9cacb0 0x55af7f9e2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 --2- 172.21.15.7:0/4106141360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9c0410 0x55af7f9c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 --2- 172.21.15.7:0/4106141360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9bfaf0 0x55af7f9bc440 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 -- 172.21.15.7:0/4106141360 >> 172.21.15.7:0/4106141360 conn(0x55af7f9bc060 msgr2=0x55af7f9c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 -- 172.21.15.7:0/4106141360 shutdown_connections 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 -- 172.21.15.7:0/4106141360 wait complete. 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 -- start start 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 Processor -- start 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 0x55af7f9c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 0x55af7f9c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 0x55af7f9c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1adf60 con 0x55af7f9cacb0 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1af3d0 con 0x55af7f9c0410 2024-09-10T10:48:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1ae630 con 0x55af7f9bfaf0 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.705+0000 7f5b46222740 10 client.0 _ll_register_callbacks cb 0x7f5b380044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b41a1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 0x55af7f9c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b40a1d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 0x55af7f9c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b4121e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 0x55af7f9c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b41a1f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 0x55af7f9c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56296/0 (socket says 172.21.15.7:56296) 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b41a1f640 1 -- 172.21.15.7:0/25586555 learned_addr learned my addr 172.21.15.7:0/25586555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b40a1d640 1 -- 172.21.15.7:0/25586555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 msgr2=0x55af7f9c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b40a1d640 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 0x55af7f9c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b40a1d640 1 -- 172.21.15.7:0/25586555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 msgr2=0x55af7f9c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b40a1d640 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 0x55af7f9c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b40a1d640 1 -- 172.21.15.7:0/25586555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7fa44bb0 con 0x55af7f9c0410 2024-09-10T10:48:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b40a1d640 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 0x55af7f9c52e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0cce90 tx=0x7f5b2c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3095d3f6b6c71de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b24ff9640 1 -- 172.21.15.7:0/25586555 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c09bd10 con 0x55af7f9c0410 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b46222740 1 -- 172.21.15.7:0/25586555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b3801f7e0 con 0x55af7f9c0410 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b24ff9640 1 -- 172.21.15.7:0/25586555 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f5b2c098380 con 0x55af7f9c0410 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b24ff9640 1 -- 172.21.15.7:0/25586555 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b2c0b4e60 con 0x55af7f9c0410 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b46222740 10 client.25160 fetch_fsmap learned FSMap version 28 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.706+0000 7f5b46222740 1 -- 172.21.15.7:0/25586555 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af7fa45e60 con 0x55af7f9c0410 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b24ff9640 1 -- 172.21.15.7:0/25586555 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b2c0b1020 con 0x55af7f9c0410 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b24ff9640 1 -- 172.21.15.7:0/25586555 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5b2c09b160 con 0x55af7f9c0410 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b46222740 10 client.25160 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b46222740 10 client.25160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b46222740 20 client.25160 populate_metadata read hostname 'smithi007' 2024-09-10T10:48:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b46222740 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55af7fa465f0 0x55af7fa48a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:48:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b46222740 4 client.25160 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:48:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b46222740 1 -- 172.21.15.7:0/25586555 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af7f92bd70 con 0x55af7fa465f0 2024-09-10T10:48:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b4121e640 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55af7fa465f0 0x55af7fa48a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b4121e640 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55af7fa465f0 0x55af7fa48a10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9bbfd4e62fa367e server_cookie=de73cd548f8f4e3e in_seq=0 out_seq=0 2024-09-10T10:48:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.707+0000 7f5b24ff9640 10 client.25160 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b24ff9640 1 -- 172.21.15.7:0/25586555 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af7f92bd70 con 0x55af7fa465f0 2024-09-10T10:48:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b24ff9640 10 client.25160 handle_command_reply: tid=0 2024-09-10T10:48:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 client.25160 shutdown 2024-09-10T10:48:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 -- 172.21.15.7:0/25586555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 msgr2=0x55af7f9c52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 0x55af7f9c52e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0cce90 tx=0x7f5b2c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 -- 172.21.15.7:0/25586555 shutdown_connections 2024-09-10T10:48:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55af7fa465f0 0x55af7fa48a10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55af7f9cacb0 0x55af7f9c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55af7f9c0410 0x55af7f9c52e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b24ff9640 0 client.25160 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 --2- 172.21.15.7:0/25586555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af7f9bfaf0 0x55af7f9c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 -- 172.21.15.7:0/25586555 >> 172.21.15.7:0/25586555 conn(0x55af7f9bc060 msgr2=0x55af7f9c1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 -- 172.21.15.7:0/25586555 shutdown_connections 2024-09-10T10:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.778+0000 7f5b46222740 1 -- 172.21.15.7:0/25586555 wait complete. 2024-09-10T10:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b46222740 20 client.25160 trim_cache size 0 max 16384 2024-09-10T10:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:48:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:48:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:48:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:48:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b43482640 1 -- 172.21.15.7:0/1582011483 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5b280a2da0 msgr2=0x7f5b280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b43482640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5b280a2da0 0x7f5b280a5260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55af7f9bcec0 tx=0x7f5b38005f90 comp rx=0 tx=0).stop 2024-09-10T10:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b43482640 1 -- 172.21.15.7:0/1582011483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3c101350 msgr2=0x7f5b3c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b43482640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3c101350 0x7f5b3c0fd320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00e9f0 tx=0x7f5b2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b43482640 1 -- 172.21.15.7:0/1582011483 shutdown_connections 2024-09-10T10:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b43482640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b3c1069c0 0x7f5b3c0fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b43482640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5b280a2da0 0x7f5b280a5260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b43482640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b3c101350 0x7f5b3c0fd320 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b43482640 1 --2- 172.21.15.7:0/1582011483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c1009b0 0x7f5b3c0fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.779+0000 7f5b43482640 1 -- 172.21.15.7:0/1582011483 >> 172.21.15.7:0/1582011483 conn(0x7f5b3c077990 msgr2=0x7f5b3c069a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.780+0000 7f5b43482640 1 -- 172.21.15.7:0/1582011483 shutdown_connections 2024-09-10T10:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:48:49.780+0000 7f5b43482640 1 -- 172.21.15.7:0/1582011483 wait complete. 2024-09-10T10:48:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:49 smithi103 ceph-mon[35160]: pgmap v91: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-09-10T10:48:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:49 smithi007 ceph-mon[30873]: pgmap v91: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-09-10T10:48:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:49 smithi133 ceph-mon[35282]: pgmap v91: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-09-10T10:48:50.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 266432 13.23 MB/sec warmup 110 sec latency 48.625 ms 2024-09-10T10:48:51.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 268989 13.22 MB/sec warmup 111 sec latency 11.827 ms 2024-09-10T10:48:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:51 smithi103 ceph-mon[35160]: pgmap v92: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 504 op/s 2024-09-10T10:48:51.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:51 smithi007 ceph-mon[30873]: pgmap v92: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 504 op/s 2024-09-10T10:48:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:51 smithi133 ceph-mon[35282]: pgmap v92: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 504 op/s 2024-09-10T10:48:52.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 270670 13.18 MB/sec warmup 112 sec latency 10.713 ms 2024-09-10T10:48:53.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 273371 13.21 MB/sec warmup 113 sec latency 53.516 ms 2024-09-10T10:48:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:53 smithi103 ceph-mon[35160]: pgmap v93: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-10T10:48:53.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:53 smithi007 ceph-mon[30873]: pgmap v93: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-10T10:48:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:53 smithi133 ceph-mon[35282]: pgmap v93: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-10T10:48:54.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 275541 13.16 MB/sec warmup 114 sec latency 20.118 ms 2024-09-10T10:48:55.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 277298 13.15 MB/sec warmup 115 sec latency 17.989 ms 2024-09-10T10:48:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:55 smithi103 ceph-mon[35160]: pgmap v94: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-10T10:48:55.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:55 smithi007 ceph-mon[30873]: pgmap v94: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-10T10:48:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:55 smithi133 ceph-mon[35282]: pgmap v94: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-10T10:48:56.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 280389 13.19 MB/sec warmup 116 sec latency 12.013 ms 2024-09-10T10:48:57.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 282601 13.15 MB/sec warmup 117 sec latency 16.313 ms 2024-09-10T10:48:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:57 smithi103 ceph-mon[35160]: pgmap v95: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 446 op/s 2024-09-10T10:48:57.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:57 smithi007 ceph-mon[30873]: pgmap v95: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 446 op/s 2024-09-10T10:48:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:57 smithi133 ceph-mon[35282]: pgmap v95: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 446 op/s 2024-09-10T10:48:58.507 INFO:tasks.workunit.client.0.smithi007.stdout: 1 284507 13.14 MB/sec warmup 118 sec latency 12.517 ms 2024-09-10T10:48:59.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 287408 13.18 MB/sec warmup 119 sec latency 43.727 ms 2024-09-10T10:48:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:59 smithi103 ceph-mon[35160]: pgmap v96: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-09-10T10:48:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:48:59 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:48:59.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:59 smithi007 ceph-mon[30873]: pgmap v96: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-09-10T10:48:59.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:48:59 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:48:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:59 smithi133 ceph-mon[35282]: pgmap v96: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-09-10T10:48:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:48:59 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:49:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:00 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:49:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:00 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:49:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:00 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:49:00.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:00 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:49:00.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:00 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:49:00.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:00 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:49:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:00 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:49:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:00 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:49:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:00 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:49:01.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 291203 13.00 MB/sec execute 1 sec latency 29.911 ms 2024-09-10T10:49:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:01 smithi103 ceph-mon[35160]: pgmap v97: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 446 op/s 2024-09-10T10:49:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:01 smithi007 ceph-mon[30873]: pgmap v97: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 446 op/s 2024-09-10T10:49:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:01 smithi133 ceph-mon[35282]: pgmap v97: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 446 op/s 2024-09-10T10:49:02.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 293770 12.59 MB/sec execute 2 sec latency 13.184 ms 2024-09-10T10:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 -- 172.21.15.7:0/2652298936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c1069c0 msgr2=0x7fbb4c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 --2- 172.21.15.7:0/2652298936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c1069c0 0x7fbb4c106da0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009930 tx=0x7fbb4802a420 comp rx=0 tx=0).stop 2024-09-10T10:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 -- 172.21.15.7:0/2652298936 shutdown_connections 2024-09-10T10:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 --2- 172.21.15.7:0/2652298936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c101350 0x7fbb4c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 --2- 172.21.15.7:0/2652298936 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4c1009b0 0x7fbb4c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 --2- 172.21.15.7:0/2652298936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c1069c0 0x7fbb4c106da0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 -- 172.21.15.7:0/2652298936 >> 172.21.15.7:0/2652298936 conn(0x7fbb4c077990 msgr2=0x7fbb4c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 -- 172.21.15.7:0/2652298936 shutdown_connections 2024-09-10T10:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 -- 172.21.15.7:0/2652298936 wait complete. 2024-09-10T10:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 Processor -- start 2024-09-10T10:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.871+0000 7fbb53303640 1 -- start start 2024-09-10T10:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb53303640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c1009b0 0x7fbb4c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb53303640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c101350 0x7fbb4c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb53303640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4c1069c0 0x7fbb4c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb53303640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c110e00 con 0x7fbb4c1009b0 2024-09-10T10:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb53303640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c110c80 con 0x7fbb4c1069c0 2024-09-10T10:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb53303640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c110f80 con 0x7fbb4c101350 2024-09-10T10:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5109f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c1009b0 0x7fbb4c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5109f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c1009b0 0x7fbb4c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51952/0 (socket says 172.21.15.7:51952) 2024-09-10T10:49:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5109f640 1 -- 172.21.15.7:0/153335754 learned_addr learned my addr 172.21.15.7:0/153335754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5089e640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c101350 0x7fbb4c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb518a0640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4c1069c0 0x7fbb4c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5109f640 1 -- 172.21.15.7:0/153335754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c101350 msgr2=0x7fbb4c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5109f640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c101350 0x7fbb4c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5109f640 1 -- 172.21.15.7:0/153335754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4c1069c0 msgr2=0x7fbb4c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5109f640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4c1069c0 0x7fbb4c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5109f640 1 -- 172.21.15.7:0/153335754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c19f5b0 con 0x7fbb4c1009b0 2024-09-10T10:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5089e640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c101350 0x7fbb4c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb518a0640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4c1069c0 0x7fbb4c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb5109f640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c1009b0 0x7fbb4c19a5c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009230 tx=0x7fbb48004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb427fc640 1 -- 172.21.15.7:0/153335754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4803f030 con 0x7fbb4c1009b0 2024-09-10T10:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.872+0000 7fbb427fc640 1 -- 172.21.15.7:0/153335754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb48004440 con 0x7fbb4c1009b0 2024-09-10T10:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.873+0000 7fbb427fc640 1 -- 172.21.15.7:0/153335754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb48035070 con 0x7fbb4c1009b0 2024-09-10T10:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.873+0000 7fbb53303640 1 -- 172.21.15.7:0/153335754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4c19f840 con 0x7fbb4c1009b0 2024-09-10T10:49:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.873+0000 7fbb53303640 1 -- 172.21.15.7:0/153335754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4c1a7130 con 0x7fbb4c1009b0 2024-09-10T10:49:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb5609a740 1 Processor -- start 2024-09-10T10:49:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb427fc640 1 -- 172.21.15.7:0/153335754 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbb48042030 con 0x7fbb4c1009b0 2024-09-10T10:49:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb5609a740 1 -- start start 2024-09-10T10:49:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 0x55d1b0192ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0193410 0x55d1b019d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b019dcb0 0x55d1b01a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb5609a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d1b00f72f0 con 0x55d1b0193410 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb5609a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1b009f270 con 0x55d1b019dcb0 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb5609a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d1b00f32b0 con 0x55d1b0192af0 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb518a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 0x55d1b0192ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb518a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 0x55d1b0192ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:52308/0 (socket says 172.21.15.7:52308) 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb5109f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0193410 0x55d1b019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb518a0640 1 -- 172.21.15.7:0/3941370406 learned_addr learned my addr 172.21.15.7:0/3941370406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb5089e640 1 --2- 172.21.15.7:0/3941370406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b019dcb0 0x55d1b01a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb518a0640 1 -- 172.21.15.7:0/3941370406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b019dcb0 msgr2=0x55d1b01a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb518a0640 1 --2- 172.21.15.7:0/3941370406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b019dcb0 0x55d1b01a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb518a0640 1 -- 172.21.15.7:0/3941370406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0193410 msgr2=0x55d1b019d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb518a0640 1 --2- 172.21.15.7:0/3941370406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0193410 0x55d1b019d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.876+0000 7fbb518a0640 1 -- 172.21.15.7:0/3941370406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b009e340 con 0x55d1b0192af0 2024-09-10T10:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.877+0000 7fbb427fc640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb380a2e20 0x7fbb380a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.877+0000 7fbb518a0640 1 --2- 172.21.15.7:0/3941370406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 0x55d1b0192ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb44011790 tx=0x7fbb44011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da167711896fcfc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.877+0000 7fbb5109f640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb380a2e20 0x7fbb380a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.877+0000 7fbb377fe640 1 -- 172.21.15.7:0/3941370406 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44003c30 con 0x55d1b0192af0 2024-09-10T10:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.877+0000 7fbb427fc640 1 -- 172.21.15.7:0/153335754 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7fbb480ef690 con 0x7fbb4c1009b0 2024-09-10T10:49:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.877+0000 7fbb377fe640 1 -- 172.21.15.7:0/3941370406 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4400fe80 con 0x55d1b0192af0 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.877+0000 7fbb377fe640 1 -- 172.21.15.7:0/3941370406 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb440193e0 con 0x55d1b0192af0 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 -- 172.21.15.7:0/3941370406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 msgr2=0x55d1b0192ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 --2- 172.21.15.7:0/3941370406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 0x55d1b0192ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb44011790 tx=0x7fbb44011c50 comp rx=0 tx=0).stop 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5109f640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb380a2e20 0x7fbb380a52e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb480ece90 tx=0x7fbb480f5550 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 -- 172.21.15.7:0/3941370406 shutdown_connections 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 --2- 172.21.15.7:0/3941370406 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b019dcb0 0x55d1b01a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 --2- 172.21.15.7:0/3941370406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0193410 0x55d1b019d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 --2- 172.21.15.7:0/3941370406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 0x55d1b0192ed0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 -- 172.21.15.7:0/3941370406 >> 172.21.15.7:0/3941370406 conn(0x55d1b018f060 msgr2=0x55d1b018f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 -- 172.21.15.7:0/3941370406 shutdown_connections 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 -- 172.21.15.7:0/3941370406 wait complete. 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 -- start start 2024-09-10T10:49:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.878+0000 7fbb5609a740 1 Processor -- start 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0192af0 0x55d1b019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0193410 0x55d1b019b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b019b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb5609a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d1b01a0880 con 0x55d1b019dcb0 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb5609a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1b0080290 con 0x55d1b0192af0 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb5609a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d1b01a0a00 con 0x55d1b0193410 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb5609a740 10 client.0 _ll_register_callbacks cb 0x55d1b00b1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb518a0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b019b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb518a0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b019b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51984/0 (socket says 172.21.15.7:51984) 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb518a0640 1 -- 172.21.15.7:0/3062126502 learned_addr learned my addr 172.21.15.7:0/3062126502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb5089e640 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0193410 0x55d1b019b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb518a0640 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0192af0 0x55d1b019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb518a0640 1 -- 172.21.15.7:0/3062126502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0193410 msgr2=0x55d1b019b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb518a0640 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0193410 0x55d1b019b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb518a0640 1 -- 172.21.15.7:0/3062126502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0192af0 msgr2=0x55d1b019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb518a0640 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0192af0 0x55d1b019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb518a0640 1 -- 172.21.15.7:0/3062126502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1afe0a210 con 0x55d1b019dcb0 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb518a0640 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b019b5f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbb44011760 tx=0x7fbb44004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20e63d535418dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb34ff9640 1 -- 172.21.15.7:0/3062126502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44011260 con 0x55d1b019dcb0 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb34ff9640 1 -- 172.21.15.7:0/3062126502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7fbb44004a40 con 0x55d1b019dcb0 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.879+0000 7fbb5609a740 1 -- 172.21.15.7:0/3062126502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1b0104a20 con 0x55d1b019dcb0 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb34ff9640 1 -- 172.21.15.7:0/3062126502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb44004c40 con 0x55d1b019dcb0 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb5609a740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-09-10T10:49:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb5609a740 1 -- 172.21.15.7:0/3062126502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1afed6620 con 0x55d1b019dcb0 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb34ff9640 1 -- 172.21.15.7:0/3062126502 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d1b0104a20 con 0x55d1b019dcb0 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb34ff9640 1 -- 172.21.15.7:0/3062126502 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbb44018070 con 0x55d1b019dcb0 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb5609a740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb5609a740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb5609a740 20 client.15486 populate_metadata read hostname 'smithi007' 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb5609a740 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d1b01adb60 0x55d1b01aff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb5609a740 4 client.15486 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb5609a740 1 -- 172.21.15.7:0/3062126502 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1b00fed70 con 0x55d1b01adb60 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.880+0000 7fbb5089e640 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d1b01adb60 0x55d1b01aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.881+0000 7fbb34ff9640 10 client.15486 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.881+0000 7fbb5089e640 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d1b01adb60 0x55d1b01aff50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a9ca471b9c249 server_cookie=518d9940885ba865 in_seq=0 out_seq=0 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.881+0000 7fbb34ff9640 1 -- 172.21.15.7:0/3062126502 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1b00fed70 con 0x55d1b01adb60 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.881+0000 7fbb34ff9640 10 client.15486 handle_command_reply: tid=0 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.881+0000 7fbb5609a740 1 client.15486 shutdown 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 1 -- 172.21.15.7:0/3062126502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 msgr2=0x55d1b019b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b019b5f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbb44011760 tx=0x7fbb44004510 comp rx=0 tx=0).stop 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 1 -- 172.21.15.7:0/3062126502 shutdown_connections 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d1b01adb60 0x55d1b01aff50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b019b5f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0193410 0x55d1b019b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 1 --2- 172.21.15.7:0/3062126502 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0192af0 0x55d1b019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 1 -- 172.21.15.7:0/3062126502 >> 172.21.15.7:0/3062126502 conn(0x55d1b018f060 msgr2=0x55d1b0194dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb34ff9640 0 client.15486 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 1 -- 172.21.15.7:0/3062126502 shutdown_connections 2024-09-10T10:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 1 -- 172.21.15.7:0/3062126502 wait complete. 2024-09-10T10:49:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.882+0000 7fbb5609a740 20 client.15486 trim_cache size 0 max 16384 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb5609a740 1 Processor -- start 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb5609a740 1 -- start start 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0192af0 0x55d1b019b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0193410 0x55d1b01b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b019dcb0 0x55d1b01b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb5609a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1b22f0 con 0x55d1b0192af0 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb5609a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1afb60 con 0x55d1b019dcb0 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb5609a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1ad9b0 con 0x55d1b0193410 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb518a0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0192af0 0x55d1b019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb518a0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0192af0 0x55d1b019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51996/0 (socket says 172.21.15.7:51996) 2024-09-10T10:49:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb518a0640 1 -- 172.21.15.7:0/1314320398 learned_addr learned my addr 172.21.15.7:0/1314320398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb5089e640 1 --2- 172.21.15.7:0/1314320398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0193410 0x55d1b01b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb518a0640 1 -- 172.21.15.7:0/1314320398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0193410 msgr2=0x55d1b01b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb518a0640 1 --2- 172.21.15.7:0/1314320398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0193410 0x55d1b01b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb518a0640 1 -- 172.21.15.7:0/1314320398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b019dcb0 msgr2=0x55d1b01b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb518a0640 1 --2- 172.21.15.7:0/1314320398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b019dcb0 0x55d1b01b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb518a0640 1 -- 172.21.15.7:0/1314320398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1b018ee90 con 0x55d1b0192af0 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.897+0000 7fbb518a0640 1 --2- 172.21.15.7:0/1314320398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0192af0 0x55d1b019b420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009560 tx=0x7fbb44005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d69f03456c2a45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb36ffd640 1 -- 172.21.15.7:0/1314320398 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400fcd0 con 0x55d1b0192af0 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb36ffd640 1 -- 172.21.15.7:0/1314320398 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb44004de0 con 0x55d1b0192af0 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb36ffd640 1 -- 172.21.15.7:0/1314320398 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44011260 con 0x55d1b0192af0 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb5609a740 1 -- 172.21.15.7:0/1314320398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0192af0 msgr2=0x55d1b019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb5609a740 1 --2- 172.21.15.7:0/1314320398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0192af0 0x55d1b019b420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009560 tx=0x7fbb44005fd0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb5609a740 1 -- 172.21.15.7:0/1314320398 shutdown_connections 2024-09-10T10:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb5609a740 1 --2- 172.21.15.7:0/1314320398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b019dcb0 0x55d1b01b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb5609a740 1 --2- 172.21.15.7:0/1314320398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0193410 0x55d1b01b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb5609a740 1 --2- 172.21.15.7:0/1314320398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b0192af0 0x55d1b019b420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb5609a740 1 -- 172.21.15.7:0/1314320398 >> 172.21.15.7:0/1314320398 conn(0x55d1b018f060 msgr2=0x55d1b019cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb5609a740 1 -- 172.21.15.7:0/1314320398 shutdown_connections 2024-09-10T10:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb5609a740 1 -- 172.21.15.7:0/1314320398 wait complete. 2024-09-10T10:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.898+0000 7fbb5609a740 1 -- start start 2024-09-10T10:49:02.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5609a740 1 Processor -- start 2024-09-10T10:49:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 0x55d1b01a6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0193410 0x55d1b01a6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5609a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b0197750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5609a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1ae020 con 0x55d1b019dcb0 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5609a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1af490 con 0x55d1b0193410 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5609a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1ae6f0 con 0x55d1b0192af0 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5609a740 10 client.0 _ll_register_callbacks cb 0x7fbb480f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb518a0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b0197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5089e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0193410 0x55d1b01a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb518a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 0x55d1b01a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5089e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0193410 0x55d1b01a6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:42564/0 (socket says 172.21.15.7:42564) 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5089e640 1 -- 172.21.15.7:0/2142858839 learned_addr learned my addr 172.21.15.7:0/2142858839 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5089e640 1 -- 172.21.15.7:0/2142858839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 msgr2=0x55d1b01a6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5089e640 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 0x55d1b01a6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5089e640 1 -- 172.21.15.7:0/2142858839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 msgr2=0x55d1b0197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5089e640 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b0197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb5089e640 1 -- 172.21.15.7:0/2142858839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1b0197f80 con 0x55d1b0193410 2024-09-10T10:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.899+0000 7fbb518a0640 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b0197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb5089e640 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0193410 0x55d1b01a6550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009fd0 tx=0x7fbb3c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd93df3817fb918c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb34ff9640 1 -- 172.21.15.7:0/2142858839 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c02b060 con 0x55d1b0193410 2024-09-10T10:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb34ff9640 1 -- 172.21.15.7:0/2142858839 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7fbb3c005020 con 0x55d1b0193410 2024-09-10T10:49:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb5609a740 1 -- 172.21.15.7:0/2142858839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1b007b900 con 0x55d1b0193410 2024-09-10T10:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb34ff9640 1 -- 172.21.15.7:0/2142858839 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb3c005220 con 0x55d1b0193410 2024-09-10T10:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb5609a740 10 client.25175 fetch_fsmap learned FSMap version 28 2024-09-10T10:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb5609a740 1 -- 172.21.15.7:0/2142858839 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d1b0218450 con 0x55d1b0193410 2024-09-10T10:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb34ff9640 1 -- 172.21.15.7:0/2142858839 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d1b007b900 con 0x55d1b0193410 2024-09-10T10:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb34ff9640 1 -- 172.21.15.7:0/2142858839 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbb3c027460 con 0x55d1b0193410 2024-09-10T10:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb5609a740 10 client.25175 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb5609a740 10 client.25175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb5609a740 20 client.25175 populate_metadata read hostname 'smithi007' 2024-09-10T10:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb5609a740 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d1b0218c60 0x55d1b021b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb5609a740 4 client.25175 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.900+0000 7fbb5609a740 1 -- 172.21.15.7:0/2142858839 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1b00c8820 con 0x55d1b0218c60 2024-09-10T10:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.901+0000 7fbb518a0640 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d1b0218c60 0x55d1b021b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.901+0000 7fbb518a0640 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d1b0218c60 0x55d1b021b080 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a872d7eb798788 server_cookie=c02850dc2d0b4930 in_seq=0 out_seq=0 2024-09-10T10:49:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.901+0000 7fbb34ff9640 10 client.25175 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.964+0000 7fbb34ff9640 1 -- 172.21.15.7:0/2142858839 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1b00c8820 con 0x55d1b0218c60 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.964+0000 7fbb34ff9640 10 client.25175 handle_command_reply: tid=0 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.964+0000 7fbb5609a740 1 client.25175 shutdown 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 1 -- 172.21.15.7:0/2142858839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0193410 msgr2=0x55d1b01a6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0193410 0x55d1b01a6550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009fd0 tx=0x7fbb3c012bc0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 1 -- 172.21.15.7:0/2142858839 shutdown_connections 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d1b0218c60 0x55d1b021b080 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1b019dcb0 0x55d1b0197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb34ff9640 0 client.25175 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1b0193410 0x55d1b01a6550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 1 --2- 172.21.15.7:0/2142858839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1b0192af0 0x55d1b01a6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 1 -- 172.21.15.7:0/2142858839 >> 172.21.15.7:0/2142858839 conn(0x55d1b018f060 msgr2=0x55d1b019c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 1 -- 172.21.15.7:0/2142858839 shutdown_connections 2024-09-10T10:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 1 -- 172.21.15.7:0/2142858839 wait complete. 2024-09-10T10:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.965+0000 7fbb5609a740 20 client.25175 trim_cache size 0 max 16384 2024-09-10T10:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 -- 172.21.15.7:0/153335754 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb380a2e20 msgr2=0x7fbb380a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb380a2e20 0x7fbb380a52e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb480ece90 tx=0x7fbb480f5550 comp rx=0 tx=0).stop 2024-09-10T10:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 -- 172.21.15.7:0/153335754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c1009b0 msgr2=0x7fbb4c19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c1009b0 0x7fbb4c19a5c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbb48009230 tx=0x7fbb48004040 comp rx=0 tx=0).stop 2024-09-10T10:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 -- 172.21.15.7:0/153335754 shutdown_connections 2024-09-10T10:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4c1069c0 0x7fbb4c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb4c101350 0x7fbb4c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fbb380a2e20 0x7fbb380a52e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 --2- 172.21.15.7:0/153335754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c1009b0 0x7fbb4c19a5c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 -- 172.21.15.7:0/153335754 >> 172.21.15.7:0/153335754 conn(0x7fbb4c077990 msgr2=0x7fbb4c0698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 -- 172.21.15.7:0/153335754 shutdown_connections 2024-09-10T10:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:02.966+0000 7fbb53303640 1 -- 172.21.15.7:0/153335754 wait complete. 2024-09-10T10:49:03.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 295404 11.10 MB/sec execute 3 sec latency 33.611 ms 2024-09-10T10:49:03.776 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:03 smithi133 ceph-mon[35282]: pgmap v98: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-09-10T10:49:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:03 smithi103 ceph-mon[35160]: pgmap v98: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-09-10T10:49:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:03 smithi007 ceph-mon[30873]: pgmap v98: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-09-10T10:49:04.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 298196 12.81 MB/sec execute 4 sec latency 46.246 ms 2024-09-10T10:49:05.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 300404 11.74 MB/sec execute 5 sec latency 10.792 ms 2024-09-10T10:49:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:05 smithi103 ceph-mon[35160]: pgmap v99: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-09-10T10:49:05.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:05 smithi007 ceph-mon[30873]: pgmap v99: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-09-10T10:49:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:05 smithi133 ceph-mon[35282]: pgmap v99: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-09-10T10:49:06.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 302323 11.86 MB/sec execute 6 sec latency 14.574 ms 2024-09-10T10:49:07.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 305224 12.64 MB/sec execute 7 sec latency 13.604 ms 2024-09-10T10:49:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:07 smithi103 ceph-mon[35160]: pgmap v100: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-09-10T10:49:07.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:07 smithi007 ceph-mon[30873]: pgmap v100: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-09-10T10:49:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:07 smithi133 ceph-mon[35282]: pgmap v100: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-09-10T10:49:08.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 307126 11.93 MB/sec execute 8 sec latency 13.355 ms 2024-09-10T10:49:09.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 309059 12.10 MB/sec execute 9 sec latency 63.427 ms 2024-09-10T10:49:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:09 smithi103 ceph-mon[35160]: pgmap v101: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 354 op/s 2024-09-10T10:49:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:09 smithi007 ceph-mon[30873]: pgmap v101: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 354 op/s 2024-09-10T10:49:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:09 smithi133 ceph-mon[35282]: pgmap v101: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 354 op/s 2024-09-10T10:49:10.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 311788 12.21 MB/sec execute 10 sec latency 17.032 ms 2024-09-10T10:49:11.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 313591 12.10 MB/sec execute 11 sec latency 12.968 ms 2024-09-10T10:49:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:11 smithi103 ceph-mon[35160]: pgmap v102: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 483 op/s 2024-09-10T10:49:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:11 smithi007 ceph-mon[30873]: pgmap v102: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 483 op/s 2024-09-10T10:49:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:11 smithi133 ceph-mon[35282]: pgmap v102: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 483 op/s 2024-09-10T10:49:12.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 316029 12.24 MB/sec execute 12 sec latency 12.621 ms 2024-09-10T10:49:13.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 318258 11.93 MB/sec execute 13 sec latency 19.041 ms 2024-09-10T10:49:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:13 smithi103 ceph-mon[35160]: pgmap v103: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 344 op/s 2024-09-10T10:49:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:13 smithi007 ceph-mon[30873]: pgmap v103: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 344 op/s 2024-09-10T10:49:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:13 smithi133 ceph-mon[35282]: pgmap v103: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 344 op/s 2024-09-10T10:49:14.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 319998 11.90 MB/sec execute 14 sec latency 59.537 ms 2024-09-10T10:49:15.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 322950 12.27 MB/sec execute 15 sec latency 89.927 ms 2024-09-10T10:49:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:15 smithi103 ceph-mon[35160]: pgmap v104: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 344 op/s 2024-09-10T10:49:15.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:15 smithi007 ceph-mon[30873]: pgmap v104: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 344 op/s 2024-09-10T10:49:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:15 smithi133 ceph-mon[35282]: pgmap v104: 97 pgs: 97 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 344 op/s 2024-09-10T10:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.060+0000 7f7b9027b640 1 -- 172.21.15.7:0/61685342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b881009d0 msgr2=0x7f7b88100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.060+0000 7f7b9027b640 1 --2- 172.21.15.7:0/61685342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b881009d0 0x7f7b88100e30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009930 tx=0x7f7b7802a420 comp rx=0 tx=0).stop 2024-09-10T10:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.060+0000 7f7b9027b640 1 -- 172.21.15.7:0/61685342 shutdown_connections 2024-09-10T10:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.060+0000 7f7b9027b640 1 --2- 172.21.15.7:0/61685342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b88101370 0x7f7b8810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.060+0000 7f7b9027b640 1 --2- 172.21.15.7:0/61685342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b881009d0 0x7f7b88100e30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.060+0000 7f7b9027b640 1 --2- 172.21.15.7:0/61685342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b881069e0 0x7f7b88106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.060+0000 7f7b9027b640 1 -- 172.21.15.7:0/61685342 >> 172.21.15.7:0/61685342 conn(0x7f7b880fc6f0 msgr2=0x7f7b880feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.060+0000 7f7b9027b640 1 -- 172.21.15.7:0/61685342 shutdown_connections 2024-09-10T10:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.060+0000 7f7b9027b640 1 -- 172.21.15.7:0/61685342 wait complete. 2024-09-10T10:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b9027b640 1 Processor -- start 2024-09-10T10:49:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b9027b640 1 -- start start 2024-09-10T10:49:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b9027b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b881009d0 0x7f7b881991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b9027b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b88101370 0x7f7b88199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b9027b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b881069e0 0x7f7b881a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b9027b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88110e70 con 0x7f7b88101370 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b9027b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88110cf0 con 0x7f7b881009d0 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b9027b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88110ff0 con 0x7f7b881069e0 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b8d816640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b88101370 0x7f7b88199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b8e017640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b881009d0 0x7f7b881991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b8d816640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b88101370 0x7f7b88199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58762/0 (socket says 172.21.15.7:58762) 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b8d816640 1 -- 172.21.15.7:0/2015615708 learned_addr learned my addr 172.21.15.7:0/2015615708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.061+0000 7f7b8e818640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b881069e0 0x7f7b881a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b8d816640 1 -- 172.21.15.7:0/2015615708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b881069e0 msgr2=0x7f7b881a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b8d816640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b881069e0 0x7f7b881a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b8d816640 1 -- 172.21.15.7:0/2015615708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b881009d0 msgr2=0x7f7b881991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b8d816640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b881009d0 0x7f7b881991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b8d816640 1 -- 172.21.15.7:0/2015615708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b881a47b0 con 0x7f7b88101370 2024-09-10T10:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b8d816640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b88101370 0x7f7b88199730 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7b7802a940 tx=0x7f7b78003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b7f7fe640 1 -- 172.21.15.7:0/2015615708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7803f070 con 0x7f7b88101370 2024-09-10T10:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b7f7fe640 1 -- 172.21.15.7:0/2015615708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b78031c90 con 0x7f7b88101370 2024-09-10T10:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b7f7fe640 1 -- 172.21.15.7:0/2015615708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78035030 con 0x7f7b88101370 2024-09-10T10:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b9027b640 1 -- 172.21.15.7:0/2015615708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b881a4980 con 0x7f7b88101370 2024-09-10T10:49:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.062+0000 7f7b9027b640 1 -- 172.21.15.7:0/2015615708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b881a4e40 con 0x7f7b88101370 2024-09-10T10:49:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.064+0000 7f7b7f7fe640 1 -- 172.21.15.7:0/2015615708 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7b78042070 con 0x7f7b88101370 2024-09-10T10:49:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b93018740 1 Processor -- start 2024-09-10T10:49:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b93018740 1 -- start start 2024-09-10T10:49:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b93018740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97aaaf0 0x55e9d97aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b93018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97ab410 0x55e9d97b5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b93018740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b93018740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e9d970f2f0 con 0x55e9d97aaaf0 2024-09-10T10:49:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b93018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e9d96b7270 con 0x55e9d97ab410 2024-09-10T10:49:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b93018740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e9d970b2b0 con 0x55e9d97b5cb0 2024-09-10T10:49:16.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b8e818640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97aaaf0 0x55e9d97aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b8e818640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97aaaf0 0x55e9d97aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58776/0 (socket says 172.21.15.7:58776) 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b8e017640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97ab410 0x55e9d97b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.065+0000 7f7b8e818640 1 -- 172.21.15.7:0/2185466259 learned_addr learned my addr 172.21.15.7:0/2185466259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b7f7fe640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7b740a2e20 0x7f7b740a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b8d816640 1 --2- 172.21.15.7:0/2185466259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b8e818640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7b740a2e20 0x7f7b740a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b8e017640 1 -- 172.21.15.7:0/2185466259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97b5cb0 msgr2=0x55e9d97b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b8e017640 1 --2- 172.21.15.7:0/2185466259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b8e017640 1 -- 172.21.15.7:0/2185466259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97aaaf0 msgr2=0x55e9d97aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b8e017640 1 --2- 172.21.15.7:0/2185466259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97aaaf0 0x55e9d97aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b8e017640 1 -- 172.21.15.7:0/2185466259 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9d96b6340 con 0x55e9d97ab410 2024-09-10T10:49:16.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b7f7fe640 1 -- 172.21.15.7:0/2015615708 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f7b780ef9d0 con 0x7f7b88101370 2024-09-10T10:49:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b8e017640 1 --2- 172.21.15.7:0/2185466259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97ab410 0x55e9d97b5770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009f80 tx=0x7f7b840118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=537c26a4a9aab548 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b8e818640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7b740a2e20 0x7f7b740a52e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009fd0 tx=0x7f7b80004730 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b7cbfe640 1 -- 172.21.15.7:0/2185466259 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84004d20 con 0x55e9d97ab410 2024-09-10T10:49:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b7cbfe640 1 -- 172.21.15.7:0/2185466259 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b84004050 con 0x55e9d97ab410 2024-09-10T10:49:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.066+0000 7f7b7cbfe640 1 -- 172.21.15.7:0/2185466259 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84005280 con 0x55e9d97ab410 2024-09-10T10:49:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 -- 172.21.15.7:0/2185466259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97ab410 msgr2=0x55e9d97b5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 --2- 172.21.15.7:0/2185466259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97ab410 0x55e9d97b5770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009f80 tx=0x7f7b840118d0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 -- 172.21.15.7:0/2185466259 shutdown_connections 2024-09-10T10:49:16.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 --2- 172.21.15.7:0/2185466259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 --2- 172.21.15.7:0/2185466259 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97ab410 0x55e9d97b5770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 --2- 172.21.15.7:0/2185466259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97aaaf0 0x55e9d97aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 -- 172.21.15.7:0/2185466259 >> 172.21.15.7:0/2185466259 conn(0x55e9d97a7060 msgr2=0x55e9d97a7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 -- 172.21.15.7:0/2185466259 shutdown_connections 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 -- 172.21.15.7:0/2185466259 wait complete. 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 -- start start 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.067+0000 7f7b93018740 1 Processor -- start 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b93018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b93018740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b93018740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b93018740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e9d97b8880 con 0x55e9d97b5cb0 2024-09-10T10:49:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b93018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e9d9698290 con 0x55e9d97aaaf0 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b93018740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e9d97b8a00 con 0x55e9d97ab410 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b93018740 10 client.0 _ll_register_callbacks cb 0x55e9d96c9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8e818640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8e818640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58792/0 (socket says 172.21.15.7:58792) 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8e818640 1 -- 172.21.15.7:0/3197241693 learned_addr learned my addr 172.21.15.7:0/3197241693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8d816640 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8e017640 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8e818640 1 -- 172.21.15.7:0/3197241693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 msgr2=0x55e9d97b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8e818640 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8e818640 1 -- 172.21.15.7:0/3197241693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 msgr2=0x55e9d97b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8e818640 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8e818640 1 -- 172.21.15.7:0/3197241693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9d9422210 con 0x55e9d97b5cb0 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8d816640 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b8e818640 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b35f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7b80004310 tx=0x7f7b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f8823f1a7d308e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.068+0000 7f7b71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b71ffb640 1 -- 172.21.15.7:0/3197241693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80024070 con 0x55e9d97b5cb0 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b93018740 1 -- 172.21.15.7:0/3197241693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b84004050 con 0x55e9d97b5cb0 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b71ffb640 1 -- 172.21.15.7:0/3197241693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f7b80012970 con 0x55e9d97b5cb0 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b71ffb640 1 -- 172.21.15.7:0/3197241693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b80030050 con 0x55e9d97b5cb0 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b93018740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b93018740 1 -- 172.21.15.7:0/3197241693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9d94ee620 con 0x55e9d97b5cb0 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b71ffb640 1 -- 172.21.15.7:0/3197241693 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b84004050 con 0x55e9d97b5cb0 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b71ffb640 1 -- 172.21.15.7:0/3197241693 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7b8001f040 con 0x55e9d97b5cb0 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b93018740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:49:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b93018740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b93018740 20 client.15504 populate_metadata read hostname 'smithi007' 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b93018740 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e9d97c5c80 0x55e9d97c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b93018740 4 client.15504 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b93018740 1 -- 172.21.15.7:0/3197241693 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9d971ca20 con 0x55e9d97c5c80 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.069+0000 7f7b8e017640 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e9d97c5c80 0x55e9d97c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.070+0000 7f7b8e017640 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e9d97c5c80 0x55e9d97c80d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=624ae6d2bdc61d6a server_cookie=7a8cb47eed0aefa9 in_seq=0 out_seq=0 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.070+0000 7f7b71ffb640 10 client.15504 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b71ffb640 1 -- 172.21.15.7:0/3197241693 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9d971ca20 con 0x55e9d97c5c80 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b71ffb640 10 client.15504 handle_command_reply: tid=0 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 client.15504 shutdown 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 -- 172.21.15.7:0/3197241693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 msgr2=0x55e9d97b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b35f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7b80004310 tx=0x7f7b800046c0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 -- 172.21.15.7:0/3197241693 shutdown_connections 2024-09-10T10:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e9d97c5c80 0x55e9d97c80d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b35f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b71ffb640 0 client.15504 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 --2- 172.21.15.7:0/3197241693 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 -- 172.21.15.7:0/3197241693 >> 172.21.15.7:0/3197241693 conn(0x55e9d97a7060 msgr2=0x55e9d97acdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 -- 172.21.15.7:0/3197241693 shutdown_connections 2024-09-10T10:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 1 -- 172.21.15.7:0/3197241693 wait complete. 2024-09-10T10:49:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.071+0000 7f7b93018740 20 client.15504 trim_cache size 0 max 16384 2024-09-10T10:49:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b93018740 1 Processor -- start 2024-09-10T10:49:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b93018740 1 -- start start 2024-09-10T10:49:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b93018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b93018740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b93018740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b93018740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881ade00 con 0x55e9d97b5cb0 2024-09-10T10:49:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b93018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881ab670 con 0x55e9d97aaaf0 2024-09-10T10:49:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b93018740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881a94c0 con 0x55e9d97ab410 2024-09-10T10:49:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b8e818640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b8e818640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58794/0 (socket says 172.21.15.7:58794) 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b8e818640 1 -- 172.21.15.7:0/2760421130 learned_addr learned my addr 172.21.15.7:0/2760421130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b8d816640 1 --2- 172.21.15.7:0/2760421130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b8e017640 1 --2- 172.21.15.7:0/2760421130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b8e818640 1 -- 172.21.15.7:0/2760421130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 msgr2=0x55e9d97a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b8e818640 1 --2- 172.21.15.7:0/2760421130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b8e818640 1 -- 172.21.15.7:0/2760421130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 msgr2=0x55e9d97ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b8e818640 1 --2- 172.21.15.7:0/2760421130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.087+0000 7f7b8e818640 1 -- 172.21.15.7:0/2760421130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9d97b10a0 con 0x55e9d97b5cb0 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b8e818640 1 --2- 172.21.15.7:0/2760421130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b3420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7b80010c10 tx=0x7f7b80009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41ce6f7da8e71a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b73fff640 1 -- 172.21.15.7:0/2760421130 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8002d020 con 0x55e9d97b5cb0 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b73fff640 1 -- 172.21.15.7:0/2760421130 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b80003e50 con 0x55e9d97b5cb0 2024-09-10T10:49:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b73fff640 1 -- 172.21.15.7:0/2760421130 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80024040 con 0x55e9d97b5cb0 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b93018740 1 -- 172.21.15.7:0/2760421130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 msgr2=0x55e9d97b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b93018740 1 --2- 172.21.15.7:0/2760421130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b3420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7b80010c10 tx=0x7f7b80009660 comp rx=0 tx=0).stop 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b93018740 1 -- 172.21.15.7:0/2760421130 shutdown_connections 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b93018740 1 --2- 172.21.15.7:0/2760421130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97b3420 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b93018740 1 --2- 172.21.15.7:0/2760421130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b93018740 1 --2- 172.21.15.7:0/2760421130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b93018740 1 -- 172.21.15.7:0/2760421130 >> 172.21.15.7:0/2760421130 conn(0x55e9d97a7060 msgr2=0x55e9d97b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b93018740 1 -- 172.21.15.7:0/2760421130 shutdown_connections 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.088+0000 7f7b93018740 1 -- 172.21.15.7:0/2760421130 wait complete. 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b93018740 1 -- start start 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b93018740 1 Processor -- start 2024-09-10T10:49:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b93018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97bdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b93018740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97be1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b93018740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b93018740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881a9b30 con 0x55e9d97b5cb0 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b93018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881aafa0 con 0x55e9d97aaaf0 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b93018740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881aa200 con 0x55e9d97ab410 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b93018740 10 client.0 _ll_register_callbacks cb 0x7f7b80012360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b8e818640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b8e017640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97bdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b8e818640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58796/0 (socket says 172.21.15.7:58796) 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b8e818640 1 -- 172.21.15.7:0/4136003526 learned_addr learned my addr 172.21.15.7:0/4136003526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b8d816640 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97be1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b8e818640 1 -- 172.21.15.7:0/4136003526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 msgr2=0x55e9d97be1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b8e818640 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97be1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b8e818640 1 -- 172.21.15.7:0/4136003526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 msgr2=0x55e9d97bdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.089+0000 7f7b8e818640 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97bdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b8e818640 1 -- 172.21.15.7:0/4136003526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9d97af6d0 con 0x55e9d97b5cb0 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b8e017640 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97bdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b8e818640 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97af190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7b80012aa0 tx=0x7f7b8000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49f04979553a0c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b71ffb640 1 -- 172.21.15.7:0/4136003526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8002d070 con 0x55e9d97b5cb0 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b71ffb640 1 -- 172.21.15.7:0/4136003526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f7b80010430 con 0x55e9d97b5cb0 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b93018740 1 -- 172.21.15.7:0/4136003526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b80003e50 con 0x55e9d97b5cb0 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b71ffb640 1 -- 172.21.15.7:0/4136003526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b80044050 con 0x55e9d97b5cb0 2024-09-10T10:49:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b93018740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b93018740 1 -- 172.21.15.7:0/4136003526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e9d98301c0 con 0x55e9d97b5cb0 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b71ffb640 1 -- 172.21.15.7:0/4136003526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b80003e50 con 0x55e9d97b5cb0 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b71ffb640 1 -- 172.21.15.7:0/4136003526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7b8001f070 con 0x55e9d97b5cb0 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b93018740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b93018740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b93018740 20 client.15510 populate_metadata read hostname 'smithi007' 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b93018740 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e9d9830830 0x55e9d9832c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b93018740 4 client.15510 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.090+0000 7f7b93018740 1 -- 172.21.15.7:0/4136003526 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e9d9716d70 con 0x55e9d9830830 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.091+0000 7f7b8e017640 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e9d9830830 0x55e9d9832c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.091+0000 7f7b8e017640 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e9d9830830 0x55e9d9832c50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a43fb64994b5036a server_cookie=1fb7e5cd45c0f817 in_seq=0 out_seq=0 2024-09-10T10:49:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.091+0000 7f7b71ffb640 10 client.15510 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.159+0000 7f7b71ffb640 1 -- 172.21.15.7:0/4136003526 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e9d9716d70 con 0x55e9d9830830 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.159+0000 7f7b71ffb640 10 client.15510 handle_command_reply: tid=0 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 client.15510 shutdown 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 -- 172.21.15.7:0/4136003526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 msgr2=0x55e9d97af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97af190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7b80012aa0 tx=0x7f7b8000f580 comp rx=0 tx=0).stop 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 -- 172.21.15.7:0/4136003526 shutdown_connections 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e9d9830830 0x55e9d9832c50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9d97b5cb0 0x55e9d97af190 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9d97ab410 0x55e9d97be1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b71ffb640 0 client.15510 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 --2- 172.21.15.7:0/4136003526 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e9d97aaaf0 0x55e9d97bdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 -- 172.21.15.7:0/4136003526 >> 172.21.15.7:0/4136003526 conn(0x55e9d97a7060 msgr2=0x55e9d97b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 -- 172.21.15.7:0/4136003526 shutdown_connections 2024-09-10T10:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 1 -- 172.21.15.7:0/4136003526 wait complete. 2024-09-10T10:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.160+0000 7f7b93018740 20 client.15510 trim_cache size 0 max 16384 2024-09-10T10:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 -- 172.21.15.7:0/2015615708 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7b740a2e20 msgr2=0x7f7b740a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7b740a2e20 0x7f7b740a52e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009fd0 tx=0x7f7b80004730 comp rx=0 tx=0).stop 2024-09-10T10:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 -- 172.21.15.7:0/2015615708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b88101370 msgr2=0x7f7b88199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b88101370 0x7f7b88199730 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7b7802a940 tx=0x7f7b78003b20 comp rx=0 tx=0).stop 2024-09-10T10:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 -- 172.21.15.7:0/2015615708 shutdown_connections 2024-09-10T10:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b881069e0 0x7f7b881a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7b740a2e20 0x7f7b740a52e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b88101370 0x7f7b88199730 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 --2- 172.21.15.7:0/2015615708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b881009d0 0x7f7b881991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 -- 172.21.15.7:0/2015615708 >> 172.21.15.7:0/2015615708 conn(0x7f7b880fc6f0 msgr2=0x7f7b880fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 -- 172.21.15.7:0/2015615708 shutdown_connections 2024-09-10T10:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:16.161+0000 7f7b9027b640 1 -- 172.21.15.7:0/2015615708 wait complete. 2024-09-10T10:49:16.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 324995 11.95 MB/sec execute 16 sec latency 39.657 ms 2024-09-10T10:49:17.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 327075 12.05 MB/sec execute 17 sec latency 20.015 ms 2024-09-10T10:49:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:17 smithi103 ceph-mon[35160]: pgmap v105: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 479 op/s 2024-09-10T10:49:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:17 smithi007 ceph-mon[30873]: pgmap v105: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 479 op/s 2024-09-10T10:49:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:17 smithi133 ceph-mon[35282]: pgmap v105: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 479 op/s 2024-09-10T10:49:18.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 330118 12.41 MB/sec execute 18 sec latency 10.864 ms 2024-09-10T10:49:19.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 332085 12.08 MB/sec execute 19 sec latency 50.549 ms 2024-09-10T10:49:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:19 smithi103 ceph-mon[35160]: pgmap v106: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 263 op/s 2024-09-10T10:49:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:19 smithi007 ceph-mon[30873]: pgmap v106: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 263 op/s 2024-09-10T10:49:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:19 smithi133 ceph-mon[35282]: pgmap v106: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 263 op/s 2024-09-10T10:49:20.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 334139 12.15 MB/sec execute 20 sec latency 23.725 ms 2024-09-10T10:49:21.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 337090 12.41 MB/sec execute 21 sec latency 16.499 ms 2024-09-10T10:49:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:21 smithi103 ceph-mon[35160]: pgmap v107: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 448 op/s 2024-09-10T10:49:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:21 smithi007 ceph-mon[30873]: pgmap v107: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 448 op/s 2024-09-10T10:49:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:21 smithi133 ceph-mon[35282]: pgmap v107: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 448 op/s 2024-09-10T10:49:22.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 339361 12.22 MB/sec execute 22 sec latency 12.457 ms 2024-09-10T10:49:23.508 INFO:tasks.workunit.client.0.smithi007.stdout: 1 341217 12.23 MB/sec execute 23 sec latency 16.108 ms 2024-09-10T10:49:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:23 smithi103 ceph-mon[35160]: pgmap v108: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-10T10:49:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:23 smithi007 ceph-mon[30873]: pgmap v108: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-10T10:49:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:23 smithi133 ceph-mon[35282]: pgmap v108: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-10T10:49:24.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 344032 12.43 MB/sec execute 24 sec latency 37.767 ms 2024-09-10T10:49:25.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 346046 12.22 MB/sec execute 25 sec latency 14.328 ms 2024-09-10T10:49:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:25 smithi103 ceph-mon[35160]: pgmap v109: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-10T10:49:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:25 smithi007 ceph-mon[30873]: pgmap v109: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-10T10:49:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:25 smithi133 ceph-mon[35282]: pgmap v109: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-10T10:49:26.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 348204 12.28 MB/sec execute 26 sec latency 12.138 ms 2024-09-10T10:49:27.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 351119 12.47 MB/sec execute 27 sec latency 12.851 ms 2024-09-10T10:49:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:27 smithi103 ceph-mon[35160]: pgmap v110: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 454 op/s 2024-09-10T10:49:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:27 smithi007 ceph-mon[30873]: pgmap v110: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 454 op/s 2024-09-10T10:49:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:27 smithi133 ceph-mon[35282]: pgmap v110: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 454 op/s 2024-09-10T10:49:28.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 353048 12.28 MB/sec execute 28 sec latency 16.560 ms 2024-09-10T10:49:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 -- 172.21.15.7:0/2095615444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952c101250 msgr2=0x7f952c107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 --2- 172.21.15.7:0/2095615444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952c101250 0x7f952c107990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9518009900 tx=0x7f951802a410 comp rx=0 tx=0).stop 2024-09-10T10:49:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 -- 172.21.15.7:0/2095615444 shutdown_connections 2024-09-10T10:49:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 --2- 172.21.15.7:0/2095615444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f952c1080f0 0x7f952c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 --2- 172.21.15.7:0/2095615444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952c101250 0x7f952c107990 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 --2- 172.21.15.7:0/2095615444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c1008a0 0x7f952c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 -- 172.21.15.7:0/2095615444 >> 172.21.15.7:0/2095615444 conn(0x7f952c075ef0 msgr2=0x7f952c076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 -- 172.21.15.7:0/2095615444 shutdown_connections 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 -- 172.21.15.7:0/2095615444 wait complete. 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 Processor -- start 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 -- start start 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f952c1008a0 0x7f952c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.252+0000 7f9532c65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c1080f0 0x7f952c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9532c65640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952c19cc20 0x7f952c19d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9532c65640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f952c111f70 con 0x7f952c19cc20 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9532c65640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f952c111df0 con 0x7f952c1008a0 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9532c65640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f952c1120f0 con 0x7f952c1080f0 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9531202640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952c19cc20 0x7f952c19d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9531202640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952c19cc20 0x7f952c19d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54402/0 (socket says 172.21.15.7:54402) 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f952bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c1080f0 0x7f952c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9531202640 1 -- 172.21.15.7:0/535361165 learned_addr learned my addr 172.21.15.7:0/535361165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9530a01640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f952c1008a0 0x7f952c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9531202640 1 -- 172.21.15.7:0/535361165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c1080f0 msgr2=0x7f952c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9531202640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c1080f0 0x7f952c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9531202640 1 -- 172.21.15.7:0/535361165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f952c1008a0 msgr2=0x7f952c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9531202640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f952c1008a0 0x7f952c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9531202640 1 -- 172.21.15.7:0/535361165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952c19d710 con 0x7f952c19cc20 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9531202640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952c19cc20 0x7f952c19d0d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9524009fd0 tx=0x7f952400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9529ffb640 1 -- 172.21.15.7:0/535361165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524017070 con 0x7f952c19cc20 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9529ffb640 1 -- 172.21.15.7:0/535361165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9524003e50 con 0x7f952c19cc20 2024-09-10T10:49:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9532c65640 1 -- 172.21.15.7:0/535361165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952c19da00 con 0x7f952c19cc20 2024-09-10T10:49:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.253+0000 7f9529ffb640 1 -- 172.21.15.7:0/535361165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240106c0 con 0x7f952c19cc20 2024-09-10T10:49:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.254+0000 7f9532c65640 1 -- 172.21.15.7:0/535361165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f952c1a5170 con 0x7f952c19cc20 2024-09-10T10:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.256+0000 7f9529ffb640 1 -- 172.21.15.7:0/535361165 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9524003970 con 0x7f952c19cc20 2024-09-10T10:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.256+0000 7f9535a04740 1 Processor -- start 2024-09-10T10:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.256+0000 7f9535a04740 1 -- start start 2024-09-10T10:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.256+0000 7f9535a04740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5aded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.256+0000 7f9535a04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.256+0000 7f9535a04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.256+0000 7f9535a04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ef2b5122f0 con 0x55ef2b5ae410 2024-09-10T10:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.256+0000 7f9535a04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef2b4ba270 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.256+0000 7f9535a04740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ef2b50e2b0 con 0x55ef2b5adaf0 2024-09-10T10:49:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9530a01640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9530a01640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54414/0 (socket says 172.21.15.7:54414) 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9530a01640 1 -- 172.21.15.7:0/3168158804 learned_addr learned my addr 172.21.15.7:0/3168158804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f952bfff640 1 --2- 172.21.15.7:0/3168158804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9529ffb640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f951c0a2da0 0x7f951c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f952bfff640 1 --2- 172.21.15.7:0/3168158804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9530a01640 1 -- 172.21.15.7:0/3168158804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5adaf0 msgr2=0x55ef2b5aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9529ffb640 1 -- 172.21.15.7:0/535361165 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f95240cf140 con 0x7f952c19cc20 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9531202640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f951c0a2da0 0x7f951c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9530a01640 1 --2- 172.21.15.7:0/3168158804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9530a01640 1 -- 172.21.15.7:0/3168158804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5b8cb0 msgr2=0x55ef2b5bb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9530a01640 1 --2- 172.21.15.7:0/3168158804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9530a01640 1 -- 172.21.15.7:0/3168158804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2b4b9340 con 0x55ef2b5ae410 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f952bfff640 1 --2- 172.21.15.7:0/3168158804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9530a01640 1 --2- 172.21.15.7:0/3168158804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b8770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9520009cf0 tx=0x7f9520011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26268fc8c2bd296 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9531202640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f951c0a2da0 0x7f951c0a5260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f952400b760 tx=0x7f9524004020 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9516ffd640 1 -- 172.21.15.7:0/3168158804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520004230 con 0x55ef2b5ae410 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9516ffd640 1 -- 172.21.15.7:0/3168158804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95200043d0 con 0x55ef2b5ae410 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.257+0000 7f9516ffd640 1 -- 172.21.15.7:0/3168158804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520019840 con 0x55ef2b5ae410 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.258+0000 7f9535a04740 1 -- 172.21.15.7:0/3168158804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5ae410 msgr2=0x55ef2b5b8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.258+0000 7f9535a04740 1 --2- 172.21.15.7:0/3168158804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b8770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9520009cf0 tx=0x7f9520011810 comp rx=0 tx=0).stop 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.258+0000 7f9535a04740 1 -- 172.21.15.7:0/3168158804 shutdown_connections 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.258+0000 7f9535a04740 1 --2- 172.21.15.7:0/3168158804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.258+0000 7f9535a04740 1 --2- 172.21.15.7:0/3168158804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b8770 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.258+0000 7f9535a04740 1 --2- 172.21.15.7:0/3168158804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.258+0000 7f9535a04740 1 -- 172.21.15.7:0/3168158804 >> 172.21.15.7:0/3168158804 conn(0x55ef2b5aa060 msgr2=0x55ef2b5aa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.258+0000 7f9535a04740 1 -- 172.21.15.7:0/3168158804 shutdown_connections 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.258+0000 7f9535a04740 1 -- 172.21.15.7:0/3168158804 wait complete. 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.258+0000 7f9535a04740 1 -- start start 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9535a04740 1 Processor -- start 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9535a04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9535a04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9535a04740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5b6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9535a04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ef2b5bb880 con 0x55ef2b5adaf0 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9535a04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef2b49b290 con 0x55ef2b5ae410 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9535a04740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ef2b5bba00 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9535a04740 10 client.0 _ll_register_callbacks cb 0x55ef2b4cc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f952bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f952bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54424/0 (socket says 172.21.15.7:54424) 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f952bfff640 1 -- 172.21.15.7:0/3852987807 learned_addr learned my addr 172.21.15.7:0/3852987807 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9530a01640 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f952bfff640 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9530a01640 1 -- 172.21.15.7:0/3852987807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5b8cb0 msgr2=0x55ef2b5b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9530a01640 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9530a01640 1 -- 172.21.15.7:0/3852987807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5adaf0 msgr2=0x55ef2b5b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9530a01640 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.259+0000 7f9530a01640 1 -- 172.21.15.7:0/3852987807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2b225210 con 0x55ef2b5ae410 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f9530a01640 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b60e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f952001b040 tx=0x7f95200052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cbd1f4af5b0c896 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f950bfff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:49:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f950bfff640 1 -- 172.21.15.7:0/3852987807 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520027070 con 0x55ef2b5ae410 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f9535a04740 1 -- 172.21.15.7:0/3852987807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95200043d0 con 0x55ef2b5ae410 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f950bfff640 1 -- 172.21.15.7:0/3852987807 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f95200037f0 con 0x55ef2b5ae410 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f950bfff640 1 -- 172.21.15.7:0/3852987807 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9520033050 con 0x55ef2b5ae410 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f9535a04740 10 client.25205 fetch_fsmap learned FSMap version 28 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f9535a04740 1 -- 172.21.15.7:0/3852987807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef2b2f1620 con 0x55ef2b5ae410 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f950bfff640 1 -- 172.21.15.7:0/3852987807 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95200043d0 con 0x55ef2b5ae410 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f950bfff640 1 -- 172.21.15.7:0/3852987807 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9520004ba0 con 0x55ef2b5ae410 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f9535a04740 10 client.25205 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f9535a04740 10 client.25205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f9535a04740 20 client.25205 populate_metadata read hostname 'smithi007' 2024-09-10T10:49:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f9535a04740 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ef2b5c8bb0 0x55ef2b5cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f9535a04740 4 client.25205 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.260+0000 7f9535a04740 1 -- 172.21.15.7:0/3852987807 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef2b51fa20 con 0x55ef2b5c8bb0 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.261+0000 7f952bfff640 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ef2b5c8bb0 0x55ef2b5cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.261+0000 7f950bfff640 10 client.25205 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.261+0000 7f952bfff640 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ef2b5c8bb0 0x55ef2b5cafa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44922ba39e9694b2 server_cookie=f81a3b752408b1a in_seq=0 out_seq=0 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f950bfff640 1 -- 172.21.15.7:0/3852987807 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef2b51fa20 con 0x55ef2b5c8bb0 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f950bfff640 10 client.25205 handle_command_reply: tid=0 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 client.25205 shutdown 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 -- 172.21.15.7:0/3852987807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 msgr2=0x55ef2b5b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b60e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f952001b040 tx=0x7f95200052f0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 -- 172.21.15.7:0/3852987807 shutdown_connections 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ef2b5c8bb0 0x55ef2b5cafa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 0x55ef2b5b60e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f950bfff640 0 client.25205 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 --2- 172.21.15.7:0/3852987807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 -- 172.21.15.7:0/3852987807 >> 172.21.15.7:0/3852987807 conn(0x55ef2b5aa060 msgr2=0x55ef2b5a91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 -- 172.21.15.7:0/3852987807 shutdown_connections 2024-09-10T10:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 1 -- 172.21.15.7:0/3852987807 wait complete. 2024-09-10T10:49:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.262+0000 7f9535a04740 20 client.25205 trim_cache size 0 max 16384 2024-09-10T10:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.282+0000 7f9535a04740 1 Processor -- start 2024-09-10T10:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.283+0000 7f9535a04740 1 -- start start 2024-09-10T10:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.283+0000 7f9535a04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.283+0000 7f9535a04740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5ae410 0x55ef2b5d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.283+0000 7f9535a04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.283+0000 7f9535a04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f952c1b0010 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.283+0000 7f9535a04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f952c1ad880 con 0x55ef2b5adaf0 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.283+0000 7f9535a04740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f952c1ab6d0 con 0x55ef2b5ae410 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.284+0000 7f9530a01640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5ae410 0x55ef2b5d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.284+0000 7f9530a01640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5ae410 0x55ef2b5d0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:34902/0 (socket says 172.21.15.7:34902) 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.285+0000 7f9530a01640 1 -- 172.21.15.7:0/697836398 learned_addr learned my addr 172.21.15.7:0/697836398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.285+0000 7f9530a01640 1 -- 172.21.15.7:0/697836398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5adaf0 msgr2=0x55ef2b5b6450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.285+0000 7f9530a01640 1 --2- 172.21.15.7:0/697836398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5b6450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.285+0000 7f9530a01640 1 -- 172.21.15.7:0/697836398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 msgr2=0x55ef2b5d0d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.285+0000 7f9530a01640 1 --2- 172.21.15.7:0/697836398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.285+0000 7f9530a01640 1 -- 172.21.15.7:0/697836398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2b5ad870 con 0x55ef2b5ae410 2024-09-10T10:49:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.298+0000 7f9530a01640 1 --2- 172.21.15.7:0/697836398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5ae410 0x55ef2b5d0800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9520000c00 tx=0x7f9520019770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898238be62b03b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.300+0000 7f95167fc640 1 -- 172.21.15.7:0/697836398 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520030020 con 0x55ef2b5ae410 2024-09-10T10:49:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.300+0000 7f95167fc640 1 -- 172.21.15.7:0/697836398 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9520009b70 con 0x55ef2b5ae410 2024-09-10T10:49:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.300+0000 7f95167fc640 1 -- 172.21.15.7:0/697836398 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520027070 con 0x55ef2b5ae410 2024-09-10T10:49:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.301+0000 7f9535a04740 1 -- 172.21.15.7:0/697836398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5ae410 msgr2=0x55ef2b5d0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.301+0000 7f9535a04740 1 --2- 172.21.15.7:0/697836398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5ae410 0x55ef2b5d0800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9520000c00 tx=0x7f9520019770 comp rx=0 tx=0).stop 2024-09-10T10:49:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.302+0000 7f9535a04740 1 -- 172.21.15.7:0/697836398 shutdown_connections 2024-09-10T10:49:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.302+0000 7f9535a04740 1 --2- 172.21.15.7:0/697836398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.302+0000 7f9535a04740 1 --2- 172.21.15.7:0/697836398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5ae410 0x55ef2b5d0800 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.302+0000 7f9535a04740 1 --2- 172.21.15.7:0/697836398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.302+0000 7f9535a04740 1 -- 172.21.15.7:0/697836398 >> 172.21.15.7:0/697836398 conn(0x55ef2b5aa060 msgr2=0x55ef2b5b7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.303+0000 7f9535a04740 1 -- 172.21.15.7:0/697836398 shutdown_connections 2024-09-10T10:49:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.303+0000 7f9535a04740 1 -- 172.21.15.7:0/697836398 wait complete. 2024-09-10T10:49:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.303+0000 7f9535a04740 1 -- start start 2024-09-10T10:49:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.303+0000 7f9535a04740 1 Processor -- start 2024-09-10T10:49:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.304+0000 7f9535a04740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.304+0000 7f9535a04740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 0x55ef2b5bc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.304+0000 7f9535a04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.304+0000 7f9535a04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f952c1abd40 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.304+0000 7f9535a04740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f952c1ad1b0 con 0x55ef2b5ae410 2024-09-10T10:49:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.304+0000 7f9535a04740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f952c1ac410 con 0x55ef2b5adaf0 2024-09-10T10:49:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.304+0000 7f9535a04740 10 client.0 _ll_register_callbacks cb 0x7f9518009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.305+0000 7f952bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.305+0000 7f952bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54440/0 (socket says 172.21.15.7:54440) 2024-09-10T10:49:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.305+0000 7f952bfff640 1 -- 172.21.15.7:0/3328091869 learned_addr learned my addr 172.21.15.7:0/3328091869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.305+0000 7f952bfff640 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.305+0000 7f9530a01640 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 0x55ef2b5bc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.305+0000 7f952bfff640 1 -- 172.21.15.7:0/3328091869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5adaf0 msgr2=0x55ef2b5c11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.305+0000 7f952bfff640 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5c11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.305+0000 7f952bfff640 1 -- 172.21.15.7:0/3328091869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 msgr2=0x55ef2b5bc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.305+0000 7f952bfff640 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 0x55ef2b5bc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.305+0000 7f952bfff640 1 -- 172.21.15.7:0/3328091869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2b5bcfd0 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.306+0000 7f952bfff640 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bca10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f952c06bba0 tx=0x7f9518009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b93dd5d0317168 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.306+0000 7f950bfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:49:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.306+0000 7f950bfff640 1 -- 172.21.15.7:0/3328091869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518035040 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.306+0000 7f950bfff640 1 -- 172.21.15.7:0/3328091869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f95180048c0 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.306+0000 7f9535a04740 1 -- 172.21.15.7:0/3328091869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9520009b70 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.306+0000 7f950bfff640 1 -- 172.21.15.7:0/3328091869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9518004ac0 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.306+0000 7f9535a04740 10 client.15525 fetch_fsmap learned FSMap version 28 2024-09-10T10:49:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.306+0000 7f9535a04740 1 -- 172.21.15.7:0/3328091869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef2b63be50 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.307+0000 7f950bfff640 1 -- 172.21.15.7:0/3328091869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9520009b70 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.307+0000 7f950bfff640 1 -- 172.21.15.7:0/3328091869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9518056060 con 0x55ef2b5b8cb0 2024-09-10T10:49:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.307+0000 7f9535a04740 10 client.15525 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T10:49:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.307+0000 7f9535a04740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.307+0000 7f9535a04740 20 client.15525 populate_metadata read hostname 'smithi007' 2024-09-10T10:49:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.307+0000 7f9535a04740 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ef2b63c5d0 0x55ef2b63e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.307+0000 7f9535a04740 4 client.15525 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:49:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.307+0000 7f9535a04740 1 -- 172.21.15.7:0/3328091869 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef2b519d70 con 0x55ef2b63c5d0 2024-09-10T10:49:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.308+0000 7f9530a01640 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ef2b63c5d0 0x55ef2b63e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.308+0000 7f950bfff640 10 client.15525 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:29.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.308+0000 7f9530a01640 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ef2b63c5d0 0x55ef2b63e9f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb9651a6c559157 server_cookie=5322e207e1c0ca30 in_seq=0 out_seq=0 2024-09-10T10:49:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.446+0000 7f950bfff640 1 -- 172.21.15.7:0/3328091869 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef2b519d70 con 0x55ef2b63c5d0 2024-09-10T10:49:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.446+0000 7f950bfff640 10 client.15525 handle_command_reply: tid=0 2024-09-10T10:49:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.446+0000 7f9535a04740 1 client.15525 shutdown 2024-09-10T10:49:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.446+0000 7f9535a04740 1 -- 172.21.15.7:0/3328091869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 msgr2=0x55ef2b5bca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.446+0000 7f9535a04740 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bca10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f952c06bba0 tx=0x7f9518009660 comp rx=0 tx=0).stop 2024-09-10T10:49:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.447+0000 7f9535a04740 1 -- 172.21.15.7:0/3328091869 shutdown_connections 2024-09-10T10:49:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.447+0000 7f9535a04740 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ef2b63c5d0 0x55ef2b63e9f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.447+0000 7f9535a04740 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ef2b5b8cb0 0x55ef2b5bca10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.447+0000 7f9535a04740 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef2b5ae410 0x55ef2b5bc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.447+0000 7f950bfff640 0 client.15525 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.447+0000 7f9535a04740 1 --2- 172.21.15.7:0/3328091869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ef2b5adaf0 0x55ef2b5c11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.447+0000 7f9535a04740 1 -- 172.21.15.7:0/3328091869 >> 172.21.15.7:0/3328091869 conn(0x55ef2b5aa060 msgr2=0x55ef2b5a89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.447+0000 7f9535a04740 1 -- 172.21.15.7:0/3328091869 shutdown_connections 2024-09-10T10:49:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.447+0000 7f9535a04740 1 -- 172.21.15.7:0/3328091869 wait complete. 2024-09-10T10:49:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.447+0000 7f9535a04740 20 client.15525 trim_cache size 0 max 16384 2024-09-10T10:49:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:49:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:49:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:49:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:49:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 -- 172.21.15.7:0/535361165 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f951c0a2da0 msgr2=0x7f951c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f951c0a2da0 0x7f951c0a5260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f952400b760 tx=0x7f9524004020 comp rx=0 tx=0).stop 2024-09-10T10:49:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 -- 172.21.15.7:0/535361165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952c19cc20 msgr2=0x7f952c19d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952c19cc20 0x7f952c19d0d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9524009fd0 tx=0x7f952400b260 comp rx=0 tx=0).stop 2024-09-10T10:49:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 -- 172.21.15.7:0/535361165 shutdown_connections 2024-09-10T10:49:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f951c0a2da0 0x7f951c0a5260 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f952c19cc20 0x7f952c19d0d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f952c1080f0 0x7f952c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 --2- 172.21.15.7:0/535361165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f952c1008a0 0x7f952c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 -- 172.21.15.7:0/535361165 >> 172.21.15.7:0/535361165 conn(0x7f952c075ef0 msgr2=0x7f952c0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 -- 172.21.15.7:0/535361165 shutdown_connections 2024-09-10T10:49:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:29.448+0000 7f9532c65640 1 -- 172.21.15.7:0/535361165 wait complete. 2024-09-10T10:49:29.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 355131 12.33 MB/sec execute 29 sec latency 70.121 ms 2024-09-10T10:49:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:29 smithi103 ceph-mon[35160]: pgmap v111: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-09-10T10:49:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:29 smithi007 ceph-mon[30873]: pgmap v111: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-09-10T10:49:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:29 smithi133 ceph-mon[35282]: pgmap v111: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-09-10T10:49:30.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 357973 12.40 MB/sec execute 30 sec latency 12.793 ms 2024-09-10T10:49:31.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 359889 12.32 MB/sec execute 31 sec latency 10.915 ms 2024-09-10T10:49:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:31 smithi103 ceph-mon[35160]: pgmap v112: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 502 op/s 2024-09-10T10:49:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:31 smithi007 ceph-mon[30873]: pgmap v112: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 502 op/s 2024-09-10T10:49:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:31 smithi133 ceph-mon[35282]: pgmap v112: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.7 MiB/s wr, 502 op/s 2024-09-10T10:49:32.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 362218 12.37 MB/sec execute 32 sec latency 16.353 ms 2024-09-10T10:49:33.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 365142 12.43 MB/sec execute 33 sec latency 12.852 ms 2024-09-10T10:49:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:33 smithi103 ceph-mon[35160]: pgmap v113: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 318 op/s 2024-09-10T10:49:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:33 smithi007 ceph-mon[30873]: pgmap v113: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 318 op/s 2024-09-10T10:49:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:33 smithi133 ceph-mon[35282]: pgmap v113: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 318 op/s 2024-09-10T10:49:34.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 366775 12.35 MB/sec execute 34 sec latency 46.787 ms 2024-09-10T10:49:35.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 369258 12.40 MB/sec execute 35 sec latency 37.673 ms 2024-09-10T10:49:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:35 smithi103 ceph-mon[35160]: pgmap v114: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 318 op/s 2024-09-10T10:49:35.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:35 smithi007 ceph-mon[30873]: pgmap v114: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 318 op/s 2024-09-10T10:49:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:35 smithi133 ceph-mon[35282]: pgmap v114: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 318 op/s 2024-09-10T10:49:36.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 371658 12.36 MB/sec execute 36 sec latency 14.474 ms 2024-09-10T10:49:37.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 373421 12.27 MB/sec execute 37 sec latency 11.278 ms 2024-09-10T10:49:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:37 smithi103 ceph-mon[35160]: pgmap v115: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 443 op/s 2024-09-10T10:49:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:37 smithi007 ceph-mon[30873]: pgmap v115: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 443 op/s 2024-09-10T10:49:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:37 smithi133 ceph-mon[35282]: pgmap v115: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 443 op/s 2024-09-10T10:49:38.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 376254 12.42 MB/sec execute 38 sec latency 13.952 ms 2024-09-10T10:49:39.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 378511 12.31 MB/sec execute 39 sec latency 11.538 ms 2024-09-10T10:49:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:39 smithi103 ceph-mon[35160]: pgmap v116: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-09-10T10:49:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:39 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:49:39.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:39 smithi007 ceph-mon[30873]: pgmap v116: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-09-10T10:49:39.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:39 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:49:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:39 smithi133 ceph-mon[35282]: pgmap v116: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-09-10T10:49:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:39 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:49:40.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 380317 12.29 MB/sec execute 40 sec latency 29.719 ms 2024-09-10T10:49:41.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 383349 12.44 MB/sec execute 41 sec latency 23.839 ms 2024-09-10T10:49:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:41 smithi103 ceph-mon[35160]: pgmap v117: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-09-10T10:49:41.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:41 smithi007 ceph-mon[30873]: pgmap v117: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-09-10T10:49:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:41 smithi133 ceph-mon[35282]: pgmap v117: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-09-10T10:49:42.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 385332 12.29 MB/sec execute 42 sec latency 13.646 ms 2024-09-10T10:49:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.545+0000 7ffb952d9640 1 -- 172.21.15.7:0/3694619508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90102ca0 msgr2=0x7ffb90103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.545+0000 7ffb952d9640 1 --2- 172.21.15.7:0/3694619508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90102ca0 0x7ffb90103080 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009900 tx=0x7ffb8002a410 comp rx=0 tx=0).stop 2024-09-10T10:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.545+0000 7ffb952d9640 1 -- 172.21.15.7:0/3694619508 shutdown_connections 2024-09-10T10:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.545+0000 7ffb952d9640 1 --2- 172.21.15.7:0/3694619508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb9010c0d0 0x7ffb9010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.545+0000 7ffb952d9640 1 --2- 172.21.15.7:0/3694619508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb901035c0 0x7ffb9010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.545+0000 7ffb952d9640 1 --2- 172.21.15.7:0/3694619508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb90102ca0 0x7ffb90103080 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.545+0000 7ffb952d9640 1 -- 172.21.15.7:0/3694619508 >> 172.21.15.7:0/3694619508 conn(0x7ffb900fc6b0 msgr2=0x7ffb900fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.545+0000 7ffb952d9640 1 -- 172.21.15.7:0/3694619508 shutdown_connections 2024-09-10T10:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.545+0000 7ffb952d9640 1 -- 172.21.15.7:0/3694619508 wait complete. 2024-09-10T10:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.545+0000 7ffb952d9640 1 Processor -- start 2024-09-10T10:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb952d9640 1 -- start start 2024-09-10T10:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb952d9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb90102ca0 0x7ffb901008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb952d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb901035c0 0x7ffb90100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb952d9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9010c0d0 0x7ffb90101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb952d9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90111ed0 con 0x7ffb9010c0d0 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb952d9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90111d50 con 0x7ffb90102ca0 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb952d9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90112050 con 0x7ffb901035c0 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb901035c0 0x7ffb90100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb90102ca0 0x7ffb901008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9010c0d0 0x7ffb90101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb90102ca0 0x7ffb901008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:47656/0 (socket says 172.21.15.7:47656) 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb901035c0 0x7ffb90100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:57634/0 (socket says 172.21.15.7:57634) 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8effd640 1 -- 172.21.15.7:0/3094863002 learned_addr learned my addr 172.21.15.7:0/3094863002 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9010c0d0 0x7ffb90101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53354/0 (socket says 172.21.15.7:53354) 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8e7fc640 1 -- 172.21.15.7:0/3094863002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb90102ca0 msgr2=0x7ffb901008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb90102ca0 0x7ffb901008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8e7fc640 1 -- 172.21.15.7:0/3094863002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9010c0d0 msgr2=0x7ffb90101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9010c0d0 0x7ffb90101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.546+0000 7ffb8e7fc640 1 -- 172.21.15.7:0/3094863002 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb901a6860 con 0x7ffb901035c0 2024-09-10T10:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.547+0000 7ffb8effd640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb90102ca0 0x7ffb901008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.547+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb901035c0 0x7ffb90100e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400e9c0 tx=0x7ffb8400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.547+0000 7ffb77fff640 1 -- 172.21.15.7:0/3094863002 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400cd80 con 0x7ffb901035c0 2024-09-10T10:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.547+0000 7ffb952d9640 1 -- 172.21.15.7:0/3094863002 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb901a6b50 con 0x7ffb901035c0 2024-09-10T10:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.547+0000 7ffb77fff640 1 -- 172.21.15.7:0/3094863002 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb84010910 con 0x7ffb901035c0 2024-09-10T10:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.547+0000 7ffb77fff640 1 -- 172.21.15.7:0/3094863002 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84018980 con 0x7ffb901035c0 2024-09-10T10:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.547+0000 7ffb952d9640 1 -- 172.21.15.7:0/3094863002 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb901a70f0 con 0x7ffb901035c0 2024-09-10T10:49:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb77fff640 1 -- 172.21.15.7:0/3094863002 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ffb8401f020 con 0x7ffb901035c0 2024-09-10T10:49:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb9806f740 1 Processor -- start 2024-09-10T10:49:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb9806f740 1 -- start start 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 0x55ac1e352ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e35d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e3600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb9806f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e2b72f0 con 0x55ac1e353410 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb9806f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e25f270 con 0x55ac1e352af0 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb9806f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e2b32b0 con 0x55ac1e35dcb0 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e35d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e35d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53362/0 (socket says 172.21.15.7:53362) 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb8f7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 0x55ac1e352ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb8effd640 1 -- 172.21.15.7:0/3321279019 learned_addr learned my addr 172.21.15.7:0/3321279019 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.550+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/3321279019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e3600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb8f7fe640 1 -- 172.21.15.7:0/3321279019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 msgr2=0x55ac1e3600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb8f7fe640 1 --2- 172.21.15.7:0/3321279019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e3600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb8f7fe640 1 -- 172.21.15.7:0/3321279019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 msgr2=0x55ac1e35d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb8f7fe640 1 --2- 172.21.15.7:0/3321279019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e35d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb8f7fe640 1 -- 172.21.15.7:0/3321279019 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1e25e340 con 0x55ac1e352af0 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb8effd640 1 --2- 172.21.15.7:0/3321279019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e35d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb77fff640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ffb780a2da0 0x7ffb780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb8f7fe640 1 --2- 172.21.15.7:0/3321279019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 0x55ac1e352ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009eb0 tx=0x7ffb880107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb81eb91af2e0be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb8effd640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ffb780a2da0 0x7ffb780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb77fff640 1 -- 172.21.15.7:0/3094863002 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7ffb840cf400 con 0x7ffb901035c0 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb757fa640 1 -- 172.21.15.7:0/3321279019 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88003b70 con 0x55ac1e352af0 2024-09-10T10:49:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb757fa640 1 -- 172.21.15.7:0/3321279019 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb88015070 con 0x55ac1e352af0 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.551+0000 7ffb757fa640 1 -- 172.21.15.7:0/3321279019 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8801a940 con 0x55ac1e352af0 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb9806f740 1 -- 172.21.15.7:0/3321279019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 msgr2=0x55ac1e352ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb8effd640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ffb780a2da0 0x7ffb780a5260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55ac1e34fec0 tx=0x7ffb80005a20 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb9806f740 1 --2- 172.21.15.7:0/3321279019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 0x55ac1e352ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009eb0 tx=0x7ffb880107a0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb9806f740 1 -- 172.21.15.7:0/3321279019 shutdown_connections 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb9806f740 1 --2- 172.21.15.7:0/3321279019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e3600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb9806f740 1 --2- 172.21.15.7:0/3321279019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e35d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb9806f740 1 --2- 172.21.15.7:0/3321279019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 0x55ac1e352ed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb9806f740 1 -- 172.21.15.7:0/3321279019 >> 172.21.15.7:0/3321279019 conn(0x55ac1e34f060 msgr2=0x55ac1e34f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb9806f740 1 -- 172.21.15.7:0/3321279019 shutdown_connections 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb9806f740 1 -- 172.21.15.7:0/3321279019 wait complete. 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.552+0000 7ffb9806f740 1 -- start start 2024-09-10T10:49:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb9806f740 1 Processor -- start 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e35ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e35b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb9806f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e360880 con 0x55ac1e352af0 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb9806f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e240290 con 0x55ac1e353410 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb9806f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ac1e360a00 con 0x55ac1e35dcb0 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb9806f740 10 client.0 _ll_register_callbacks cb 0x55ac1e271140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e35ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e35ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53374/0 (socket says 172.21.15.7:53374) 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:47684/0 (socket says 172.21.15.7:47684) 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8effd640 1 -- 172.21.15.7:0/4116767077 learned_addr learned my addr 172.21.15.7:0/4116767077 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8f7fe640 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e35b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8e7fc640 1 -- 172.21.15.7:0/4116767077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 msgr2=0x55ac1e35b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e35b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8e7fc640 1 -- 172.21.15.7:0/4116767077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 msgr2=0x55ac1e35ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e35ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8e7fc640 1 -- 172.21.15.7:0/4116767077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1dfca210 con 0x55ac1e353410 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb840ccb50 tx=0x7ffb840d2b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d25d106cee51eec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.553+0000 7ffb72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb72ff5640 1 -- 172.21.15.7:0/4116767077 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84084e60 con 0x55ac1e353410 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb9806f740 1 -- 172.21.15.7:0/4116767077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb88015070 con 0x55ac1e353410 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb72ff5640 1 -- 172.21.15.7:0/4116767077 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7ffb84097f90 con 0x55ac1e353410 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb72ff5640 1 -- 172.21.15.7:0/4116767077 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb840b4050 con 0x55ac1e353410 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb9806f740 10 client.25220 fetch_fsmap learned FSMap version 29 2024-09-10T10:49:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb9806f740 1 -- 172.21.15.7:0/4116767077 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac1e096620 con 0x55ac1e353410 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb72ff5640 1 -- 172.21.15.7:0/4116767077 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb88015070 con 0x55ac1e353410 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb72ff5640 1 -- 172.21.15.7:0/4116767077 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7ffb84010ac0 con 0x55ac1e353410 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb9806f740 10 client.25220 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb9806f740 10 client.25220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb9806f740 20 client.25220 populate_metadata read hostname 'smithi007' 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb9806f740 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ac1e36dd40 0x55ac1e370130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb9806f740 4 client.25220 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb9806f740 1 -- 172.21.15.7:0/4116767077 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac1e2c4a20 con 0x55ac1e36dd40 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.554+0000 7ffb8effd640 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ac1e36dd40 0x55ac1e370130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.555+0000 7ffb8effd640 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ac1e36dd40 0x55ac1e370130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6074afdc97ba566c server_cookie=3c67c823116ac606 in_seq=0 out_seq=0 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.555+0000 7ffb72ff5640 10 client.25220 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb72ff5640 1 -- 172.21.15.7:0/4116767077 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac1e2c4a20 con 0x55ac1e36dd40 2024-09-10T10:49:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb72ff5640 10 client.25220 handle_command_reply: tid=0 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 client.25220 shutdown 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 -- 172.21.15.7:0/4116767077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 msgr2=0x55ac1e35b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb840ccb50 tx=0x7ffb840d2b40 comp rx=0 tx=0).stop 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 -- 172.21.15.7:0/4116767077 shutdown_connections 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ac1e36dd40 0x55ac1e370130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e35b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b180 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb72ff5640 0 client.25220 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 --2- 172.21.15.7:0/4116767077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e35ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 -- 172.21.15.7:0/4116767077 >> 172.21.15.7:0/4116767077 conn(0x55ac1e34f060 msgr2=0x55ac1e354dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 -- 172.21.15.7:0/4116767077 shutdown_connections 2024-09-10T10:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 1 -- 172.21.15.7:0/4116767077 wait complete. 2024-09-10T10:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.556+0000 7ffb9806f740 20 client.25220 trim_cache size 0 max 16384 2024-09-10T10:49:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb9806f740 1 Processor -- start 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb9806f740 1 -- start start 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e3555b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e3759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb9806f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901b2280 con 0x55ac1e352af0 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb9806f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901afb20 con 0x55ac1e353410 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb9806f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901ad970 con 0x55ac1e35dcb0 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e3555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e3555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53380/0 (socket says 172.21.15.7:53380) 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb8effd640 1 -- 172.21.15.7:0/2758250185 learned_addr learned my addr 172.21.15.7:0/2758250185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/2758250185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.572+0000 7ffb8f7fe640 1 --2- 172.21.15.7:0/2758250185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e3759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.573+0000 7ffb8effd640 1 -- 172.21.15.7:0/2758250185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 msgr2=0x55ac1e3759b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.573+0000 7ffb8effd640 1 --2- 172.21.15.7:0/2758250185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e3759b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.573+0000 7ffb8effd640 1 -- 172.21.15.7:0/2758250185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 msgr2=0x55ac1e35b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.573+0000 7ffb8effd640 1 --2- 172.21.15.7:0/2758250185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.573+0000 7ffb8effd640 1 -- 172.21.15.7:0/2758250185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1e34fcf0 con 0x55ac1e352af0 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.573+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/2758250185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.573+0000 7ffb8effd640 1 --2- 172.21.15.7:0/2758250185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e3555b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ffb80031b60 tx=0x7ffb8003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb54b5de5fec7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.573+0000 7ffb74ff9640 1 -- 172.21.15.7:0/2758250185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80035040 con 0x55ac1e352af0 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.573+0000 7ffb74ff9640 1 -- 172.21.15.7:0/2758250185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb80005520 con 0x55ac1e352af0 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.573+0000 7ffb74ff9640 1 -- 172.21.15.7:0/2758250185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80053650 con 0x55ac1e352af0 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 -- 172.21.15.7:0/2758250185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 msgr2=0x55ac1e3555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 --2- 172.21.15.7:0/2758250185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e3555b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ffb80031b60 tx=0x7ffb8003f000 comp rx=0 tx=0).stop 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 -- 172.21.15.7:0/2758250185 shutdown_connections 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 --2- 172.21.15.7:0/2758250185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e3759b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 --2- 172.21.15.7:0/2758250185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e353410 0x55ac1e35b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 --2- 172.21.15.7:0/2758250185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e352af0 0x55ac1e3555b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 -- 172.21.15.7:0/2758250185 >> 172.21.15.7:0/2758250185 conn(0x55ac1e34f060 msgr2=0x55ac1e35ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 -- 172.21.15.7:0/2758250185 shutdown_connections 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 -- 172.21.15.7:0/2758250185 wait complete. 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 -- start start 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 Processor -- start 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 0x55ac1e366330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e3578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e357e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901adfe0 con 0x55ac1e353410 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901af450 con 0x55ac1e352af0 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.574+0000 7ffb9806f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901ae6b0 con 0x55ac1e35dcb0 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb9806f740 10 client.0 _ll_register_callbacks cb 0x7ffb84009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e3578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e3578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53394/0 (socket says 172.21.15.7:53394) 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8e7fc640 1 -- 172.21.15.7:0/424778863 learned_addr learned my addr 172.21.15.7:0/424778863 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8f7fe640 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e357e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8effd640 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 0x55ac1e366330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8e7fc640 1 -- 172.21.15.7:0/424778863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 msgr2=0x55ac1e357e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e357e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8e7fc640 1 -- 172.21.15.7:0/424778863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 msgr2=0x55ac1e366330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 0x55ac1e366330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8e7fc640 1 -- 172.21.15.7:0/424778863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1e358650 con 0x55ac1e353410 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb8e7fc640 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e3578e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffb84099bd0 tx=0x7ffb84097cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357e36c20c81d43f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb72ff5640 1 -- 172.21.15.7:0/424778863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb840b8060 con 0x55ac1e353410 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb72ff5640 1 -- 172.21.15.7:0/424778863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7ffb840a4100 con 0x55ac1e353410 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb9806f740 1 -- 172.21.15.7:0/424778863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb80005520 con 0x55ac1e353410 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb72ff5640 1 -- 172.21.15.7:0/424778863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb84084e60 con 0x55ac1e353410 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb9806f740 10 client.15543 fetch_fsmap learned FSMap version 29 2024-09-10T10:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.575+0000 7ffb9806f740 1 -- 172.21.15.7:0/424778863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac1e3d8900 con 0x55ac1e353410 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.576+0000 7ffb72ff5640 1 -- 172.21.15.7:0/424778863 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb80005520 con 0x55ac1e353410 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.576+0000 7ffb72ff5640 1 -- 172.21.15.7:0/424778863 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7ffb8409ad70 con 0x55ac1e353410 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.576+0000 7ffb9806f740 10 client.15543 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.576+0000 7ffb9806f740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.576+0000 7ffb9806f740 20 client.15543 populate_metadata read hostname 'smithi007' 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.576+0000 7ffb9806f740 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ac1e36dd40 0x55ac1e3db110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.576+0000 7ffb9806f740 4 client.15543 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.576+0000 7ffb9806f740 1 -- 172.21.15.7:0/424778863 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac1e2bed70 con 0x55ac1e36dd40 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.576+0000 7ffb8effd640 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ac1e36dd40 0x55ac1e3db110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.577+0000 7ffb72ff5640 10 client.15543 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.577+0000 7ffb8effd640 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ac1e36dd40 0x55ac1e3db110 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3edc86c504b6c95f server_cookie=fdf0d2a9236d781d in_seq=0 out_seq=0 2024-09-10T10:49:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.652+0000 7ffb72ff5640 1 -- 172.21.15.7:0/424778863 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac1e2bed70 con 0x55ac1e36dd40 2024-09-10T10:49:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.652+0000 7ffb72ff5640 10 client.15543 handle_command_reply: tid=0 2024-09-10T10:49:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.652+0000 7ffb9806f740 1 client.15543 shutdown 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.652+0000 7ffb9806f740 1 -- 172.21.15.7:0/424778863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 msgr2=0x55ac1e3578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.652+0000 7ffb9806f740 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e3578e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffb84099bd0 tx=0x7ffb84097cf0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.652+0000 7ffb9806f740 1 -- 172.21.15.7:0/424778863 shutdown_connections 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.652+0000 7ffb9806f740 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ac1e36dd40 0x55ac1e3db110 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb9806f740 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1e35dcb0 0x55ac1e357e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb9806f740 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac1e353410 0x55ac1e3578e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb72ff5640 0 client.15543 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb9806f740 1 --2- 172.21.15.7:0/424778863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ac1e352af0 0x55ac1e366330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb9806f740 1 -- 172.21.15.7:0/424778863 >> 172.21.15.7:0/424778863 conn(0x55ac1e34f060 msgr2=0x55ac1e34d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb9806f740 1 -- 172.21.15.7:0/424778863 shutdown_connections 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb9806f740 1 -- 172.21.15.7:0/424778863 wait complete. 2024-09-10T10:49:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb9806f740 20 client.15543 trim_cache size 0 max 16384 2024-09-10T10:49:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:49:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:49:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:49:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:49:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb952d9640 1 -- 172.21.15.7:0/3094863002 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ffb780a2da0 msgr2=0x7ffb780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb952d9640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ffb780a2da0 0x7ffb780a5260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55ac1e34fec0 tx=0x7ffb80005a20 comp rx=0 tx=0).stop 2024-09-10T10:49:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb952d9640 1 -- 172.21.15.7:0/3094863002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb901035c0 msgr2=0x7ffb90100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb952d9640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb901035c0 0x7ffb90100e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400e9c0 tx=0x7ffb8400ee80 comp rx=0 tx=0).stop 2024-09-10T10:49:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb952d9640 1 -- 172.21.15.7:0/3094863002 shutdown_connections 2024-09-10T10:49:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb952d9640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7ffb780a2da0 0x7ffb780a5260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb952d9640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9010c0d0 0x7ffb90101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb952d9640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb901035c0 0x7ffb90100e30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb952d9640 1 --2- 172.21.15.7:0/3094863002 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb90102ca0 0x7ffb901008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.653+0000 7ffb952d9640 1 -- 172.21.15.7:0/3094863002 >> 172.21.15.7:0/3094863002 conn(0x7ffb900fc6b0 msgr2=0x7ffb900fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.654+0000 7ffb952d9640 1 -- 172.21.15.7:0/3094863002 shutdown_connections 2024-09-10T10:49:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:42.654+0000 7ffb952d9640 1 -- 172.21.15.7:0/3094863002 wait complete. 2024-09-10T10:49:43.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 387268 12.32 MB/sec execute 43 sec latency 23.938 ms 2024-09-10T10:49:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:43 smithi103 ceph-mon[35160]: pgmap v118: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2024-09-10T10:49:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:43 smithi007 ceph-mon[30873]: pgmap v118: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2024-09-10T10:49:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:43 smithi133 ceph-mon[35282]: pgmap v118: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2024-09-10T10:49:44.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 390378 12.44 MB/sec execute 44 sec latency 14.928 ms 2024-09-10T10:49:45.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 392260 12.32 MB/sec execute 45 sec latency 38.550 ms 2024-09-10T10:49:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:45 smithi103 ceph-mon[35160]: pgmap v119: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2024-09-10T10:49:45.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:45 smithi007 ceph-mon[30873]: pgmap v119: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2024-09-10T10:49:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:45 smithi133 ceph-mon[35282]: pgmap v119: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 255 op/s 2024-09-10T10:49:46.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 394436 12.35 MB/sec execute 46 sec latency 14.749 ms 2024-09-10T10:49:47.509 INFO:tasks.workunit.client.0.smithi007.stdout: 1 397443 12.46 MB/sec execute 47 sec latency 13.669 ms 2024-09-10T10:49:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:47 smithi103 ceph-mon[35160]: pgmap v120: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-09-10T10:49:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:47 smithi007 ceph-mon[30873]: pgmap v120: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-09-10T10:49:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:47 smithi133 ceph-mon[35282]: pgmap v120: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-09-10T10:49:48.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 399344 12.35 MB/sec execute 48 sec latency 13.318 ms 2024-09-10T10:49:49.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 401525 12.37 MB/sec execute 49 sec latency 18.226 ms 2024-09-10T10:49:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:49 smithi103 ceph-mon[35160]: pgmap v121: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 318 op/s 2024-09-10T10:49:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:49 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:49:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:49 smithi007 ceph-mon[30873]: pgmap v121: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 318 op/s 2024-09-10T10:49:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:49 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:49:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:49 smithi133 ceph-mon[35282]: pgmap v121: 97 pgs: 97 active+clean; 105 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 318 op/s 2024-09-10T10:49:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:49 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:49:50.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 404519 12.48 MB/sec execute 50 sec latency 43.156 ms 2024-09-10T10:49:51.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 406492 12.37 MB/sec execute 51 sec latency 12.395 ms 2024-09-10T10:49:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:51 smithi103 ceph-mon[35160]: pgmap v122: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 455 op/s 2024-09-10T10:49:51.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:51 smithi007 ceph-mon[30873]: pgmap v122: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 455 op/s 2024-09-10T10:49:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:51 smithi133 ceph-mon[35282]: pgmap v122: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.6 MiB/s wr, 455 op/s 2024-09-10T10:49:52.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 408671 12.40 MB/sec execute 52 sec latency 15.460 ms 2024-09-10T10:49:53.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 411709 12.51 MB/sec execute 53 sec latency 10.715 ms 2024-09-10T10:49:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:53 smithi103 ceph-mon[35160]: pgmap v123: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-09-10T10:49:53.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:53 smithi007 ceph-mon[30873]: pgmap v123: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-09-10T10:49:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:53 smithi133 ceph-mon[35282]: pgmap v123: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-09-10T10:49:54.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 413842 12.42 MB/sec execute 54 sec latency 17.706 ms 2024-09-10T10:49:55.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 415598 12.41 MB/sec execute 55 sec latency 45.831 ms 2024-09-10T10:49:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.747+0000 7f44ca72c640 1 -- 172.21.15.7:0/1129797850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44c410c0d0 msgr2=0x7f44c410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.747+0000 7f44ca72c640 1 --2- 172.21.15.7:0/1129797850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44c410c0d0 0x7f44c410e550 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009be0 tx=0x7f44bc02a750 comp rx=0 tx=0).stop 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.747+0000 7f44ca72c640 1 -- 172.21.15.7:0/1129797850 shutdown_connections 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.747+0000 7f44ca72c640 1 --2- 172.21.15.7:0/1129797850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44c410c0d0 0x7f44c410e550 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.747+0000 7f44ca72c640 1 --2- 172.21.15.7:0/1129797850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c41035a0 0x7f44c410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.747+0000 7f44ca72c640 1 --2- 172.21.15.7:0/1129797850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44c4102c80 0x7f44c4103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.747+0000 7f44ca72c640 1 -- 172.21.15.7:0/1129797850 >> 172.21.15.7:0/1129797850 conn(0x7f44c4077990 msgr2=0x7f44c40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.747+0000 7f44ca72c640 1 -- 172.21.15.7:0/1129797850 shutdown_connections 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.747+0000 7f44ca72c640 1 -- 172.21.15.7:0/1129797850 wait complete. 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44ca72c640 1 Processor -- start 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44ca72c640 1 -- start start 2024-09-10T10:49:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44ca72c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44c4102c80 0x7f44c419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44ca72c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44c41035a0 0x7f44c419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44ca72c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c410c0d0 0x7f44c419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44ca72c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4111e60 con 0x7f44c410c0d0 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44ca72c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4111ce0 con 0x7f44c4102c80 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44ca72c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4111fe0 con 0x7f44c41035a0 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44c8cc9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c410c0d0 0x7f44c419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44c3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44c4102c80 0x7f44c419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44c8cc9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c410c0d0 0x7f44c419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59766/0 (socket says 172.21.15.7:59766) 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44c3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44c4102c80 0x7f44c419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:56668/0 (socket says 172.21.15.7:56668) 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44c8cc9640 1 -- 172.21.15.7:0/3669835878 learned_addr learned my addr 172.21.15.7:0/3669835878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.748+0000 7f44c37fe640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44c41035a0 0x7f44c419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44c3fff640 1 -- 172.21.15.7:0/3669835878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44c41035a0 msgr2=0x7f44c419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44c3fff640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44c41035a0 0x7f44c419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44c3fff640 1 -- 172.21.15.7:0/3669835878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c410c0d0 msgr2=0x7f44c419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44c3fff640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c410c0d0 0x7f44c419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44c3fff640 1 -- 172.21.15.7:0/3669835878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c419f4f0 con 0x7f44c4102c80 2024-09-10T10:49:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44c3fff640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44c4102c80 0x7f44c419a500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44b400e9f0 tx=0x7f44b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44c17fa640 1 -- 172.21.15.7:0/3669835878 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b400ccc0 con 0x7f44c4102c80 2024-09-10T10:49:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44c17fa640 1 -- 172.21.15.7:0/3669835878 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b4005020 con 0x7f44c4102c80 2024-09-10T10:49:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44ca72c640 1 -- 172.21.15.7:0/3669835878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c419f7e0 con 0x7f44c4102c80 2024-09-10T10:49:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44c17fa640 1 -- 172.21.15.7:0/3669835878 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40049e0 con 0x7f44c4102c80 2024-09-10T10:49:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.749+0000 7f44ca72c640 1 -- 172.21.15.7:0/3669835878 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41a7070 con 0x7f44c4102c80 2024-09-10T10:49:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.752+0000 7f44c17fa640 1 -- 172.21.15.7:0/3669835878 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f44b4005560 con 0x7f44c4102c80 2024-09-10T10:49:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44cd4bd740 1 Processor -- start 2024-09-10T10:49:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44cd4bd740 1 -- start start 2024-09-10T10:49:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 0x55f18b7d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f18b72a2f0 con 0x55f18b7c5af0 2024-09-10T10:49:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f18b6d2270 con 0x55f18b7d0cb0 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f18b7262b0 con 0x55f18b7c6410 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c8cc9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c8cc9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59772/0 (socket says 172.21.15.7:59772) 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c8cc9640 1 -- 172.21.15.7:0/1280555938 learned_addr learned my addr 172.21.15.7:0/1280555938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c37fe640 1 --2- 172.21.15.7:0/1280555938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 0x55f18b7d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c8cc9640 1 -- 172.21.15.7:0/1280555938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 msgr2=0x55f18b7d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/1280555938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c8cc9640 1 -- 172.21.15.7:0/1280555938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 msgr2=0x55f18b7d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/1280555938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 0x55f18b7d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c8cc9640 1 -- 172.21.15.7:0/1280555938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18b6d1340 con 0x55f18b7c5af0 2024-09-10T10:49:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c17fa640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f44ac0a2e20 0x7f44ac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c3fff640 1 --2- 172.21.15.7:0/1280555938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.753+0000 7f44c37fe640 1 --2- 172.21.15.7:0/1280555938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 0x55f18b7d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.754+0000 7f44c3fff640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f44ac0a2e20 0x7f44ac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.754+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/1280555938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7c5ed0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f44bc005c80 tx=0x7f44bc030a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7030f6877c7b69f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.754+0000 7f44c17fa640 1 -- 172.21.15.7:0/3669835878 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f44b40cf340 con 0x7f44c4102c80 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.754+0000 7f44aa7fc640 1 -- 172.21.15.7:0/1280555938 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc042030 con 0x55f18b7c5af0 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.754+0000 7f44aa7fc640 1 -- 172.21.15.7:0/1280555938 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44bc037040 con 0x55f18b7c5af0 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.754+0000 7f44aa7fc640 1 -- 172.21.15.7:0/1280555938 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc036070 con 0x55f18b7c5af0 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 -- 172.21.15.7:0/1280555938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 msgr2=0x55f18b7c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/1280555938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7c5ed0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f44bc005c80 tx=0x7f44bc030a30 comp rx=0 tx=0).stop 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44c3fff640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f44ac0a2e20 0x7f44ac0a52e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44b40ccb40 tx=0x7f44b4086040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 -- 172.21.15.7:0/1280555938 shutdown_connections 2024-09-10T10:49:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/1280555938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 0x55f18b7d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/1280555938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/1280555938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7c5ed0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 -- 172.21.15.7:0/1280555938 >> 172.21.15.7:0/1280555938 conn(0x55f18b7c2060 msgr2=0x55f18b7c2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 -- 172.21.15.7:0/1280555938 shutdown_connections 2024-09-10T10:49:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 -- 172.21.15.7:0/1280555938 wait complete. 2024-09-10T10:49:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 -- start start 2024-09-10T10:49:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.755+0000 7f44cd4bd740 1 Processor -- start 2024-09-10T10:49:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c5af0 0x55f18b7cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f18b7d3880 con 0x55f18b7d0cb0 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f18b6b3290 con 0x55f18b7c5af0 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f18b7d3a00 con 0x55f18b7c6410 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44cd4bd740 10 client.0 _ll_register_callbacks cb 0x55f18b6e4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c8cc9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c8cc9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59780/0 (socket says 172.21.15.7:59780) 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c8cc9640 1 -- 172.21.15.7:0/2932706055 learned_addr learned my addr 172.21.15.7:0/2932706055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c5af0 0x55f18b7cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c8cc9640 1 -- 172.21.15.7:0/2932706055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 msgr2=0x55f18b7ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c8cc9640 1 -- 172.21.15.7:0/2932706055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c5af0 msgr2=0x55f18b7cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c5af0 0x55f18b7cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c8cc9640 1 -- 172.21.15.7:0/2932706055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18b43d210 con 0x55f18b7d0cb0 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c37fe640 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7ce600 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f44bc003bb0 tx=0x7f44bc003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f1369f04b08591 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44a37fe640 1 -- 172.21.15.7:0/2932706055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc036050 con 0x55f18b7d0cb0 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44cd4bd740 1 -- 172.21.15.7:0/2932706055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44bc037040 con 0x55f18b7d0cb0 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.756+0000 7f44a37fe640 1 -- 172.21.15.7:0/2932706055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f44bc0303a0 con 0x55f18b7d0cb0 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44a37fe640 1 -- 172.21.15.7:0/2932706055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44bc05b050 con 0x55f18b7d0cb0 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44cd4bd740 10 client.15555 fetch_fsmap learned FSMap version 30 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44cd4bd740 1 -- 172.21.15.7:0/2932706055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f18b509620 con 0x55f18b7d0cb0 2024-09-10T10:49:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44a37fe640 1 -- 172.21.15.7:0/2932706055 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44bc037040 con 0x55f18b7d0cb0 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44a37fe640 1 -- 172.21.15.7:0/2932706055 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f44bc03c040 con 0x55f18b7d0cb0 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44cd4bd740 10 client.15555 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44cd4bd740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44cd4bd740 20 client.15555 populate_metadata read hostname 'smithi007' 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f18b7e0a80 0x55f18b7e2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44cd4bd740 4 client.15555 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44cd4bd740 1 -- 172.21.15.7:0/2932706055 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f18b737a20 con 0x55f18b7e0a80 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.757+0000 7f44c37fe640 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f18b7e0a80 0x55f18b7e2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.758+0000 7f44c37fe640 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f18b7e0a80 0x55f18b7e2e70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e0be4f625a0472 server_cookie=49aa2d9060836f6e in_seq=0 out_seq=0 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.758+0000 7f44a37fe640 10 client.15555 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.758+0000 7f44a37fe640 1 -- 172.21.15.7:0/2932706055 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f18b737a20 con 0x55f18b7e0a80 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.758+0000 7f44a37fe640 10 client.15555 handle_command_reply: tid=0 2024-09-10T10:49:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.758+0000 7f44cd4bd740 1 client.15555 shutdown 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 1 -- 172.21.15.7:0/2932706055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 msgr2=0x55f18b7ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7ce600 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f44bc003bb0 tx=0x7f44bc003ab0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 1 -- 172.21.15.7:0/2932706055 shutdown_connections 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f18b7e0a80 0x55f18b7e2e70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7ce600 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44a37fe640 0 client.15555 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/2932706055 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c5af0 0x55f18b7cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 1 -- 172.21.15.7:0/2932706055 >> 172.21.15.7:0/2932706055 conn(0x55f18b7c2060 msgr2=0x55f18b7c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 1 -- 172.21.15.7:0/2932706055 shutdown_connections 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 1 -- 172.21.15.7:0/2932706055 wait complete. 2024-09-10T10:49:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.759+0000 7f44cd4bd740 20 client.15555 trim_cache size 0 max 16384 2024-09-10T10:49:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44cd4bd740 1 Processor -- start 2024-09-10T10:49:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44cd4bd740 1 -- start start 2024-09-10T10:49:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c5af0 0x55f18b7ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c6410 0x55f18b7cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7e8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41b2160 con 0x55f18b7d0cb0 2024-09-10T10:49:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41afa30 con 0x55f18b7c6410 2024-09-10T10:49:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41ad880 con 0x55f18b7c5af0 2024-09-10T10:49:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c8cc9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c5af0 0x55f18b7ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c6410 0x55f18b7cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c8cc9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c6410 0x55f18b7cfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:56692/0 (socket says 172.21.15.7:56692) 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c37fe640 1 -- 172.21.15.7:0/28186433 learned_addr learned my addr 172.21.15.7:0/28186433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c37fe640 1 -- 172.21.15.7:0/28186433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c5af0 msgr2=0x55f18b7ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c37fe640 1 --2- 172.21.15.7:0/28186433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c5af0 0x55f18b7ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c37fe640 1 -- 172.21.15.7:0/28186433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 msgr2=0x55f18b7e8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c37fe640 1 --2- 172.21.15.7:0/28186433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7e8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c37fe640 1 -- 172.21.15.7:0/28186433 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18b7c6240 con 0x55f18b7c6410 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.774+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/28186433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44c37fe640 1 --2- 172.21.15.7:0/28186433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c6410 0x55f18b7cfaf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55f18b7c8690 tx=0x7f44b8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb36df7d523cf211 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44a9ffb640 1 -- 172.21.15.7:0/28186433 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8005220 con 0x55f18b7c6410 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44a9ffb640 1 -- 172.21.15.7:0/28186433 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b8004c80 con 0x55f18b7c6410 2024-09-10T10:49:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44a9ffb640 1 -- 172.21.15.7:0/28186433 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8027820 con 0x55f18b7c6410 2024-09-10T10:49:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44cd4bd740 1 -- 172.21.15.7:0/28186433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c6410 msgr2=0x55f18b7cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/28186433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c6410 0x55f18b7cfaf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55f18b7c8690 tx=0x7f44b8009260 comp rx=0 tx=0).stop 2024-09-10T10:49:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44cd4bd740 1 -- 172.21.15.7:0/28186433 shutdown_connections 2024-09-10T10:49:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/28186433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7d0cb0 0x55f18b7e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/28186433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7c6410 0x55f18b7cfaf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/28186433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c5af0 0x55f18b7ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.775+0000 7f44cd4bd740 1 -- 172.21.15.7:0/28186433 >> 172.21.15.7:0/28186433 conn(0x55f18b7c2060 msgr2=0x55f18b7cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 1 -- 172.21.15.7:0/28186433 shutdown_connections 2024-09-10T10:49:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 1 -- 172.21.15.7:0/28186433 wait complete. 2024-09-10T10:49:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 1 -- start start 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 1 Processor -- start 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7d8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 0x55f18b7ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41adef0 con 0x55f18b7c5af0 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41af360 con 0x55f18b7d0cb0 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44c41ae5c0 con 0x55f18b7c6410 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44cd4bd740 10 client.0 _ll_register_callbacks cb 0x7f44b40d2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44c8cc9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7d8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:52952/0 (socket says 172.21.15.7:52952) 2024-09-10T10:49:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.776+0000 7f44c37fe640 1 -- 172.21.15.7:0/3171479399 learned_addr learned my addr 172.21.15.7:0/3171479399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 0x55f18b7ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44c37fe640 1 -- 172.21.15.7:0/3171479399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 msgr2=0x55f18b7ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44c37fe640 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 0x55f18b7ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44c37fe640 1 -- 172.21.15.7:0/3171479399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 msgr2=0x55f18b7d8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44c37fe640 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7d8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44c37fe640 1 -- 172.21.15.7:0/3171479399 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18b7ca700 con 0x55f18b7c6410 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7d8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44c37fe640 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d91d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44b80096c0 tx=0x7f44b8009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=debe345737c75cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44a37fe640 1 -- 172.21.15.7:0/3171479399 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8004a30 con 0x55f18b7c6410 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44cd4bd740 1 -- 172.21.15.7:0/3171479399 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b8004c80 con 0x55f18b7c6410 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44a37fe640 1 -- 172.21.15.7:0/3171479399 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8520+0+0 (secure 0 0 0) 0x7f44b8005000 con 0x55f18b7c6410 2024-09-10T10:49:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44a37fe640 1 -- 172.21.15.7:0/3171479399 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44b8005200 con 0x55f18b7c6410 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44cd4bd740 10 client.25249 fetch_fsmap learned FSMap version 30 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.777+0000 7f44cd4bd740 1 -- 172.21.15.7:0/3171479399 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f18b84ac40 con 0x55f18b7c6410 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44a37fe640 1 -- 172.21.15.7:0/3171479399 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b8004c80 con 0x55f18b7c6410 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44a37fe640 1 -- 172.21.15.7:0/3171479399 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f44b8032470 con 0x55f18b7c6410 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44cd4bd740 10 client.25249 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44cd4bd740 10 client.25249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44cd4bd740 20 client.25249 populate_metadata read hostname 'smithi007' 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f18b84b3a0 0x55f18b84d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44cd4bd740 4 client.25249 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44cd4bd740 1 -- 172.21.15.7:0/3171479399 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f18b731d70 con 0x55f18b84b3a0 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f18b84b3a0 0x55f18b84d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44c8cc9640 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f18b84b3a0 0x55f18b84d7c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b4fcfd61ca9a9d server_cookie=c341e61d29d0b428 in_seq=0 out_seq=0 2024-09-10T10:49:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.778+0000 7f44a37fe640 10 client.25249 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.854+0000 7f44a37fe640 1 -- 172.21.15.7:0/3171479399 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f18b731d70 con 0x55f18b84b3a0 2024-09-10T10:49:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.854+0000 7f44a37fe640 10 client.25249 handle_command_reply: tid=0 2024-09-10T10:49:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.854+0000 7f44cd4bd740 1 client.25249 shutdown 2024-09-10T10:49:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 1 -- 172.21.15.7:0/3171479399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 msgr2=0x55f18b7d91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d91d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44b80096c0 tx=0x7f44b8009d00 comp rx=0 tx=0).stop 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 1 -- 172.21.15.7:0/3171479399 shutdown_connections 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f18b84b3a0 0x55f18b84d7c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f18b7d0cb0 0x55f18b7ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f18b7c6410 0x55f18b7d91d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 1 --2- 172.21.15.7:0/3171479399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f18b7c5af0 0x55f18b7d8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 1 -- 172.21.15.7:0/3171479399 >> 172.21.15.7:0/3171479399 conn(0x55f18b7c2060 msgr2=0x55f18b7c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44a37fe640 0 client.25249 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 1 -- 172.21.15.7:0/3171479399 shutdown_connections 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 1 -- 172.21.15.7:0/3171479399 wait complete. 2024-09-10T10:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.855+0000 7f44cd4bd740 20 client.25249 trim_cache size 0 max 16384 2024-09-10T10:49:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:49:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:49:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:49:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 -- 172.21.15.7:0/3669835878 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f44ac0a2e20 msgr2=0x7f44ac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f44ac0a2e20 0x7f44ac0a52e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44b40ccb40 tx=0x7f44b4086040 comp rx=0 tx=0).stop 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 -- 172.21.15.7:0/3669835878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44c4102c80 msgr2=0x7f44c419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44c4102c80 0x7f44c419a500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44b400e9f0 tx=0x7f44b400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 -- 172.21.15.7:0/3669835878 shutdown_connections 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f44ac0a2e20 0x7f44ac0a52e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c410c0d0 0x7f44c419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44c41035a0 0x7f44c419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 --2- 172.21.15.7:0/3669835878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f44c4102c80 0x7f44c419a500 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 -- 172.21.15.7:0/3669835878 >> 172.21.15.7:0/3669835878 conn(0x7f44c4077990 msgr2=0x7f44c4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 -- 172.21.15.7:0/3669835878 shutdown_connections 2024-09-10T10:49:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:49:55.856+0000 7f44ca72c640 1 -- 172.21.15.7:0/3669835878 wait complete. 2024-09-10T10:49:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:55 smithi103 ceph-mon[35160]: pgmap v124: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-09-10T10:49:55.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:55 smithi007 ceph-mon[30873]: pgmap v124: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-09-10T10:49:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:55 smithi133 ceph-mon[35282]: pgmap v124: 97 pgs: 97 active+clean; 114 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 326 op/s 2024-09-10T10:49:56.475 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:49:56 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: ignoring --setuser ceph since I am not root 2024-09-10T10:49:56.475 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:49:56 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: ignoring --setgroup ceph since I am not root 2024-09-10T10:49:56.475 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:49:56 smithi133 ceph-mds[65473]: -- 172.21.15.133:0/1944296832 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f29ebbb860 con 0x55f29eb99000 2024-09-10T10:49:56.475 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:49:56 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: starting mds.f at 2024-09-10T10:49:56.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 418340 12.45 MB/sec execute 56 sec latency 43.670 ms 2024-09-10T10:49:56.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:56 smithi103 ceph-mon[35160]: daemon mds.f restarted 2024-09-10T10:49:56.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:56 smithi007 ceph-mon[30873]: daemon mds.f restarted 2024-09-10T10:49:56.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:56 smithi133 ceph-mon[35282]: daemon mds.f restarted 2024-09-10T10:49:57.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 420102 12.40 MB/sec execute 57 sec latency 15.864 ms 2024-09-10T10:49:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:57 smithi103 ceph-mon[35160]: pgmap v125: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-09-10T10:49:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:57 smithi103 ceph-mon[35160]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T10:49:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:57 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6834/1353106187,v1:172.21.15.133:6835/1353106187] up:boot 2024-09-10T10:49:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:57 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T10:49:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:57 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:49:57.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:57 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:49:57.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:57 smithi007 ceph-mon[30873]: pgmap v125: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-09-10T10:49:57.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:57 smithi007 ceph-mon[30873]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T10:49:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:57 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6834/1353106187,v1:172.21.15.133:6835/1353106187] up:boot 2024-09-10T10:49:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:57 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T10:49:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:57 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:49:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:57 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:49:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:57 smithi133 ceph-mon[35282]: pgmap v125: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 451 op/s 2024-09-10T10:49:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:57 smithi133 ceph-mon[35282]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T10:49:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:57 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6834/1353106187,v1:172.21.15.133:6835/1353106187] up:boot 2024-09-10T10:49:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:57 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T10:49:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:57 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:49:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:57 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:49:58.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 422546 12.43 MB/sec execute 58 sec latency 12.787 ms 2024-09-10T10:49:59.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 425129 12.43 MB/sec execute 59 sec latency 17.012 ms 2024-09-10T10:49:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:49:59 smithi103 ceph-mon[35160]: pgmap v127: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 315 op/s 2024-09-10T10:49:59.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:49:59 smithi007 ceph-mon[30873]: pgmap v127: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 315 op/s 2024-09-10T10:49:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:49:59 smithi133 ceph-mon[35282]: pgmap v127: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 315 op/s 2024-09-10T10:50:00.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 426732 12.36 MB/sec execute 60 sec latency 60.825 ms 2024-09-10T10:50:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:00 smithi103 ceph-mon[35160]: overall HEALTH_OK 2024-09-10T10:50:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:00 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:50:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:00 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:50:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:00 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:50:00.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:00 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:50:00.937 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:00 smithi007 ceph-mon[30873]: overall HEALTH_OK 2024-09-10T10:50:00.937 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:00 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:50:00.937 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:00 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:50:00.938 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:00 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:50:00.938 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:00 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:50:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:00 smithi133 ceph-mon[35282]: overall HEALTH_OK 2024-09-10T10:50:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:00 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:50:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:00 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:50:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:00 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:50:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:00 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:50:01.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 429013 12.37 MB/sec execute 61 sec latency 23.206 ms 2024-09-10T10:50:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:01 smithi103 ceph-mon[35160]: pgmap v128: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-10T10:50:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:01 smithi007 ceph-mon[30873]: pgmap v128: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-10T10:50:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:01 smithi133 ceph-mon[35282]: pgmap v128: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-10T10:50:02.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 430780 12.33 MB/sec execute 62 sec latency 16.395 ms 2024-09-10T10:50:03.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 433152 12.35 MB/sec execute 63 sec latency 12.269 ms 2024-09-10T10:50:03.925 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:03 smithi103 ceph-mon[35160]: pgmap v129: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-10T10:50:03.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:03 smithi007 ceph-mon[30873]: pgmap v129: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-10T10:50:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:03 smithi133 ceph-mon[35282]: pgmap v129: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-10T10:50:04.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 435522 12.33 MB/sec execute 64 sec latency 14.567 ms 2024-09-10T10:50:05.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 436901 12.27 MB/sec execute 65 sec latency 45.049 ms 2024-09-10T10:50:05.737 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:05 smithi007 ceph-mon[30873]: pgmap v130: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-10T10:50:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:05 smithi103 ceph-mon[35160]: pgmap v130: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-10T10:50:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:05 smithi133 ceph-mon[35282]: pgmap v130: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-10T10:50:06.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 439864 12.35 MB/sec execute 66 sec latency 19.950 ms 2024-09-10T10:50:07.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 441550 12.26 MB/sec execute 67 sec latency 14.814 ms 2024-09-10T10:50:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:07 smithi103 ceph-mon[35160]: pgmap v131: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-09-10T10:50:07.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:07 smithi007 ceph-mon[30873]: pgmap v131: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-09-10T10:50:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:07 smithi133 ceph-mon[35282]: pgmap v131: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 404 op/s 2024-09-10T10:50:08.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 443878 12.29 MB/sec execute 68 sec latency 12.346 ms 2024-09-10T10:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.949+0000 7fba28d8d640 1 -- 172.21.15.7:0/3320279664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba24102470 msgr2=0x7fba2410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.949+0000 7fba28d8d640 1 --2- 172.21.15.7:0/3320279664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba24102470 0x7fba2410e9a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009cd0 tx=0x7fba1c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.949+0000 7fba28d8d640 1 -- 172.21.15.7:0/3320279664 shutdown_connections 2024-09-10T10:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.949+0000 7fba28d8d640 1 --2- 172.21.15.7:0/3320279664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba24102470 0x7fba2410e9a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.949+0000 7fba28d8d640 1 --2- 172.21.15.7:0/3320279664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba24101ad0 0x7fba24101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.949+0000 7fba28d8d640 1 --2- 172.21.15.7:0/3320279664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24107b70 0x7fba24107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.949+0000 7fba28d8d640 1 -- 172.21.15.7:0/3320279664 >> 172.21.15.7:0/3320279664 conn(0x7fba240fd7d0 msgr2=0x7fba240ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.949+0000 7fba28d8d640 1 -- 172.21.15.7:0/3320279664 shutdown_connections 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.949+0000 7fba28d8d640 1 -- 172.21.15.7:0/3320279664 wait complete. 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba28d8d640 1 Processor -- start 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba28d8d640 1 -- start start 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba28d8d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba24101ad0 0x7fba241982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba28d8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba24102470 0x7fba24198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba28d8d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24107b70 0x7fba2419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba28d8d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba24111ee0 con 0x7fba24102470 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba28d8d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba24111d60 con 0x7fba24101ad0 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba28d8d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fba24112060 con 0x7fba24107b70 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba1bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba24102470 0x7fba24198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba1bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba24102470 0x7fba24198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40002/0 (socket says 172.21.15.7:40002) 2024-09-10T10:50:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba1bfff640 1 -- 172.21.15.7:0/10950902 learned_addr learned my addr 172.21.15.7:0/10950902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.950+0000 7fba2259c640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba24101ad0 0x7fba241982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba22d9d640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24107b70 0x7fba2419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba1bfff640 1 -- 172.21.15.7:0/10950902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24107b70 msgr2=0x7fba2419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba1bfff640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24107b70 0x7fba2419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba1bfff640 1 -- 172.21.15.7:0/10950902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba24101ad0 msgr2=0x7fba241982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba1bfff640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba24101ad0 0x7fba241982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba1bfff640 1 -- 172.21.15.7:0/10950902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2419d2c0 con 0x7fba24102470 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba1bfff640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba24102470 0x7fba24198810 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00ea60 tx=0x7fba0c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba1b7fe640 1 -- 172.21.15.7:0/10950902 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c00cde0 con 0x7fba24102470 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba1b7fe640 1 -- 172.21.15.7:0/10950902 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0c0035f0 con 0x7fba24102470 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba28d8d640 1 -- 172.21.15.7:0/10950902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2419d5b0 con 0x7fba24102470 2024-09-10T10:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba1b7fe640 1 -- 172.21.15.7:0/10950902 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c010670 con 0x7fba24102470 2024-09-10T10:50:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.951+0000 7fba28d8d640 1 -- 172.21.15.7:0/10950902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba241a4e40 con 0x7fba24102470 2024-09-10T10:50:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.954+0000 7fba1b7fe640 1 -- 172.21.15.7:0/10950902 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fba0c0037a0 con 0x7fba24102470 2024-09-10T10:50:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.954+0000 7fba2bc01740 1 Processor -- start 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.954+0000 7fba2bc01740 1 -- start start 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e89947ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e899488770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 0x55e89948b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba2bc01740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e8993e22f0 con 0x55e89947e410 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba2bc01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e89938a270 con 0x55e89947daf0 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba2bc01740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e8993de2b0 con 0x55e899488cb0 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba22d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e89947ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba1bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 0x55e89948b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba2259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e899488770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba22d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e89947ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:54756/0 (socket says 172.21.15.7:54756) 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba22d9d640 1 -- 172.21.15.7:0/60923247 learned_addr learned my addr 172.21.15.7:0/60923247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba2259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e899488770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40004/0 (socket says 172.21.15.7:40004) 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba22d9d640 1 -- 172.21.15.7:0/60923247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 msgr2=0x55e89948b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba22d9d640 1 --2- 172.21.15.7:0/60923247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 0x55e89948b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba1b7fe640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fba100a2e20 0x7fba100a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba22d9d640 1 -- 172.21.15.7:0/60923247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 msgr2=0x55e899488770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba22d9d640 1 --2- 172.21.15.7:0/60923247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e899488770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba22d9d640 1 -- 172.21.15.7:0/60923247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e899389340 con 0x55e89947daf0 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba1bfff640 1 --2- 172.21.15.7:0/60923247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 0x55e89948b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba1b7fe640 1 -- 172.21.15.7:0/10950902 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fba0c0cf6c0 con 0x7fba24102470 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba22d9d640 1 --2- 172.21.15.7:0/60923247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e89947ded0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009cd0 tx=0x7fba1c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=195a3c93811558cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.955+0000 7fba22d9d640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fba100a2e20 0x7fba100a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba18ff9640 1 -- 172.21.15.7:0/60923247 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c048070 con 0x55e89947daf0 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba18ff9640 1 -- 172.21.15.7:0/60923247 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c036070 con 0x55e89947daf0 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba18ff9640 1 -- 172.21.15.7:0/60923247 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0305a0 con 0x55e89947daf0 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba2bc01740 1 -- 172.21.15.7:0/60923247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 msgr2=0x55e89947ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba22d9d640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fba100a2e20 0x7fba100a52e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fba1c045000 tx=0x7fba1c030c50 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba2bc01740 1 --2- 172.21.15.7:0/60923247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e89947ded0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009cd0 tx=0x7fba1c009a20 comp rx=0 tx=0).stop 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba2bc01740 1 -- 172.21.15.7:0/60923247 shutdown_connections 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba2bc01740 1 --2- 172.21.15.7:0/60923247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 0x55e89948b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba2bc01740 1 --2- 172.21.15.7:0/60923247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e899488770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba2bc01740 1 --2- 172.21.15.7:0/60923247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e89947ded0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba2bc01740 1 -- 172.21.15.7:0/60923247 >> 172.21.15.7:0/60923247 conn(0x55e89947a060 msgr2=0x55e89947a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.956+0000 7fba2bc01740 1 -- 172.21.15.7:0/60923247 shutdown_connections 2024-09-10T10:50:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2bc01740 1 -- 172.21.15.7:0/60923247 wait complete. 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2bc01740 1 -- start start 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2bc01740 1 Processor -- start 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947daf0 0x55e899485b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947e410 0x55e899486050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 0x55e899486590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2bc01740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e89948b880 con 0x55e899488cb0 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2bc01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e89936b290 con 0x55e89947e410 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2bc01740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e89948ba00 con 0x55e89947daf0 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2bc01740 10 client.0 _ll_register_callbacks cb 0x55e89939c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947daf0 0x55e899485b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba22d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 0x55e899486590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba1bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947e410 0x55e899486050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947daf0 0x55e899485b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:33828/0 (socket says 172.21.15.7:33828) 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.957+0000 7fba2259c640 1 -- 172.21.15.7:0/2410656425 learned_addr learned my addr 172.21.15.7:0/2410656425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba2259c640 1 -- 172.21.15.7:0/2410656425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947e410 msgr2=0x55e899486050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba2259c640 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947e410 0x55e899486050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba2259c640 1 -- 172.21.15.7:0/2410656425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 msgr2=0x55e899486590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba2259c640 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 0x55e899486590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba2259c640 1 -- 172.21.15.7:0/2410656425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8990f5210 con 0x55e89947daf0 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba22d9d640 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 0x55e899486590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba2259c640 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947daf0 0x55e899485b10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fba140126e0 tx=0x7fba14012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b426c4e1e9f7554f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba09ffb640 1 -- 172.21.15.7:0/2410656425 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140120d0 con 0x55e89947daf0 2024-09-10T10:50:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba09ffb640 1 -- 172.21.15.7:0/2410656425 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fba1400f920 con 0x55e89947daf0 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba2bc01740 1 -- 172.21.15.7:0/2410656425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1c036070 con 0x55e89947daf0 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba09ffb640 1 -- 172.21.15.7:0/2410656425 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fba14012270 con 0x55e89947daf0 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba2bc01740 10 client.25267 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba2bc01740 1 -- 172.21.15.7:0/2410656425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e8991c1620 con 0x55e89947daf0 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba09ffb640 1 -- 172.21.15.7:0/2410656425 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c036070 con 0x55e89947daf0 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.958+0000 7fba09ffb640 1 -- 172.21.15.7:0/2410656425 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fba1401c420 con 0x55e89947daf0 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.959+0000 7fba2bc01740 10 client.25267 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.959+0000 7fba2bc01740 10 client.25267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.959+0000 7fba2bc01740 20 client.25267 populate_metadata read hostname 'smithi007' 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.959+0000 7fba2bc01740 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e899498bd0 0x55e89949b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.959+0000 7fba2bc01740 4 client.25267 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.959+0000 7fba2bc01740 1 -- 172.21.15.7:0/2410656425 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8993efa20 con 0x55e899498bd0 2024-09-10T10:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.959+0000 7fba1bfff640 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e899498bd0 0x55e89949b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.959+0000 7fba1bfff640 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e899498bd0 0x55e89949b000 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9bcbf40266fc3cc server_cookie=5f52874c4918a9fd in_seq=0 out_seq=0 2024-09-10T10:50:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.959+0000 7fba09ffb640 10 client.25267 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba09ffb640 1 -- 172.21.15.7:0/2410656425 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8993efa20 con 0x55e899498bd0 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba09ffb640 10 client.25267 handle_command_reply: tid=0 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 client.25267 shutdown 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 -- 172.21.15.7:0/2410656425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947daf0 msgr2=0x55e899485b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947daf0 0x55e899485b10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fba140126e0 tx=0x7fba14012ba0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 -- 172.21.15.7:0/2410656425 shutdown_connections 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e899498bd0 0x55e89949b000 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 0x55e899486590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947e410 0x55e899486050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 --2- 172.21.15.7:0/2410656425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947daf0 0x55e899485b10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba09ffb640 0 client.25267 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 -- 172.21.15.7:0/2410656425 >> 172.21.15.7:0/2410656425 conn(0x55e89947a060 msgr2=0x55e89947fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 -- 172.21.15.7:0/2410656425 shutdown_connections 2024-09-10T10:50:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 1 -- 172.21.15.7:0/2410656425 wait complete. 2024-09-10T10:50:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.960+0000 7fba2bc01740 20 client.25267 trim_cache size 0 max 16384 2024-09-10T10:50:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2bc01740 1 Processor -- start 2024-09-10T10:50:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2bc01740 1 -- start start 2024-09-10T10:50:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e899480560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947e410 0x55e8994863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 0x55e8994a0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2bc01740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba241affe0 con 0x55e899488cb0 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2bc01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba241ad8b0 con 0x55e89947daf0 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2bc01740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fba241ab700 con 0x55e89947e410 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba22d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 0x55e8994a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e899480560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba1bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947e410 0x55e8994863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e899480560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:54782/0 (socket says 172.21.15.7:54782) 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba2259c640 1 -- 172.21.15.7:0/78826603 learned_addr learned my addr 172.21.15.7:0/78826603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.976+0000 7fba1bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947e410 0x55e8994863c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:33842/0 (socket says 172.21.15.7:33842) 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.977+0000 7fba22d9d640 1 -- 172.21.15.7:0/78826603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947e410 msgr2=0x55e8994863c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.977+0000 7fba22d9d640 1 --2- 172.21.15.7:0/78826603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947e410 0x55e8994863c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.977+0000 7fba22d9d640 1 -- 172.21.15.7:0/78826603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 msgr2=0x55e899480560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.977+0000 7fba22d9d640 1 --2- 172.21.15.7:0/78826603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e899480560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.977+0000 7fba22d9d640 1 -- 172.21.15.7:0/78826603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8994752d0 con 0x55e899488cb0 2024-09-10T10:50:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.977+0000 7fba2259c640 1 --2- 172.21.15.7:0/78826603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e899480560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.977+0000 7fba22d9d640 1 --2- 172.21.15.7:0/78826603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 0x55e8994a0960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fba1c02ae00 tx=0x7fba1c050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9470b25293d62727 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.977+0000 7fba0bfff640 1 -- 172.21.15.7:0/78826603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c05f070 con 0x55e899488cb0 2024-09-10T10:50:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.977+0000 7fba0bfff640 1 -- 172.21.15.7:0/78826603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c053040 con 0x55e899488cb0 2024-09-10T10:50:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.977+0000 7fba0bfff640 1 -- 172.21.15.7:0/78826603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c048040 con 0x55e899488cb0 2024-09-10T10:50:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 -- 172.21.15.7:0/78826603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 msgr2=0x55e8994a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 --2- 172.21.15.7:0/78826603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 0x55e8994a0960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fba1c02ae00 tx=0x7fba1c050000 comp rx=0 tx=0).stop 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 -- 172.21.15.7:0/78826603 shutdown_connections 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 --2- 172.21.15.7:0/78826603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e899488cb0 0x55e8994a0960 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 --2- 172.21.15.7:0/78826603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e89947e410 0x55e8994863c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 --2- 172.21.15.7:0/78826603 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e899480560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 -- 172.21.15.7:0/78826603 >> 172.21.15.7:0/78826603 conn(0x55e89947a060 msgr2=0x55e899487b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 -- 172.21.15.7:0/78826603 shutdown_connections 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 -- 172.21.15.7:0/78826603 wait complete. 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 -- start start 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 Processor -- start 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e8994834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e8994a0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 0x55e8994a0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba241abd70 con 0x55e89947e410 2024-09-10T10:50:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba241ad1e0 con 0x55e89947daf0 2024-09-10T10:50:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fba241ac440 con 0x55e899488cb0 2024-09-10T10:50:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2bc01740 10 client.0 _ll_register_callbacks cb 0x7fba14009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba1bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e8994a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e8994834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba1bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e8994a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40042/0 (socket says 172.21.15.7:40042) 2024-09-10T10:50:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba2259c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e8994834d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:54798/0 (socket says 172.21.15.7:54798) 2024-09-10T10:50:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba1bfff640 1 -- 172.21.15.7:0/4202425950 learned_addr learned my addr 172.21.15.7:0/4202425950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.978+0000 7fba22d9d640 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 0x55e8994a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba1bfff640 1 -- 172.21.15.7:0/4202425950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 msgr2=0x55e8994a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba1bfff640 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 0x55e8994a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba1bfff640 1 -- 172.21.15.7:0/4202425950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 msgr2=0x55e8994834d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba1bfff640 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e8994834d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba1bfff640 1 -- 172.21.15.7:0/4202425950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8994a0f30 con 0x55e89947e410 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba1bfff640 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e8994a0300 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0cd410 tx=0x7fba0c003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5cd646547fc877 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba09ffb640 1 -- 172.21.15.7:0/4202425950 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c09ff90 con 0x55e89947e410 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba09ffb640 1 -- 172.21.15.7:0/4202425950 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fba0c0cd090 con 0x55e89947e410 2024-09-10T10:50:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba2bc01740 1 -- 172.21.15.7:0/4202425950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1c053040 con 0x55e89947e410 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba09ffb640 1 -- 172.21.15.7:0/4202425950 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fba0c0a4850 con 0x55e89947e410 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba2bc01740 10 client.15585 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba2bc01740 1 -- 172.21.15.7:0/4202425950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e899507830 con 0x55e89947e410 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.979+0000 7fba09ffb640 1 -- 172.21.15.7:0/4202425950 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c053040 con 0x55e89947e410 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.980+0000 7fba09ffb640 1 -- 172.21.15.7:0/4202425950 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fba0c0b4070 con 0x55e89947e410 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.980+0000 7fba2bc01740 10 client.15585 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.980+0000 7fba2bc01740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.980+0000 7fba2bc01740 20 client.15585 populate_metadata read hostname 'smithi007' 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.980+0000 7fba2bc01740 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e899507fb0 0x55e89950a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.980+0000 7fba2bc01740 4 client.15585 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.980+0000 7fba2bc01740 1 -- 172.21.15.7:0/4202425950 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8993e9d70 con 0x55e899507fb0 2024-09-10T10:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.980+0000 7fba2259c640 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e899507fb0 0x55e89950a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.980+0000 7fba2259c640 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e899507fb0 0x55e89950a3d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32c81d5d14dfcb45 server_cookie=9971317ac70ca77e in_seq=0 out_seq=0 2024-09-10T10:50:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:08.980+0000 7fba09ffb640 10 client.15585 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba09ffb640 1 -- 172.21.15.7:0/4202425950 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8993e9d70 con 0x55e899507fb0 2024-09-10T10:50:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba09ffb640 10 client.15585 handle_command_reply: tid=0 2024-09-10T10:50:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 client.15585 shutdown 2024-09-10T10:50:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 -- 172.21.15.7:0/4202425950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 msgr2=0x55e8994a0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e8994a0300 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0cd410 tx=0x7fba0c003a90 comp rx=0 tx=0).stop 2024-09-10T10:50:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 -- 172.21.15.7:0/4202425950 shutdown_connections 2024-09-10T10:50:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e899507fb0 0x55e89950a3d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e899488cb0 0x55e8994a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e89947e410 0x55e8994a0300 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 --2- 172.21.15.7:0/4202425950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e89947daf0 0x55e8994834d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 -- 172.21.15.7:0/4202425950 >> 172.21.15.7:0/4202425950 conn(0x55e89947a060 msgr2=0x55e899487370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba09ffb640 0 client.15585 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 -- 172.21.15.7:0/4202425950 shutdown_connections 2024-09-10T10:50:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 1 -- 172.21.15.7:0/4202425950 wait complete. 2024-09-10T10:50:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.051+0000 7fba2bc01740 20 client.15585 trim_cache size 0 max 16384 2024-09-10T10:50:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:50:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:50:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:50:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:50:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 -- 172.21.15.7:0/10950902 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fba100a2e20 msgr2=0x7fba100a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fba100a2e20 0x7fba100a52e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fba1c045000 tx=0x7fba1c030c50 comp rx=0 tx=0).stop 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 -- 172.21.15.7:0/10950902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba24102470 msgr2=0x7fba24198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba24102470 0x7fba24198810 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00ea60 tx=0x7fba0c00ef20 comp rx=0 tx=0).stop 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 -- 172.21.15.7:0/10950902 shutdown_connections 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba24107b70 0x7fba2419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fba100a2e20 0x7fba100a52e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba24102470 0x7fba24198810 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 --2- 172.21.15.7:0/10950902 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba24101ad0 0x7fba241982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 -- 172.21.15.7:0/10950902 >> 172.21.15.7:0/10950902 conn(0x7fba240fd7d0 msgr2=0x7fba240fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 -- 172.21.15.7:0/10950902 shutdown_connections 2024-09-10T10:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:09.052+0000 7fba28d8d640 1 -- 172.21.15.7:0/10950902 wait complete. 2024-09-10T10:50:09.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 446573 12.30 MB/sec execute 69 sec latency 63.860 ms 2024-09-10T10:50:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:09 smithi103 ceph-mon[35160]: pgmap v132: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-09-10T10:50:09.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:09 smithi007 ceph-mon[30873]: pgmap v132: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-09-10T10:50:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:09 smithi133 ceph-mon[35282]: pgmap v132: 97 pgs: 97 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 358 op/s 2024-09-10T10:50:10.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 448331 12.28 MB/sec execute 70 sec latency 25.666 ms 2024-09-10T10:50:11.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 450890 12.31 MB/sec execute 71 sec latency 12.764 ms 2024-09-10T10:50:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:11 smithi103 ceph-mon[35160]: pgmap v133: 97 pgs: 97 active+clean; 108 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-10T10:50:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:11 smithi007 ceph-mon[30873]: pgmap v133: 97 pgs: 97 active+clean; 108 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-10T10:50:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:11 smithi133 ceph-mon[35282]: pgmap v133: 97 pgs: 97 active+clean; 108 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-10T10:50:12.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 453681 12.32 MB/sec execute 72 sec latency 15.397 ms 2024-09-10T10:50:13.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 455505 12.30 MB/sec execute 73 sec latency 37.875 ms 2024-09-10T10:50:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:13 smithi103 ceph-mon[35160]: pgmap v134: 97 pgs: 97 active+clean; 108 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 344 op/s 2024-09-10T10:50:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:13 smithi007 ceph-mon[30873]: pgmap v134: 97 pgs: 97 active+clean; 108 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 344 op/s 2024-09-10T10:50:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:13 smithi133 ceph-mon[35282]: pgmap v134: 97 pgs: 97 active+clean; 108 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 344 op/s 2024-09-10T10:50:14.510 INFO:tasks.workunit.client.0.smithi007.stdout: 1 457946 12.32 MB/sec execute 74 sec latency 12.525 ms 2024-09-10T10:50:15.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 458886 12.42 MB/sec execute 75 sec latency 72.649 ms 2024-09-10T10:50:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:15 smithi103 ceph-mon[35160]: pgmap v135: 97 pgs: 97 active+clean; 108 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 344 op/s 2024-09-10T10:50:15.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:15 smithi007 ceph-mon[30873]: pgmap v135: 97 pgs: 97 active+clean; 108 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 344 op/s 2024-09-10T10:50:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:15 smithi133 ceph-mon[35282]: pgmap v135: 97 pgs: 97 active+clean; 108 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 344 op/s 2024-09-10T10:50:16.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 462001 12.51 MB/sec execute 76 sec latency 16.216 ms 2024-09-10T10:50:17.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 464292 12.46 MB/sec execute 77 sec latency 18.064 ms 2024-09-10T10:50:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:17 smithi103 ceph-mon[35160]: pgmap v136: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 475 op/s 2024-09-10T10:50:17.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:17 smithi007 ceph-mon[30873]: pgmap v136: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 475 op/s 2024-09-10T10:50:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:17 smithi133 ceph-mon[35282]: pgmap v136: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 475 op/s 2024-09-10T10:50:18.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 466251 12.46 MB/sec execute 78 sec latency 12.007 ms 2024-09-10T10:50:19.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 469246 12.54 MB/sec execute 79 sec latency 12.139 ms 2024-09-10T10:50:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:19 smithi103 ceph-mon[35160]: pgmap v137: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 317 op/s 2024-09-10T10:50:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:19 smithi007 ceph-mon[30873]: pgmap v137: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 317 op/s 2024-09-10T10:50:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:19 smithi133 ceph-mon[35282]: pgmap v137: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 317 op/s 2024-09-10T10:50:20.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 471572 12.48 MB/sec execute 80 sec latency 32.928 ms 2024-09-10T10:50:21.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 473453 12.47 MB/sec execute 81 sec latency 29.458 ms 2024-09-10T10:50:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:21 smithi103 ceph-mon[35160]: pgmap v138: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 454 op/s 2024-09-10T10:50:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:21 smithi007 ceph-mon[30873]: pgmap v138: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 454 op/s 2024-09-10T10:50:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:21 smithi133 ceph-mon[35282]: pgmap v138: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 454 op/s 2024-09-10T10:50:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 -- 172.21.15.7:0/47092488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd908107b80 msgr2=0x7fd908107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 --2- 172.21.15.7:0/47092488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd908107b80 0x7fd908107f60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd904009930 tx=0x7fd90402a420 comp rx=0 tx=0).stop 2024-09-10T10:50:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 -- 172.21.15.7:0/47092488 shutdown_connections 2024-09-10T10:50:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 --2- 172.21.15.7:0/47092488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd908102480 0x7fd90810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 --2- 172.21.15.7:0/47092488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd908101ae0 0x7fd908101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 --2- 172.21.15.7:0/47092488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd908107b80 0x7fd908107f60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 -- 172.21.15.7:0/47092488 >> 172.21.15.7:0/47092488 conn(0x7fd9080fd7e0 msgr2=0x7fd9080ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 -- 172.21.15.7:0/47092488 shutdown_connections 2024-09-10T10:50:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 -- 172.21.15.7:0/47092488 wait complete. 2024-09-10T10:50:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 Processor -- start 2024-09-10T10:50:22.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.147+0000 7fd9104f8640 1 -- start start 2024-09-10T10:50:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd9104f8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd908101ae0 0x7fd908198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd9104f8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd908102480 0x7fd908198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd9104f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd908107b80 0x7fd90819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd9104f8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd908111f80 con 0x7fd908107b80 2024-09-10T10:50:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd9104f8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd908111e00 con 0x7fd908101ae0 2024-09-10T10:50:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd9104f8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd908112100 con 0x7fd908102480 2024-09-10T10:50:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90ea95640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd908107b80 0x7fd90819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90da93640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd908102480 0x7fd908198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90ea95640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd908107b80 0x7fd90819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33220/0 (socket says 172.21.15.7:33220) 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90ea95640 1 -- 172.21.15.7:0/4165518266 learned_addr learned my addr 172.21.15.7:0/4165518266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90e294640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd908101ae0 0x7fd908198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90ea95640 1 -- 172.21.15.7:0/4165518266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd908102480 msgr2=0x7fd908198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90ea95640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd908102480 0x7fd908198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90ea95640 1 -- 172.21.15.7:0/4165518266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd908101ae0 msgr2=0x7fd908198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90ea95640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd908101ae0 0x7fd908198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90ea95640 1 -- 172.21.15.7:0/4165518266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90819d340 con 0x7fd908107b80 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90e294640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd908101ae0 0x7fd908198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd90ea95640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd908107b80 0x7fd90819cc10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd90000eb10 tx=0x7fd90000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.148+0000 7fd8ff7fe640 1 -- 172.21.15.7:0/4165518266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900004b20 con 0x7fd908107b80 2024-09-10T10:50:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.149+0000 7fd8ff7fe640 1 -- 172.21.15.7:0/4165518266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd900004cc0 con 0x7fd908107b80 2024-09-10T10:50:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.149+0000 7fd9104f8640 1 -- 172.21.15.7:0/4165518266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90819d630 con 0x7fd908107b80 2024-09-10T10:50:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.149+0000 7fd8ff7fe640 1 -- 172.21.15.7:0/4165518266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900010760 con 0x7fd908107b80 2024-09-10T10:50:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.149+0000 7fd9104f8640 1 -- 172.21.15.7:0/4165518266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9081a4ec0 con 0x7fd908107b80 2024-09-10T10:50:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.152+0000 7fd8ff7fe640 1 -- 172.21.15.7:0/4165518266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd900005020 con 0x7fd908107b80 2024-09-10T10:50:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.152+0000 7fd913296740 1 Processor -- start 2024-09-10T10:50:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.152+0000 7fd913296740 1 -- start start 2024-09-10T10:50:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd913296740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab212bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd913296740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2136770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd913296740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab2136cb0 0x556ab21390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd913296740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556ab20902f0 con 0x556ab212baf0 2024-09-10T10:50:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd913296740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556ab2038270 con 0x556ab212c410 2024-09-10T10:50:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd913296740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ab208c2b0 con 0x556ab2136cb0 2024-09-10T10:50:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90da93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab212bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90e294640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2136770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90da93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab212bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33228/0 (socket says 172.21.15.7:33228) 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90e294640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2136770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:49744/0 (socket says 172.21.15.7:49744) 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90da93640 1 -- 172.21.15.7:0/1644549682 learned_addr learned my addr 172.21.15.7:0/1644549682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90da93640 1 --2- 172.21.15.7:0/1644549682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab2136cb0 0x556ab21390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd8ff7fe640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd8f40a2e20 0x7fd8f40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90e294640 1 -- 172.21.15.7:0/1644549682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab2136cb0 msgr2=0x556ab21390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90e294640 1 --2- 172.21.15.7:0/1644549682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab2136cb0 0x556ab21390a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90e294640 1 -- 172.21.15.7:0/1644549682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 msgr2=0x556ab212bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90e294640 1 --2- 172.21.15.7:0/1644549682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab212bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90e294640 1 -- 172.21.15.7:0/1644549682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab2037340 con 0x556ab212c410 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.153+0000 7fd90ea95640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd8f40a2e20 0x7fd8f40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.154+0000 7fd8ff7fe640 1 -- 172.21.15.7:0/4165518266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fd9000cfa20 con 0x7fd908107b80 2024-09-10T10:50:22.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.154+0000 7fd90e294640 1 --2- 172.21.15.7:0/1644549682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2136770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd90402a940 tx=0x7fd904000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32fb0dbe349b2c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.154+0000 7fd8fcbfe640 1 -- 172.21.15.7:0/1644549682 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904035040 con 0x556ab212c410 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.154+0000 7fd90ea95640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd8f40a2e20 0x7fd8f40a52e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd900098910 tx=0x7fd900067010 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.154+0000 7fd8fcbfe640 1 -- 172.21.15.7:0/1644549682 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd904004760 con 0x556ab212c410 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.154+0000 7fd8fcbfe640 1 -- 172.21.15.7:0/1644549682 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904003830 con 0x556ab212c410 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 -- 172.21.15.7:0/1644549682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 msgr2=0x556ab2136770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 --2- 172.21.15.7:0/1644549682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2136770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd90402a940 tx=0x7fd904000bc0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 -- 172.21.15.7:0/1644549682 shutdown_connections 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 --2- 172.21.15.7:0/1644549682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab2136cb0 0x556ab21390a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 --2- 172.21.15.7:0/1644549682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2136770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 --2- 172.21.15.7:0/1644549682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab212bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 -- 172.21.15.7:0/1644549682 >> 172.21.15.7:0/1644549682 conn(0x556ab2128060 msgr2=0x556ab2128440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 -- 172.21.15.7:0/1644549682 shutdown_connections 2024-09-10T10:50:22.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 -- 172.21.15.7:0/1644549682 wait complete. 2024-09-10T10:50:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 -- start start 2024-09-10T10:50:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 Processor -- start 2024-09-10T10:50:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.155+0000 7fd913296740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab2136670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd913296740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2133b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd913296740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab2136cb0 0x556ab21340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd913296740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556ab213dc90 con 0x556ab212baf0 2024-09-10T10:50:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd913296740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556ab2019290 con 0x556ab212c410 2024-09-10T10:50:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd913296740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ab213de10 con 0x556ab2136cb0 2024-09-10T10:50:22.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd913296740 10 client.0 _ll_register_callbacks cb 0x556ab204a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90da93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab2136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90e294640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90e294640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2133b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:49752/0 (socket says 172.21.15.7:49752) 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90e294640 1 -- 172.21.15.7:0/1669230913 learned_addr learned my addr 172.21.15.7:0/1669230913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90da93640 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab2136cb0 0x556ab21340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90e294640 1 -- 172.21.15.7:0/1669230913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab2136cb0 msgr2=0x556ab21340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90e294640 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab2136cb0 0x556ab21340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90e294640 1 -- 172.21.15.7:0/1669230913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 msgr2=0x556ab2136670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90e294640 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab2136670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90e294640 1 -- 172.21.15.7:0/1669230913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab1da3210 con 0x556ab212c410 2024-09-10T10:50:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd90e294640 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2133b90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd904009a50 tx=0x7fd904003800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c7cc5de35527d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd8f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/1669230913 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904052060 con 0x556ab212c410 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/1669230913 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fd904003cf0 con 0x556ab212c410 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.156+0000 7fd913296740 1 -- 172.21.15.7:0/1669230913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd904004760 con 0x556ab212c410 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/1669230913 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd904035070 con 0x556ab212c410 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd913296740 10 client.25280 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd913296740 1 -- 172.21.15.7:0/1669230913 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556ab1e6f620 con 0x556ab212c410 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/1669230913 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd904004760 con 0x556ab212c410 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/1669230913 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd904056760 con 0x556ab212c410 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd913296740 10 client.25280 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd913296740 10 client.25280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd913296740 20 client.25280 populate_metadata read hostname 'smithi007' 2024-09-10T10:50:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd913296740 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556ab214f2a0 0x556ab2151690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd913296740 4 client.25280 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd913296740 1 -- 172.21.15.7:0/1669230913 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ab209da20 con 0x556ab214f2a0 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.157+0000 7fd90da93640 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556ab214f2a0 0x556ab2151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.158+0000 7fd8f1ffb640 10 client.25280 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.158+0000 7fd90da93640 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556ab214f2a0 0x556ab2151690 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab9d32b30c96bdd server_cookie=fad2cb51defd8f1f in_seq=0 out_seq=0 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.158+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/1669230913 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ab209da20 con 0x556ab214f2a0 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.158+0000 7fd8f1ffb640 10 client.25280 handle_command_reply: tid=0 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.158+0000 7fd913296740 1 client.25280 shutdown 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 1 -- 172.21.15.7:0/1669230913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 msgr2=0x556ab2133b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2133b90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd904009a50 tx=0x7fd904003800 comp rx=0 tx=0).stop 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 1 -- 172.21.15.7:0/1669230913 shutdown_connections 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556ab214f2a0 0x556ab2151690 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab2136cb0 0x556ab21340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2133b90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd8f1ffb640 0 client.25280 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 1 --2- 172.21.15.7:0/1669230913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab2136670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 1 -- 172.21.15.7:0/1669230913 >> 172.21.15.7:0/1669230913 conn(0x556ab2128060 msgr2=0x556ab212ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 1 -- 172.21.15.7:0/1669230913 shutdown_connections 2024-09-10T10:50:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 1 -- 172.21.15.7:0/1669230913 wait complete. 2024-09-10T10:50:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.159+0000 7fd913296740 20 client.25280 trim_cache size 0 max 16384 2024-09-10T10:50:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.175+0000 7fd913296740 1 Processor -- start 2024-09-10T10:50:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.175+0000 7fd913296740 1 -- start start 2024-09-10T10:50:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd913296740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212baf0 0x556ab2135b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd913296740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2156c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd913296740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab2136cb0 0x556ab21571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd913296740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9081ade60 con 0x556ab2136cb0 2024-09-10T10:50:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd913296740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9081ab730 con 0x556ab212c410 2024-09-10T10:50:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd913296740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd9081a95a0 con 0x556ab212baf0 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90da93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab2136cb0 0x556ab21571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90da93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab2136cb0 0x556ab21571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33238/0 (socket says 172.21.15.7:33238) 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90da93640 1 -- 172.21.15.7:0/3092845033 learned_addr learned my addr 172.21.15.7:0/3092845033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90da93640 1 --2- 172.21.15.7:0/3092845033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212baf0 0x556ab2135b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90e294640 1 --2- 172.21.15.7:0/3092845033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90da93640 1 -- 172.21.15.7:0/3092845033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212baf0 msgr2=0x556ab2135b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90da93640 1 --2- 172.21.15.7:0/3092845033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212baf0 0x556ab2135b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90da93640 1 -- 172.21.15.7:0/3092845033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 msgr2=0x556ab2156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90da93640 1 --2- 172.21.15.7:0/3092845033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90da93640 1 -- 172.21.15.7:0/3092845033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab2151a00 con 0x556ab2136cb0 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd90da93640 1 --2- 172.21.15.7:0/3092845033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab2136cb0 0x556ab21571a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8005b90 tx=0x7fd8f8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae171640233d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd8f3fff640 1 -- 172.21.15.7:0/3092845033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f801c040 con 0x556ab2136cb0 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd8f3fff640 1 -- 172.21.15.7:0/3092845033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8f8003cd0 con 0x556ab2136cb0 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.176+0000 7fd8f3fff640 1 -- 172.21.15.7:0/3092845033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8010430 con 0x556ab2136cb0 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.177+0000 7fd913296740 1 -- 172.21.15.7:0/3092845033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab2136cb0 msgr2=0x556ab21571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.177+0000 7fd913296740 1 --2- 172.21.15.7:0/3092845033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab2136cb0 0x556ab21571a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8005b90 tx=0x7fd8f8005f90 comp rx=0 tx=0).stop 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.177+0000 7fd913296740 1 -- 172.21.15.7:0/3092845033 shutdown_connections 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.177+0000 7fd913296740 1 --2- 172.21.15.7:0/3092845033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab2136cb0 0x556ab21571a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.177+0000 7fd913296740 1 --2- 172.21.15.7:0/3092845033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab212c410 0x556ab2156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.177+0000 7fd913296740 1 --2- 172.21.15.7:0/3092845033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212baf0 0x556ab2135b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.177+0000 7fd913296740 1 -- 172.21.15.7:0/3092845033 >> 172.21.15.7:0/3092845033 conn(0x556ab2128060 msgr2=0x556ab21353f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.177+0000 7fd913296740 1 -- 172.21.15.7:0/3092845033 shutdown_connections 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.177+0000 7fd913296740 1 -- 172.21.15.7:0/3092845033 wait complete. 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd913296740 1 -- start start 2024-09-10T10:50:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd913296740 1 Processor -- start 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd913296740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab2147c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd913296740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212c410 0x556ab2148150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd913296740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab2136cb0 0x556ab21b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd913296740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9081a9bf0 con 0x556ab212baf0 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd913296740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9081ab060 con 0x556ab2136cb0 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd913296740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd9081aa2c0 con 0x556ab212c410 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd913296740 10 client.0 _ll_register_callbacks cb 0x7fd8f8009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd90da93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab2147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd90e294640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212c410 0x556ab2148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd90da93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab2147c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33254/0 (socket says 172.21.15.7:33254) 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd90da93640 1 -- 172.21.15.7:0/2990279093 learned_addr learned my addr 172.21.15.7:0/2990279093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd90da93640 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab2136cb0 0x556ab21b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd90e294640 1 -- 172.21.15.7:0/2990279093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab2136cb0 msgr2=0x556ab21b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd90e294640 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab2136cb0 0x556ab21b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd90e294640 1 -- 172.21.15.7:0/2990279093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 msgr2=0x556ab2147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd90e294640 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab2147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.178+0000 7fd90e294640 1 -- 172.21.15.7:0/2990279093 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab212c240 con 0x556ab212c410 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd90e294640 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212c410 0x556ab2148150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd904009a20 tx=0x7fd904056a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f6f99e49203c1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd8f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/2990279093 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904052070 con 0x556ab212c410 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd913296740 1 -- 172.21.15.7:0/2990279093 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8f8003cd0 con 0x556ab212c410 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/2990279093 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fd904005240 con 0x556ab212c410 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/2990279093 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd904003870 con 0x556ab212c410 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd913296740 10 client.25291 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd913296740 1 -- 172.21.15.7:0/2990279093 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556ab21ba170 con 0x556ab212c410 2024-09-10T10:50:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/2990279093 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8f8003cd0 con 0x556ab212c410 2024-09-10T10:50:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/2990279093 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fd904004cd0 con 0x556ab212c410 2024-09-10T10:50:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd913296740 10 client.25291 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd913296740 10 client.25291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd913296740 20 client.25291 populate_metadata read hostname 'smithi007' 2024-09-10T10:50:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd913296740 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556ab21ba890 0x556ab21bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd913296740 4 client.25291 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:50:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.179+0000 7fd913296740 1 -- 172.21.15.7:0/2990279093 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ab2097d70 con 0x556ab21ba890 2024-09-10T10:50:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.180+0000 7fd90da93640 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556ab21ba890 0x556ab21bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.180+0000 7fd8f1ffb640 10 client.25291 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.180+0000 7fd90da93640 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556ab21ba890 0x556ab21bccb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af8f57351928236 server_cookie=7dc7b73c8e80f7e6 in_seq=0 out_seq=0 2024-09-10T10:50:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.248+0000 7fd8f1ffb640 1 -- 172.21.15.7:0/2990279093 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ab2097d70 con 0x556ab21ba890 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.248+0000 7fd8f1ffb640 10 client.25291 handle_command_reply: tid=0 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.248+0000 7fd913296740 1 client.25291 shutdown 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 1 -- 172.21.15.7:0/2990279093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212c410 msgr2=0x556ab2148150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212c410 0x556ab2148150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd904009a20 tx=0x7fd904056a10 comp rx=0 tx=0).stop 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 1 -- 172.21.15.7:0/2990279093 shutdown_connections 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556ab21ba890 0x556ab21bccb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556ab2136cb0 0x556ab21b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ab212c410 0x556ab2148150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd8f1ffb640 0 client.25291 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 1 --2- 172.21.15.7:0/2990279093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ab212baf0 0x556ab2147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 1 -- 172.21.15.7:0/2990279093 >> 172.21.15.7:0/2990279093 conn(0x556ab2128060 msgr2=0x556ab21271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 1 -- 172.21.15.7:0/2990279093 shutdown_connections 2024-09-10T10:50:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 1 -- 172.21.15.7:0/2990279093 wait complete. 2024-09-10T10:50:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.249+0000 7fd913296740 20 client.25291 trim_cache size 0 max 16384 2024-09-10T10:50:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:50:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:50:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:50:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:50:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 -- 172.21.15.7:0/4165518266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd8f40a2e20 msgr2=0x7fd8f40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd8f40a2e20 0x7fd8f40a52e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd900098910 tx=0x7fd900067010 comp rx=0 tx=0).stop 2024-09-10T10:50:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 -- 172.21.15.7:0/4165518266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd908107b80 msgr2=0x7fd90819cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd908107b80 0x7fd90819cc10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd90000eb10 tx=0x7fd90000c2d0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 -- 172.21.15.7:0/4165518266 shutdown_connections 2024-09-10T10:50:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd8f40a2e20 0x7fd8f40a52e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd908107b80 0x7fd90819cc10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd908102480 0x7fd908198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 --2- 172.21.15.7:0/4165518266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd908101ae0 0x7fd908198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 -- 172.21.15.7:0/4165518266 >> 172.21.15.7:0/4165518266 conn(0x7fd9080fd7e0 msgr2=0x7fd9080fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 -- 172.21.15.7:0/4165518266 shutdown_connections 2024-09-10T10:50:22.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:22.250+0000 7fd9104f8640 1 -- 172.21.15.7:0/4165518266 wait complete. 2024-09-10T10:50:22.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 476312 12.54 MB/sec execute 82 sec latency 26.518 ms 2024-09-10T10:50:23.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 478723 12.49 MB/sec execute 83 sec latency 16.378 ms 2024-09-10T10:50:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:23 smithi103 ceph-mon[35160]: pgmap v139: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-09-10T10:50:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:23 smithi007 ceph-mon[30873]: pgmap v139: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-09-10T10:50:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:23 smithi133 ceph-mon[35282]: pgmap v139: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-09-10T10:50:24.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 480948 12.50 MB/sec execute 84 sec latency 10.671 ms 2024-09-10T10:50:25.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 483507 12.55 MB/sec execute 85 sec latency 134.510 ms 2024-09-10T10:50:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:25 smithi103 ceph-mon[35160]: pgmap v140: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-09-10T10:50:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:25 smithi007 ceph-mon[30873]: pgmap v140: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-09-10T10:50:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:25 smithi133 ceph-mon[35282]: pgmap v140: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-09-10T10:50:26.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 485697 12.50 MB/sec execute 86 sec latency 62.013 ms 2024-09-10T10:50:27.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 487739 12.50 MB/sec execute 87 sec latency 20.524 ms 2024-09-10T10:50:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:27 smithi103 ceph-mon[35160]: pgmap v141: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-09-10T10:50:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:27 smithi007 ceph-mon[30873]: pgmap v141: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-09-10T10:50:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:27 smithi133 ceph-mon[35282]: pgmap v141: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-09-10T10:50:28.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 490594 12.56 MB/sec execute 88 sec latency 13.308 ms 2024-09-10T10:50:29.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 493021 12.54 MB/sec execute 89 sec latency 16.702 ms 2024-09-10T10:50:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:29 smithi103 ceph-mon[35160]: pgmap v142: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 328 op/s 2024-09-10T10:50:29.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:29 smithi007 ceph-mon[30873]: pgmap v142: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 328 op/s 2024-09-10T10:50:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:29 smithi133 ceph-mon[35282]: pgmap v142: 97 pgs: 97 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 328 op/s 2024-09-10T10:50:30.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 494833 12.50 MB/sec execute 90 sec latency 12.692 ms 2024-09-10T10:50:31.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 497535 12.55 MB/sec execute 91 sec latency 59.740 ms 2024-09-10T10:50:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:31 smithi103 ceph-mon[35160]: pgmap v143: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-09-10T10:50:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:31 smithi007 ceph-mon[30873]: pgmap v143: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-09-10T10:50:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:31 smithi133 ceph-mon[35282]: pgmap v143: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-09-10T10:50:32.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 499295 12.49 MB/sec execute 92 sec latency 23.610 ms 2024-09-10T10:50:33.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 501451 12.50 MB/sec execute 93 sec latency 23.042 ms 2024-09-10T10:50:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:33 smithi103 ceph-mon[35160]: pgmap v144: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-09-10T10:50:33.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:33 smithi007 ceph-mon[30873]: pgmap v144: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-09-10T10:50:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:33 smithi133 ceph-mon[35282]: pgmap v144: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-09-10T10:50:34.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 504319 12.52 MB/sec execute 94 sec latency 15.908 ms 2024-09-10T10:50:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.346+0000 7f7001f49640 1 -- 172.21.15.7:0/831888145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc103f40 msgr2=0x7f6ffc104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.346+0000 7f7001f49640 1 --2- 172.21.15.7:0/831888145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc103f40 0x7f6ffc104320 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009900 tx=0x7f6fec02a410 comp rx=0 tx=0).stop 2024-09-10T10:50:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.347+0000 7f7001f49640 1 -- 172.21.15.7:0/831888145 shutdown_connections 2024-09-10T10:50:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.347+0000 7f7001f49640 1 --2- 172.21.15.7:0/831888145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ffc109090 0x7f6ffc110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.347+0000 7f7001f49640 1 --2- 172.21.15.7:0/831888145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc1048f0 0x7f6ffc108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.347+0000 7f7001f49640 1 --2- 172.21.15.7:0/831888145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc103f40 0x7f6ffc104320 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.347+0000 7f7001f49640 1 -- 172.21.15.7:0/831888145 >> 172.21.15.7:0/831888145 conn(0x7f6ffc0ff970 msgr2=0x7f6ffc101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.347+0000 7f7001f49640 1 -- 172.21.15.7:0/831888145 shutdown_connections 2024-09-10T10:50:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.347+0000 7f7001f49640 1 -- 172.21.15.7:0/831888145 wait complete. 2024-09-10T10:50:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.347+0000 7f7001f49640 1 Processor -- start 2024-09-10T10:50:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.347+0000 7f7001f49640 1 -- start start 2024-09-10T10:50:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.347+0000 7f7001f49640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc103f40 0x7f6ffc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f7001f49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ffc109090 0x7f6ffc19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f7001f49640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc19edf0 0x7f6ffc19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f7001f49640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1140e0 con 0x7f6ffc103f40 2024-09-10T10:50:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f7001f49640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc113f60 con 0x7f6ffc109090 2024-09-10T10:50:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f7001f49640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc114260 con 0x7f6ffc19edf0 2024-09-10T10:50:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc103f40 0x7f6ffc19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc103f40 0x7f6ffc19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48086/0 (socket says 172.21.15.7:48086) 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/2169139597 learned_addr learned my addr 172.21.15.7:0/2169139597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ffc109090 0x7f6ffc19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffbfff640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc19edf0 0x7f6ffc19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/2169139597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc19edf0 msgr2=0x7f6ffc19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffb7fe640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc19edf0 0x7f6ffc19f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/2169139597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ffc109090 msgr2=0x7f6ffc19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffb7fe640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ffc109090 0x7f6ffc19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/2169139597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc19f8e0 con 0x7f6ffc103f40 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffbfff640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc19edf0 0x7f6ffc19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ffc109090 0x7f6ffc19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ffb7fe640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc103f40 0x7f6ffc19a530 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6fec006010 tx=0x7f6fec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ff8ff9640 1 -- 172.21.15.7:0/2169139597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec03f030 con 0x7f6ffc103f40 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f7001f49640 1 -- 172.21.15.7:0/2169139597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc19fb70 con 0x7f6ffc103f40 2024-09-10T10:50:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ff8ff9640 1 -- 172.21.15.7:0/2169139597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fec004440 con 0x7f6ffc103f40 2024-09-10T10:50:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.348+0000 7f6ff8ff9640 1 -- 172.21.15.7:0/2169139597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec035070 con 0x7f6ffc103f40 2024-09-10T10:50:35.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.349+0000 7f7001f49640 1 -- 172.21.15.7:0/2169139597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc1a7330 con 0x7f6ffc103f40 2024-09-10T10:50:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f7004dbb740 1 Processor -- start 2024-09-10T10:50:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f7004dbb740 1 -- start start 2024-09-10T10:50:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ff8ff9640 1 -- 172.21.15.7:0/2169139597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6fec042030 con 0x7f6ffc103f40 2024-09-10T10:50:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 0x55efbcb28ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 0x55efbcb33770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f7004dbb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55efbca8d2f0 con 0x55efbcb33cb0 2024-09-10T10:50:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f7004dbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55efbca35270 con 0x55efbcb28af0 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f7004dbb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55efbca892b0 con 0x55efbcb29410 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 0x55efbcb33770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb360a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48090/0 (socket says 172.21.15.7:48090) 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffaffd640 1 -- 172.21.15.7:0/2766580834 learned_addr learned my addr 172.21.15.7:0/2766580834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffbfff640 1 --2- 172.21.15.7:0/2766580834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 0x55efbcb28ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffaffd640 1 -- 172.21.15.7:0/2766580834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 msgr2=0x55efbcb33770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/2766580834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 0x55efbcb33770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffaffd640 1 -- 172.21.15.7:0/2766580834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 msgr2=0x55efbcb28ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/2766580834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 0x55efbcb28ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffaffd640 1 -- 172.21.15.7:0/2766580834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efbca34340 con 0x55efbcb33cb0 2024-09-10T10:50:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.352+0000 7f6ffbfff640 1 --2- 172.21.15.7:0/2766580834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 0x55efbcb28ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.353+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/2766580834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb360a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00114f0 tx=0x7f6ff00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cdcfe80ae22707 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.353+0000 7f6ff8ff9640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6fe40a2da0 0x7f6fe40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.353+0000 7f6fe1ffb640 1 -- 172.21.15.7:0/2766580834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0005020 con 0x55efbcb33cb0 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.353+0000 7f6ffbfff640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6fe40a2da0 0x7f6fe40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.353+0000 7f6ff8ff9640 1 -- 172.21.15.7:0/2169139597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f6fec0ef790 con 0x7f6ffc103f40 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.353+0000 7f6fe1ffb640 1 -- 172.21.15.7:0/2766580834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff00047a0 con 0x55efbcb33cb0 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.353+0000 7f6fe1ffb640 1 -- 172.21.15.7:0/2766580834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000f9a0 con 0x55efbcb33cb0 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 -- 172.21.15.7:0/2766580834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 msgr2=0x55efbcb360a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 --2- 172.21.15.7:0/2766580834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb360a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00114f0 tx=0x7f6ff00119b0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f6ffbfff640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6fe40a2da0 0x7f6fe40a5260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4005fd0 tx=0x7f6ff4009500 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 -- 172.21.15.7:0/2766580834 shutdown_connections 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 --2- 172.21.15.7:0/2766580834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb360a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 --2- 172.21.15.7:0/2766580834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 0x55efbcb33770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 --2- 172.21.15.7:0/2766580834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 0x55efbcb28ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 -- 172.21.15.7:0/2766580834 >> 172.21.15.7:0/2766580834 conn(0x55efbcb25060 msgr2=0x55efbcb25440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 -- 172.21.15.7:0/2766580834 shutdown_connections 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 -- 172.21.15.7:0/2766580834 wait complete. 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 -- start start 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.354+0000 7f7004dbb740 1 Processor -- start 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 0x55efbcb30b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 0x55efbcb310a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f7004dbb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55efbcb36880 con 0x55efbcb33cb0 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f7004dbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55efbca16290 con 0x55efbcb28af0 2024-09-10T10:50:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f7004dbb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55efbcb36a00 con 0x55efbcb29410 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f7004dbb740 10 client.0 _ll_register_callbacks cb 0x55efbca47140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 0x55efbcb310a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 0x55efbcb30b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 0x55efbcb310a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:54254/0 (socket says 172.21.15.7:54254) 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffaffd640 1 -- 172.21.15.7:0/4122184636 learned_addr learned my addr 172.21.15.7:0/4122184636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffaffd640 1 -- 172.21.15.7:0/4122184636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 msgr2=0x55efbcb30b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 0x55efbcb30b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffaffd640 1 -- 172.21.15.7:0/4122184636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 msgr2=0x55efbcb315e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb315e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffaffd640 1 -- 172.21.15.7:0/4122184636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efbc7a0210 con 0x55efbcb29410 2024-09-10T10:50:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffb7fe640 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 0x55efbcb30b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffbfff640 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb315e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 0x55efbcb310a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ff001b040 tx=0x7f6ff00051f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9f7f9d8e68cba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6fdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4122184636 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0004030 con 0x55efbcb29410 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f7004dbb740 1 -- 172.21.15.7:0/4122184636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff00047a0 con 0x55efbcb29410 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.355+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4122184636 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f6ff00041d0 con 0x55efbcb29410 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4122184636 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ff0033050 con 0x55efbcb29410 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f7004dbb740 10 client.25303 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f7004dbb740 1 -- 172.21.15.7:0/4122184636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55efbc86c620 con 0x55efbcb29410 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4122184636 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff00047a0 con 0x55efbcb29410 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4122184636 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6ff00035f0 con 0x55efbcb29410 2024-09-10T10:50:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f7004dbb740 10 client.25303 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f7004dbb740 10 client.25303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f7004dbb740 20 client.25303 populate_metadata read hostname 'smithi007' 2024-09-10T10:50:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55efbcb43ae0 0x55efbcb45ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f7004dbb740 4 client.25303 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f7004dbb740 1 -- 172.21.15.7:0/4122184636 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efbca9aa20 con 0x55efbcb43ae0 2024-09-10T10:50:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.356+0000 7f6ffb7fe640 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55efbcb43ae0 0x55efbcb45ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.357+0000 7f6ffb7fe640 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55efbcb43ae0 0x55efbcb45ed0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad6ad5fd4c6552da server_cookie=bad7c59fec82cd18 in_seq=0 out_seq=0 2024-09-10T10:50:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.357+0000 7f6fdaffd640 10 client.25303 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.357+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4122184636 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efbca9aa20 con 0x55efbcb43ae0 2024-09-10T10:50:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.357+0000 7f6fdaffd640 10 client.25303 handle_command_reply: tid=0 2024-09-10T10:50:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.357+0000 7f7004dbb740 1 client.25303 shutdown 2024-09-10T10:50:35.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 1 -- 172.21.15.7:0/4122184636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 msgr2=0x55efbcb310a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 0x55efbcb310a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ff001b040 tx=0x7f6ff00051f0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 1 -- 172.21.15.7:0/4122184636 shutdown_connections 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55efbcb43ae0 0x55efbcb45ed0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb33cb0 0x55efbcb315e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f6fdaffd640 0 client.25303 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb29410 0x55efbcb310a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4122184636 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb28af0 0x55efbcb30b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 1 -- 172.21.15.7:0/4122184636 >> 172.21.15.7:0/4122184636 conn(0x55efbcb25060 msgr2=0x55efbcb2ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 1 -- 172.21.15.7:0/4122184636 shutdown_connections 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 1 -- 172.21.15.7:0/4122184636 wait complete. 2024-09-10T10:50:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.358+0000 7f7004dbb740 20 client.25303 trim_cache size 0 max 16384 2024-09-10T10:50:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f7004dbb740 1 Processor -- start 2024-09-10T10:50:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f7004dbb740 1 -- start start 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb28af0 0x55efbcb2b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb29410 0x55efbcb31410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb33cb0 0x55efbcb32ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f7004dbb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1b21e0 con 0x55efbcb29410 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f7004dbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1afab0 con 0x55efbcb33cb0 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f7004dbb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1ad920 con 0x55efbcb28af0 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f6ffaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb29410 0x55efbcb31410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb33cb0 0x55efbcb32ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f6ffaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb29410 0x55efbcb31410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48106/0 (socket says 172.21.15.7:48106) 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb28af0 0x55efbcb2b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f6ffaffd640 1 -- 172.21.15.7:0/2168474185 learned_addr learned my addr 172.21.15.7:0/2168474185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f6ffaffd640 1 -- 172.21.15.7:0/2168474185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb28af0 msgr2=0x55efbcb2b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:35.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/2168474185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb28af0 0x55efbcb2b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f6ffaffd640 1 -- 172.21.15.7:0/2168474185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb33cb0 msgr2=0x55efbcb32ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:35.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/2168474185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb33cb0 0x55efbcb32ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.374+0000 7f6ffaffd640 1 -- 172.21.15.7:0/2168474185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efbcb336b0 con 0x55efbcb29410 2024-09-10T10:50:35.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.375+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/2168474185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb29410 0x55efbcb31410 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0003a40 tx=0x7f6ff0005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff7de17d44f4d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.375+0000 7f6fe17fa640 1 -- 172.21.15.7:0/2168474185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0009b30 con 0x55efbcb29410 2024-09-10T10:50:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.375+0000 7f6fe17fa640 1 -- 172.21.15.7:0/2168474185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff0016020 con 0x55efbcb29410 2024-09-10T10:50:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.375+0000 7f6fe17fa640 1 -- 172.21.15.7:0/2168474185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0004760 con 0x55efbcb29410 2024-09-10T10:50:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.375+0000 7f7004dbb740 1 -- 172.21.15.7:0/2168474185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb29410 msgr2=0x55efbcb31410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.375+0000 7f7004dbb740 1 --2- 172.21.15.7:0/2168474185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb29410 0x55efbcb31410 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0003a40 tx=0x7f6ff0005d30 comp rx=0 tx=0).stop 2024-09-10T10:50:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 -- 172.21.15.7:0/2168474185 shutdown_connections 2024-09-10T10:50:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 --2- 172.21.15.7:0/2168474185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb33cb0 0x55efbcb32ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 --2- 172.21.15.7:0/2168474185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb29410 0x55efbcb31410 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 --2- 172.21.15.7:0/2168474185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb28af0 0x55efbcb2b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 -- 172.21.15.7:0/2168474185 >> 172.21.15.7:0/2168474185 conn(0x55efbcb25060 msgr2=0x55efbcb323c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 -- 172.21.15.7:0/2168474185 shutdown_connections 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 -- 172.21.15.7:0/2168474185 wait complete. 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 -- start start 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 Processor -- start 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb28af0 0x55efbcb2e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb29410 0x55efbcb3bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb33cb0 0x55efbcb3c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1adf70 con 0x55efbcb28af0 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1af3e0 con 0x55efbcb29410 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.376+0000 7f7004dbb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1ae640 con 0x55efbcb33cb0 2024-09-10T10:50:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f7004dbb740 10 client.0 _ll_register_callbacks cb 0x7f6fec0f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb28af0 0x55efbcb2e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb28af0 0x55efbcb2e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48116/0 (socket says 172.21.15.7:48116) 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffaffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb29410 0x55efbcb3bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/4116465331 learned_addr learned my addr 172.21.15.7:0/4116465331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffbfff640 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb33cb0 0x55efbcb3c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/4116465331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb33cb0 msgr2=0x55efbcb3c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffb7fe640 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb33cb0 0x55efbcb3c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/4116465331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb29410 msgr2=0x55efbcb3bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffb7fe640 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb29410 0x55efbcb3bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/4116465331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efbcb2ce20 con 0x55efbcb28af0 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb29410 0x55efbcb3bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6ffb7fe640 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb28af0 0x55efbcb2e7d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0ed4e0 tx=0x7f6fec0b80c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a277e9ae53c083 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6fdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4116465331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec049040 con 0x55efbcb28af0 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4116465331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f6fec046110 con 0x55efbcb28af0 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f7004dbb740 1 -- 172.21.15.7:0/4116465331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff0016020 con 0x55efbcb28af0 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4116465331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6fec0bb6c0 con 0x55efbcb28af0 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f7004dbb740 10 client.15621 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.377+0000 7f7004dbb740 1 -- 172.21.15.7:0/4116465331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55efbcbad9c0 con 0x55efbcb28af0 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4116465331 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff0016020 con 0x55efbcb28af0 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4116465331 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f6fec0ed160 con 0x55efbcb28af0 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f7004dbb740 10 client.15621 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f7004dbb740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f7004dbb740 20 client.15621 populate_metadata read hostname 'smithi007' 2024-09-10T10:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55efbcbae130 0x55efbcbb0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f7004dbb740 4 client.15621 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f7004dbb740 1 -- 172.21.15.7:0/4116465331 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55efbca94d70 con 0x55efbcbae130 2024-09-10T10:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55efbcbae130 0x55efbcbb0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f6fdaffd640 10 client.15621 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.378+0000 7f6ffaffd640 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55efbcbae130 0x55efbcbb0520 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b79de318dee4f1d server_cookie=6f1aa9084235ae85 in_seq=0 out_seq=0 2024-09-10T10:50:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.447+0000 7f6fdaffd640 1 -- 172.21.15.7:0/4116465331 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55efbca94d70 con 0x55efbcbae130 2024-09-10T10:50:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.447+0000 7f6fdaffd640 10 client.15621 handle_command_reply: tid=0 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.447+0000 7f7004dbb740 1 client.15621 shutdown 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 1 -- 172.21.15.7:0/4116465331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb28af0 msgr2=0x55efbcb2e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb28af0 0x55efbcb2e7d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0ed4e0 tx=0x7f6fec0b80c0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 1 -- 172.21.15.7:0/4116465331 shutdown_connections 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55efbcbae130 0x55efbcbb0520 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f6fdaffd640 0 client.15621 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efbcb33cb0 0x55efbcb3c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55efbcb29410 0x55efbcb3bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 1 --2- 172.21.15.7:0/4116465331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55efbcb28af0 0x55efbcb2e7d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 1 -- 172.21.15.7:0/4116465331 >> 172.21.15.7:0/4116465331 conn(0x55efbcb25060 msgr2=0x55efbcb241c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 1 -- 172.21.15.7:0/4116465331 shutdown_connections 2024-09-10T10:50:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 1 -- 172.21.15.7:0/4116465331 wait complete. 2024-09-10T10:50:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.448+0000 7f7004dbb740 20 client.15621 trim_cache size 0 max 16384 2024-09-10T10:50:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:50:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:50:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:50:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:50:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.449+0000 7f7001f49640 1 -- 172.21.15.7:0/2169139597 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6fe40a2da0 msgr2=0x7f6fe40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.449+0000 7f7001f49640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6fe40a2da0 0x7f6fe40a5260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4005fd0 tx=0x7f6ff4009500 comp rx=0 tx=0).stop 2024-09-10T10:50:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.449+0000 7f7001f49640 1 -- 172.21.15.7:0/2169139597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc103f40 msgr2=0x7f6ffc19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.449+0000 7f7001f49640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc103f40 0x7f6ffc19a530 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6fec006010 tx=0x7f6fec004040 comp rx=0 tx=0).stop 2024-09-10T10:50:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.449+0000 7f7001f49640 1 -- 172.21.15.7:0/2169139597 shutdown_connections 2024-09-10T10:50:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.449+0000 7f7001f49640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc19edf0 0x7f6ffc19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.449+0000 7f7001f49640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ffc109090 0x7f6ffc19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.449+0000 7f7001f49640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6fe40a2da0 0x7f6fe40a5260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.449+0000 7f7001f49640 1 --2- 172.21.15.7:0/2169139597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc103f40 0x7f6ffc19a530 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.449+0000 7f7001f49640 1 -- 172.21.15.7:0/2169139597 >> 172.21.15.7:0/2169139597 conn(0x7f6ffc0ff970 msgr2=0x7f6ffc100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.450+0000 7f7001f49640 1 -- 172.21.15.7:0/2169139597 shutdown_connections 2024-09-10T10:50:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:35.450+0000 7f7001f49640 1 -- 172.21.15.7:0/2169139597 wait complete. 2024-09-10T10:50:35.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 506050 12.49 MB/sec execute 95 sec latency 61.258 ms 2024-09-10T10:50:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:35 smithi103 ceph-mon[35160]: pgmap v145: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-09-10T10:50:35.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:35 smithi007 ceph-mon[30873]: pgmap v145: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-09-10T10:50:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:35 smithi133 ceph-mon[35282]: pgmap v145: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 322 op/s 2024-09-10T10:50:36.511 INFO:tasks.workunit.client.0.smithi007.stdout: 1 508388 12.51 MB/sec execute 96 sec latency 44.988 ms 2024-09-10T10:50:37.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 510800 12.49 MB/sec execute 97 sec latency 20.495 ms 2024-09-10T10:50:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:37 smithi103 ceph-mon[35160]: pgmap v146: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 450 op/s 2024-09-10T10:50:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:37 smithi007 ceph-mon[30873]: pgmap v146: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 450 op/s 2024-09-10T10:50:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:37 smithi133 ceph-mon[35282]: pgmap v146: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 450 op/s 2024-09-10T10:50:38.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 512688 12.47 MB/sec execute 98 sec latency 11.314 ms 2024-09-10T10:50:39.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 515488 12.52 MB/sec execute 99 sec latency 13.987 ms 2024-09-10T10:50:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:39 smithi103 ceph-mon[35160]: pgmap v147: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 258 op/s 2024-09-10T10:50:39.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:39 smithi007 ceph-mon[30873]: pgmap v147: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 258 op/s 2024-09-10T10:50:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:39 smithi133 ceph-mon[35282]: pgmap v147: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 258 op/s 2024-09-10T10:50:40.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 518296 12.52 MB/sec execute 100 sec latency 13.274 ms 2024-09-10T10:50:41.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 520121 12.51 MB/sec execute 101 sec latency 64.181 ms 2024-09-10T10:50:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:41 smithi103 ceph-mon[35160]: pgmap v148: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 436 op/s 2024-09-10T10:50:41.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:41 smithi007 ceph-mon[30873]: pgmap v148: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 436 op/s 2024-09-10T10:50:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:41 smithi133 ceph-mon[35282]: pgmap v148: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 436 op/s 2024-09-10T10:50:42.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 522606 12.52 MB/sec execute 102 sec latency 41.880 ms 2024-09-10T10:50:43.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 525416 12.53 MB/sec execute 103 sec latency 13.401 ms 2024-09-10T10:50:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:43 smithi103 ceph-mon[35160]: pgmap v149: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-10T10:50:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:43 smithi007 ceph-mon[30873]: pgmap v149: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-10T10:50:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:43 smithi133 ceph-mon[35282]: pgmap v149: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-10T10:50:44.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 527638 12.52 MB/sec execute 104 sec latency 10.801 ms 2024-09-10T10:50:45.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 529926 12.53 MB/sec execute 105 sec latency 14.743 ms 2024-09-10T10:50:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:45 smithi103 ceph-mon[35160]: pgmap v150: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-10T10:50:45.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:45 smithi007 ceph-mon[30873]: pgmap v150: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-10T10:50:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:45 smithi133 ceph-mon[35282]: pgmap v150: 97 pgs: 97 active+clean; 123 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 305 op/s 2024-09-10T10:50:46.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 532882 12.58 MB/sec execute 106 sec latency 36.828 ms 2024-09-10T10:50:47.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 535087 12.53 MB/sec execute 107 sec latency 15.982 ms 2024-09-10T10:50:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:47 smithi103 ceph-mon[35160]: pgmap v151: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-09-10T10:50:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:47 smithi007 ceph-mon[30873]: pgmap v151: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-09-10T10:50:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:47 smithi133 ceph-mon[35282]: pgmap v151: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-09-10T10:50:48.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 537211 12.54 MB/sec execute 108 sec latency 11.282 ms 2024-09-10T10:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.551+0000 7f45bf6af640 1 -- 172.21.15.7:0/2687709969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8107b70 msgr2=0x7f45b8107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.551+0000 7f45bf6af640 1 --2- 172.21.15.7:0/2687709969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8107b70 0x7f45b8107f50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009cd0 tx=0x7f45b402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.551+0000 7f45bf6af640 1 -- 172.21.15.7:0/2687709969 shutdown_connections 2024-09-10T10:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.551+0000 7f45bf6af640 1 --2- 172.21.15.7:0/2687709969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b8102470 0x7f45b810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.551+0000 7f45bf6af640 1 --2- 172.21.15.7:0/2687709969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b8101ad0 0x7f45b8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.551+0000 7f45bf6af640 1 --2- 172.21.15.7:0/2687709969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8107b70 0x7f45b8107f50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.551+0000 7f45bf6af640 1 -- 172.21.15.7:0/2687709969 >> 172.21.15.7:0/2687709969 conn(0x7f45b80fd7d0 msgr2=0x7f45b80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.551+0000 7f45bf6af640 1 -- 172.21.15.7:0/2687709969 shutdown_connections 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.551+0000 7f45bf6af640 1 -- 172.21.15.7:0/2687709969 wait complete. 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bf6af640 1 Processor -- start 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bf6af640 1 -- start start 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bf6af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8101ad0 0x7f45b8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bf6af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b8102470 0x7f45b8198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bf6af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b8107b70 0x7f45b819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bf6af640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f45b8111ee0 con 0x7f45b8101ad0 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bf6af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45b8111d60 con 0x7f45b8107b70 2024-09-10T10:50:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bf6af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45b8112060 con 0x7f45b8102470 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bd44b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8101ad0 0x7f45b8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bd44b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8101ad0 0x7f45b8198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49738/0 (socket says 172.21.15.7:49738) 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bcc4a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b8102470 0x7f45b8198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bd44b640 1 -- 172.21.15.7:0/1153187013 learned_addr learned my addr 172.21.15.7:0/1153187013 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bdc4c640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b8107b70 0x7f45b819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bd44b640 1 -- 172.21.15.7:0/1153187013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b8102470 msgr2=0x7f45b8198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.552+0000 7f45bd44b640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b8102470 0x7f45b8198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.553+0000 7f45bd44b640 1 -- 172.21.15.7:0/1153187013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b8107b70 msgr2=0x7f45b819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.553+0000 7f45bd44b640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b8107b70 0x7f45b819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.553+0000 7f45bd44b640 1 -- 172.21.15.7:0/1153187013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b819d300 con 0x7f45b8101ad0 2024-09-10T10:50:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.553+0000 7f45bd44b640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8101ad0 0x7f45b8198310 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005e30 tx=0x7f45b4032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.553+0000 7f45ae7fc640 1 -- 172.21.15.7:0/1153187013 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4004040 con 0x7f45b8101ad0 2024-09-10T10:50:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.553+0000 7f45ae7fc640 1 -- 172.21.15.7:0/1153187013 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b40046c0 con 0x7f45b8101ad0 2024-09-10T10:50:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.553+0000 7f45bf6af640 1 -- 172.21.15.7:0/1153187013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b819d590 con 0x7f45b8101ad0 2024-09-10T10:50:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.553+0000 7f45ae7fc640 1 -- 172.21.15.7:0/1153187013 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4046480 con 0x7f45b8101ad0 2024-09-10T10:50:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.553+0000 7f45bf6af640 1 -- 172.21.15.7:0/1153187013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b81a4e80 con 0x7f45b8101ad0 2024-09-10T10:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.555+0000 7f45ae7fc640 1 -- 172.21.15.7:0/1153187013 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f45b40305a0 con 0x7f45b8101ad0 2024-09-10T10:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45c243e740 1 Processor -- start 2024-09-10T10:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45c243e740 1 -- start start 2024-09-10T10:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45c243e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81baf0 0x55ad8d81bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45c243e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81c410 0x55ad8d826770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45c243e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d826cb0 0x55ad8d8290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45c243e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ad8d7802f0 con 0x55ad8d81c410 2024-09-10T10:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45c243e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ad8d728270 con 0x55ad8d81baf0 2024-09-10T10:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45c243e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad8d77c2b0 con 0x55ad8d826cb0 2024-09-10T10:50:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45bd44b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81c410 0x55ad8d826770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45bdc4c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81baf0 0x55ad8d81bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45bd44b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81c410 0x55ad8d826770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49740/0 (socket says 172.21.15.7:49740) 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45bd44b640 1 -- 172.21.15.7:0/3890241534 learned_addr learned my addr 172.21.15.7:0/3890241534 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.556+0000 7f45bcc4a640 1 --2- 172.21.15.7:0/3890241534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d826cb0 0x55ad8d8290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45ae7fc640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f45a40a2da0 0x7f45a40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45bd44b640 1 -- 172.21.15.7:0/3890241534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d826cb0 msgr2=0x55ad8d8290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45bd44b640 1 --2- 172.21.15.7:0/3890241534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d826cb0 0x55ad8d8290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45bd44b640 1 -- 172.21.15.7:0/3890241534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81baf0 msgr2=0x55ad8d81bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45bd44b640 1 --2- 172.21.15.7:0/3890241534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81baf0 0x55ad8d81bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45bd44b640 1 -- 172.21.15.7:0/3890241534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad8d727340 con 0x55ad8d81c410 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45bdc4c640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f45a40a2da0 0x7f45a40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45ae7fc640 1 -- 172.21.15.7:0/1153187013 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f45b402e030 con 0x7f45b8101ad0 2024-09-10T10:50:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45bd44b640 1 --2- 172.21.15.7:0/3890241534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81c410 0x55ad8d826770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f45b40edf70 tx=0x7f45b40e7990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c641bee673b47dfb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45a37fe640 1 -- 172.21.15.7:0/3890241534 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40bc940 con 0x55ad8d81c410 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45a37fe640 1 -- 172.21.15.7:0/3890241534 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b4036020 con 0x55ad8d81c410 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.557+0000 7f45a37fe640 1 -- 172.21.15.7:0/3890241534 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40bc760 con 0x55ad8d81c410 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.558+0000 7f45c243e740 1 -- 172.21.15.7:0/3890241534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81c410 msgr2=0x55ad8d826770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.558+0000 7f45c243e740 1 --2- 172.21.15.7:0/3890241534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81c410 0x55ad8d826770 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f45b40edf70 tx=0x7f45b40e7990 comp rx=0 tx=0).stop 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.558+0000 7f45bdc4c640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f45a40a2da0 0x7f45a40a5260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f45b000b6e0 tx=0x7f45b00046c0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.558+0000 7f45c243e740 1 -- 172.21.15.7:0/3890241534 shutdown_connections 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.558+0000 7f45c243e740 1 --2- 172.21.15.7:0/3890241534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d826cb0 0x55ad8d8290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.558+0000 7f45c243e740 1 --2- 172.21.15.7:0/3890241534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81c410 0x55ad8d826770 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.558+0000 7f45c243e740 1 --2- 172.21.15.7:0/3890241534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81baf0 0x55ad8d81bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.558+0000 7f45c243e740 1 -- 172.21.15.7:0/3890241534 >> 172.21.15.7:0/3890241534 conn(0x55ad8d818060 msgr2=0x55ad8d818440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.558+0000 7f45c243e740 1 -- 172.21.15.7:0/3890241534 shutdown_connections 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.558+0000 7f45c243e740 1 -- 172.21.15.7:0/3890241534 wait complete. 2024-09-10T10:50:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45c243e740 1 -- start start 2024-09-10T10:50:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45c243e740 1 Processor -- start 2024-09-10T10:50:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45c243e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d823bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45c243e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d81c410 0x55ad8d824130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45c243e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d826cb0 0x55ad8d824670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45c243e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ad8d829640 con 0x55ad8d81baf0 2024-09-10T10:50:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45c243e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ad8d709290 con 0x55ad8d826cb0 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45c243e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad8d8297c0 con 0x55ad8d81c410 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45c243e740 10 client.0 _ll_register_callbacks cb 0x55ad8d73a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45bd44b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d823bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45bd44b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d823bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49752/0 (socket says 172.21.15.7:49752) 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45bd44b640 1 -- 172.21.15.7:0/3109028781 learned_addr learned my addr 172.21.15.7:0/3109028781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45bcc4a640 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d81c410 0x55ad8d824130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.559+0000 7f45bdc4c640 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d826cb0 0x55ad8d824670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45bd44b640 1 -- 172.21.15.7:0/3109028781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d81c410 msgr2=0x55ad8d824130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45bd44b640 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d81c410 0x55ad8d824130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45bd44b640 1 -- 172.21.15.7:0/3109028781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d826cb0 msgr2=0x55ad8d824670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45bd44b640 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d826cb0 0x55ad8d824670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45bd44b640 1 -- 172.21.15.7:0/3109028781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad8d493210 con 0x55ad8d81baf0 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45bd44b640 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d823bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009620 tx=0x7f45b4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31322288602a8ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45a0ff9640 1 -- 172.21.15.7:0/3109028781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40bc760 con 0x55ad8d81baf0 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45a0ff9640 1 -- 172.21.15.7:0/3109028781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f45b40bb100 con 0x55ad8d81baf0 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45c243e740 1 -- 172.21.15.7:0/3109028781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b4036020 con 0x55ad8d81baf0 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45a0ff9640 1 -- 172.21.15.7:0/3109028781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f45b40bc940 con 0x55ad8d81baf0 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45c243e740 10 client.15633 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.560+0000 7f45c243e740 1 -- 172.21.15.7:0/3109028781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ad8d55f620 con 0x55ad8d81baf0 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.561+0000 7f45a0ff9640 1 -- 172.21.15.7:0/3109028781 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b4036020 con 0x55ad8d81baf0 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.561+0000 7f45a0ff9640 1 -- 172.21.15.7:0/3109028781 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f45b404e070 con 0x55ad8d81baf0 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.561+0000 7f45c243e740 10 client.15633 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.561+0000 7f45c243e740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.561+0000 7f45c243e740 20 client.15633 populate_metadata read hostname 'smithi007' 2024-09-10T10:50:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.561+0000 7f45c243e740 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad8d836800 0x55ad8d838c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.561+0000 7f45c243e740 4 client.15633 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.561+0000 7f45c243e740 1 -- 172.21.15.7:0/3109028781 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad8d78da20 con 0x55ad8d836800 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.561+0000 7f45bcc4a640 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad8d836800 0x55ad8d838c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.561+0000 7f45bcc4a640 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad8d836800 0x55ad8d838c40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec9b9061ffe309d server_cookie=f1a144cd5ed6d5e4 in_seq=0 out_seq=0 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.562+0000 7f45a0ff9640 10 client.15633 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.562+0000 7f45a0ff9640 1 -- 172.21.15.7:0/3109028781 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad8d78da20 con 0x55ad8d836800 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.562+0000 7f45a0ff9640 10 client.15633 handle_command_reply: tid=0 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.562+0000 7f45c243e740 1 client.15633 shutdown 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 1 -- 172.21.15.7:0/3109028781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 msgr2=0x55ad8d823bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d823bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009620 tx=0x7f45b4005ef0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 1 -- 172.21.15.7:0/3109028781 shutdown_connections 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad8d836800 0x55ad8d838c40 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d826cb0 0x55ad8d824670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d81c410 0x55ad8d824130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 1 --2- 172.21.15.7:0/3109028781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d823bf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 1 -- 172.21.15.7:0/3109028781 >> 172.21.15.7:0/3109028781 conn(0x55ad8d818060 msgr2=0x55ad8d81dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45a0ff9640 0 client.15633 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 1 -- 172.21.15.7:0/3109028781 shutdown_connections 2024-09-10T10:50:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 1 -- 172.21.15.7:0/3109028781 wait complete. 2024-09-10T10:50:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.563+0000 7f45c243e740 20 client.15633 trim_cache size 0 max 16384 2024-09-10T10:50:48.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.579+0000 7f45c243e740 1 Processor -- start 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.579+0000 7f45c243e740 1 -- start start 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.579+0000 7f45c243e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d8244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.579+0000 7f45c243e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d81c410 0x55ad8d825b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.579+0000 7f45c243e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d826cb0 0x55ad8d83e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.579+0000 7f45c243e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81aff40 con 0x55ad8d81baf0 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.579+0000 7f45c243e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81ad810 con 0x55ad8d826cb0 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.579+0000 7f45c243e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81ab660 con 0x55ad8d81c410 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bd44b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d8244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bdc4c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d826cb0 0x55ad8d83e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bd44b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d8244a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49768/0 (socket says 172.21.15.7:49768) 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bd44b640 1 -- 172.21.15.7:0/2678995449 learned_addr learned my addr 172.21.15.7:0/2678995449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bcc4a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d81c410 0x55ad8d825b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bd44b640 1 -- 172.21.15.7:0/2678995449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d81c410 msgr2=0x55ad8d825b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bd44b640 1 --2- 172.21.15.7:0/2678995449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d81c410 0x55ad8d825b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bd44b640 1 -- 172.21.15.7:0/2678995449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d826cb0 msgr2=0x55ad8d83e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bd44b640 1 --2- 172.21.15.7:0/2678995449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d826cb0 0x55ad8d83e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bd44b640 1 -- 172.21.15.7:0/2678995449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad8d822b80 con 0x55ad8d81baf0 2024-09-10T10:50:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45bd44b640 1 --2- 172.21.15.7:0/2678995449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d8244a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009430 tx=0x7f45b40c5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4492ac67e0a0195a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45a2ffd640 1 -- 172.21.15.7:0/2678995449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40bc760 con 0x55ad8d81baf0 2024-09-10T10:50:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45a2ffd640 1 -- 172.21.15.7:0/2678995449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b4036030 con 0x55ad8d81baf0 2024-09-10T10:50:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.580+0000 7f45a2ffd640 1 -- 172.21.15.7:0/2678995449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40bce00 con 0x55ad8d81baf0 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.581+0000 7f45c243e740 1 -- 172.21.15.7:0/2678995449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 msgr2=0x55ad8d8244a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.581+0000 7f45c243e740 1 --2- 172.21.15.7:0/2678995449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d8244a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009430 tx=0x7f45b40c5b60 comp rx=0 tx=0).stop 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.581+0000 7f45c243e740 1 -- 172.21.15.7:0/2678995449 shutdown_connections 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.581+0000 7f45c243e740 1 --2- 172.21.15.7:0/2678995449 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d826cb0 0x55ad8d83e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.581+0000 7f45c243e740 1 --2- 172.21.15.7:0/2678995449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d81c410 0x55ad8d825b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.581+0000 7f45c243e740 1 --2- 172.21.15.7:0/2678995449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d8244a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.581+0000 7f45c243e740 1 -- 172.21.15.7:0/2678995449 >> 172.21.15.7:0/2678995449 conn(0x55ad8d818060 msgr2=0x55ad8d825450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.581+0000 7f45c243e740 1 -- 172.21.15.7:0/2678995449 shutdown_connections 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.581+0000 7f45c243e740 1 -- 172.21.15.7:0/2678995449 wait complete. 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.581+0000 7f45c243e740 1 -- start start 2024-09-10T10:50:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45c243e740 1 Processor -- start 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45c243e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d82a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45c243e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81c410 0x55ad8d82a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45c243e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d826cb0 0x55ad8d8994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45c243e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81abcd0 con 0x55ad8d81baf0 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45c243e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81ad140 con 0x55ad8d81c410 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45c243e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81ac3a0 con 0x55ad8d826cb0 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45c243e740 10 client.0 _ll_register_callbacks cb 0x7f45b0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45bd44b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d82a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45bcc4a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81c410 0x55ad8d82a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45bd44b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d82a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49776/0 (socket says 172.21.15.7:49776) 2024-09-10T10:50:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45bd44b640 1 -- 172.21.15.7:0/1813806536 learned_addr learned my addr 172.21.15.7:0/1813806536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.582+0000 7f45bdc4c640 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d826cb0 0x55ad8d8994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45bcc4a640 1 -- 172.21.15.7:0/1813806536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d826cb0 msgr2=0x55ad8d8994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45bcc4a640 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d826cb0 0x55ad8d8994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45bcc4a640 1 -- 172.21.15.7:0/1813806536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 msgr2=0x55ad8d82a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45bcc4a640 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d82a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45bcc4a640 1 -- 172.21.15.7:0/1813806536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad8d899b80 con 0x55ad8d81c410 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45bd44b640 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d82a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45bcc4a640 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81c410 0x55ad8d82a630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55ad8d825230 tx=0x7f45a80044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6852bc75fe5c8f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45a0ff9640 1 -- 172.21.15.7:0/1813806536 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8022030 con 0x55ad8d81c410 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45a0ff9640 1 -- 172.21.15.7:0/1813806536 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f45a8003db0 con 0x55ad8d81c410 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45c243e740 1 -- 172.21.15.7:0/1813806536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b4036030 con 0x55ad8d81c410 2024-09-10T10:50:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45a0ff9640 1 -- 172.21.15.7:0/1813806536 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f45a801f3a0 con 0x55ad8d81c410 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45c243e740 10 client.25319 fetch_fsmap learned FSMap version 32 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45c243e740 1 -- 172.21.15.7:0/1813806536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ad8d8a0fe0 con 0x55ad8d81c410 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.583+0000 7f45a0ff9640 1 -- 172.21.15.7:0/1813806536 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45b4036030 con 0x55ad8d81c410 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.584+0000 7f45a0ff9640 1 -- 172.21.15.7:0/1813806536 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f45a8030a50 con 0x55ad8d81c410 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.584+0000 7f45c243e740 10 client.25319 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.584+0000 7f45c243e740 10 client.25319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.584+0000 7f45c243e740 20 client.25319 populate_metadata read hostname 'smithi007' 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.584+0000 7f45c243e740 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad8d8a16e0 0x55ad8d8a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.584+0000 7f45c243e740 4 client.25319 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.584+0000 7f45c243e740 1 -- 172.21.15.7:0/1813806536 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad8d787d70 con 0x55ad8d8a16e0 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.584+0000 7f45bd44b640 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad8d8a16e0 0x55ad8d8a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.584+0000 7f45bd44b640 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad8d8a16e0 0x55ad8d8a3b00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce54b352fc583da server_cookie=b559c94fddfae856 in_seq=0 out_seq=0 2024-09-10T10:50:48.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.584+0000 7f45a0ff9640 10 client.25319 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.659+0000 7f45a0ff9640 1 -- 172.21.15.7:0/1813806536 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad8d787d70 con 0x55ad8d8a16e0 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.659+0000 7f45a0ff9640 10 client.25319 handle_command_reply: tid=0 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 client.25319 shutdown 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 -- 172.21.15.7:0/1813806536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81c410 msgr2=0x55ad8d82a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81c410 0x55ad8d82a630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55ad8d825230 tx=0x7f45a80044d0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 -- 172.21.15.7:0/1813806536 shutdown_connections 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55ad8d8a16e0 0x55ad8d8a3b00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad8d826cb0 0x55ad8d8994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ad8d81c410 0x55ad8d82a630 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 --2- 172.21.15.7:0/1813806536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad8d81baf0 0x55ad8d82a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 -- 172.21.15.7:0/1813806536 >> 172.21.15.7:0/1813806536 conn(0x55ad8d818060 msgr2=0x55ad8d81dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45a0ff9640 0 client.25319 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:50:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 -- 172.21.15.7:0/1813806536 shutdown_connections 2024-09-10T10:50:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 1 -- 172.21.15.7:0/1813806536 wait complete. 2024-09-10T10:50:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.660+0000 7f45c243e740 20 client.25319 trim_cache size 0 max 16384 2024-09-10T10:50:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:50:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:50:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:50:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:50:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.661+0000 7f45bf6af640 1 -- 172.21.15.7:0/1153187013 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f45a40a2da0 msgr2=0x7f45a40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.661+0000 7f45bf6af640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f45a40a2da0 0x7f45a40a5260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f45b000b6e0 tx=0x7f45b00046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.661+0000 7f45bf6af640 1 -- 172.21.15.7:0/1153187013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8101ad0 msgr2=0x7f45b8198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.661+0000 7f45bf6af640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8101ad0 0x7f45b8198310 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005e30 tx=0x7f45b4032fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.662+0000 7f45bf6af640 1 -- 172.21.15.7:0/1153187013 shutdown_connections 2024-09-10T10:50:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.662+0000 7f45bf6af640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45b8107b70 0x7f45b819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.662+0000 7f45bf6af640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45b8102470 0x7f45b8198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.662+0000 7f45bf6af640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f45a40a2da0 0x7f45a40a5260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.662+0000 7f45bf6af640 1 --2- 172.21.15.7:0/1153187013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b8101ad0 0x7f45b8198310 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.662+0000 7f45bf6af640 1 -- 172.21.15.7:0/1153187013 >> 172.21.15.7:0/1153187013 conn(0x7f45b80fd7d0 msgr2=0x7f45b80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.662+0000 7f45bf6af640 1 -- 172.21.15.7:0/1153187013 shutdown_connections 2024-09-10T10:50:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:50:48.662+0000 7f45bf6af640 1 -- 172.21.15.7:0/1153187013 wait complete. 2024-09-10T10:50:49.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 540114 12.59 MB/sec execute 109 sec latency 21.458 ms 2024-09-10T10:50:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:49 smithi103 ceph-mon[35160]: pgmap v152: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 314 op/s 2024-09-10T10:50:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:49 smithi007 ceph-mon[30873]: pgmap v152: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 314 op/s 2024-09-10T10:50:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:49 smithi133 ceph-mon[35282]: pgmap v152: 97 pgs: 97 active+clean; 130 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 314 op/s 2024-09-10T10:50:50.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 542349 12.55 MB/sec execute 110 sec latency 11.906 ms 2024-09-10T10:50:51.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 544082 12.54 MB/sec execute 111 sec latency 38.247 ms 2024-09-10T10:50:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:51 smithi103 ceph-mon[35160]: pgmap v153: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 499 op/s 2024-09-10T10:50:51.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:51 smithi007 ceph-mon[30873]: pgmap v153: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 499 op/s 2024-09-10T10:50:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:51 smithi133 ceph-mon[35282]: pgmap v153: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 499 op/s 2024-09-10T10:50:52.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 547031 12.59 MB/sec execute 112 sec latency 16.849 ms 2024-09-10T10:50:53.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 548896 12.54 MB/sec execute 113 sec latency 11.088 ms 2024-09-10T10:50:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:53 smithi103 ceph-mon[35160]: pgmap v154: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 321 op/s 2024-09-10T10:50:53.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:53 smithi007 ceph-mon[30873]: pgmap v154: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 321 op/s 2024-09-10T10:50:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:53 smithi133 ceph-mon[35282]: pgmap v154: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 321 op/s 2024-09-10T10:50:54.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 551170 12.55 MB/sec execute 114 sec latency 13.004 ms 2024-09-10T10:50:55.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 554156 12.59 MB/sec execute 115 sec latency 12.240 ms 2024-09-10T10:50:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:55 smithi103 ceph-mon[35160]: pgmap v155: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 321 op/s 2024-09-10T10:50:55.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:55 smithi007 ceph-mon[30873]: pgmap v155: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 321 op/s 2024-09-10T10:50:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:55 smithi133 ceph-mon[35282]: pgmap v155: 97 pgs: 97 active+clean; 145 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 321 op/s 2024-09-10T10:50:56.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 555809 12.54 MB/sec execute 116 sec latency 32.584 ms 2024-09-10T10:50:57.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 558206 12.55 MB/sec execute 117 sec latency 20.901 ms 2024-09-10T10:50:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:57 smithi103 ceph-mon[35160]: pgmap v156: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 458 op/s 2024-09-10T10:50:57.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:57 smithi007 ceph-mon[30873]: pgmap v156: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 458 op/s 2024-09-10T10:50:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:57 smithi133 ceph-mon[35282]: pgmap v156: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 458 op/s 2024-09-10T10:50:58.512 INFO:tasks.workunit.client.0.smithi007.stdout: 1 561366 12.60 MB/sec execute 118 sec latency 11.466 ms 2024-09-10T10:50:59.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 563639 12.56 MB/sec execute 119 sec latency 11.380 ms 2024-09-10T10:50:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:50:59 smithi103 ceph-mon[35160]: pgmap v157: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 321 op/s 2024-09-10T10:50:59.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:50:59 smithi007 ceph-mon[30873]: pgmap v157: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 321 op/s 2024-09-10T10:50:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:50:59 smithi133 ceph-mon[35282]: pgmap v157: 97 pgs: 97 active+clean; 152 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 321 op/s 2024-09-10T10:51:00.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 565586 12.56 MB/sec execute 120 sec latency 12.891 ms 2024-09-10T10:51:01.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 567562 12.55 MB/sec execute 121 sec latency 47.498 ms 2024-09-10T10:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 -- 172.21.15.7:0/1573366141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e30102470 msgr2=0x7f1e3010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 --2- 172.21.15.7:0/1573366141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e30102470 0x7f1e3010e9a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009cd0 tx=0x7f1e2802a8e0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 -- 172.21.15.7:0/1573366141 shutdown_connections 2024-09-10T10:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 --2- 172.21.15.7:0/1573366141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e30102470 0x7f1e3010e9a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 --2- 172.21.15.7:0/1573366141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30101ad0 0x7f1e30101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 --2- 172.21.15.7:0/1573366141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e30107b70 0x7f1e30107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 -- 172.21.15.7:0/1573366141 >> 172.21.15.7:0/1573366141 conn(0x7f1e300fd7d0 msgr2=0x7f1e300ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 -- 172.21.15.7:0/1573366141 shutdown_connections 2024-09-10T10:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 -- 172.21.15.7:0/1573366141 wait complete. 2024-09-10T10:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 Processor -- start 2024-09-10T10:51:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.759+0000 7f1e36311640 1 -- start start 2024-09-10T10:51:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e36311640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e30101ad0 0x7f1e300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e36311640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e30102470 0x7f1e30074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e36311640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30107b70 0x7f1e30074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e36311640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30111f20 con 0x7f1e30101ad0 2024-09-10T10:51:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e36311640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30111da0 con 0x7f1e30102470 2024-09-10T10:51:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e36311640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301120a0 con 0x7f1e30107b70 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e2ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e30101ad0 0x7f1e300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e2ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e30101ad0 0x7f1e300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43318/0 (socket says 172.21.15.7:43318) 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e2ffff640 1 -- 172.21.15.7:0/3860672884 learned_addr learned my addr 172.21.15.7:0/3860672884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e2f7fe640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e30102470 0x7f1e30074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e348ae640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30107b70 0x7f1e30074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e2ffff640 1 -- 172.21.15.7:0/3860672884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30107b70 msgr2=0x7f1e30074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e2ffff640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30107b70 0x7f1e30074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e2ffff640 1 -- 172.21.15.7:0/3860672884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e30102470 msgr2=0x7f1e30074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e2ffff640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e30102470 0x7f1e30074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e2ffff640 1 -- 172.21.15.7:0/3860672884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e301a6810 con 0x7f1e30101ad0 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.760+0000 7f1e2ffff640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e30101ad0 0x7f1e300744f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000e9f0 tx=0x7f1e2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.761+0000 7f1e2d7fa640 1 -- 172.21.15.7:0/3860672884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2000cd80 con 0x7f1e30101ad0 2024-09-10T10:51:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.761+0000 7f1e2d7fa640 1 -- 172.21.15.7:0/3860672884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e20010910 con 0x7f1e30101ad0 2024-09-10T10:51:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.761+0000 7f1e36311640 1 -- 172.21.15.7:0/3860672884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e301a6b00 con 0x7f1e30101ad0 2024-09-10T10:51:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.761+0000 7f1e2d7fa640 1 -- 172.21.15.7:0/3860672884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20018990 con 0x7f1e30101ad0 2024-09-10T10:51:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.761+0000 7f1e36311640 1 -- 172.21.15.7:0/3860672884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e301a7050 con 0x7f1e30101ad0 2024-09-10T10:51:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.763+0000 7f1e2d7fa640 1 -- 172.21.15.7:0/3860672884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1e20018b30 con 0x7f1e30101ad0 2024-09-10T10:51:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.763+0000 7f1e390a5740 1 Processor -- start 2024-09-10T10:51:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.763+0000 7f1e390a5740 1 -- start start 2024-09-10T10:51:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.763+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d90ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.763+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16d9b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d9bcb0 0x564d16d9e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e390a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564d16cf52f0 con 0x564d16d9bcb0 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e390a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564d16c9d270 con 0x564d16d90af0 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e390a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564d16cf12b0 con 0x564d16d91410 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e348ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d90ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e2ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16d9b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d9bcb0 0x564d16d9e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e348ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d90ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:35644/0 (socket says 172.21.15.7:35644) 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e348ae640 1 -- 172.21.15.7:0/2814351262 learned_addr learned my addr 172.21.15.7:0/2814351262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e348ae640 1 -- 172.21.15.7:0/2814351262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 msgr2=0x564d16d9b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e348ae640 1 --2- 172.21.15.7:0/2814351262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16d9b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e348ae640 1 -- 172.21.15.7:0/2814351262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d9bcb0 msgr2=0x564d16d9e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e348ae640 1 --2- 172.21.15.7:0/2814351262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d9bcb0 0x564d16d9e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e348ae640 1 -- 172.21.15.7:0/2814351262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d16c9c340 con 0x564d16d90af0 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e2f7fe640 1 --2- 172.21.15.7:0/2814351262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d9bcb0 0x564d16d9e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e2d7fa640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1e180a2da0 0x7f1e180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e2ffff640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1e180a2da0 0x7f1e180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e348ae640 1 --2- 172.21.15.7:0/2814351262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d90ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009cd0 tx=0x7f1e28009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3e49d701ca92bcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e2d7fa640 1 -- 172.21.15.7:0/3860672884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f1e200cf780 con 0x7f1e30101ad0 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e167fc640 1 -- 172.21.15.7:0/2814351262 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28048070 con 0x564d16d90af0 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e167fc640 1 -- 172.21.15.7:0/2814351262 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e28036070 con 0x564d16d90af0 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.764+0000 7f1e167fc640 1 -- 172.21.15.7:0/2814351262 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28004440 con 0x564d16d90af0 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e390a5740 1 -- 172.21.15.7:0/2814351262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 msgr2=0x564d16d90ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e2ffff640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1e180a2da0 0x7f1e180a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x564d16d8dec0 tx=0x7f1e200d5530 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e390a5740 1 --2- 172.21.15.7:0/2814351262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d90ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009cd0 tx=0x7f1e28009a20 comp rx=0 tx=0).stop 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e390a5740 1 -- 172.21.15.7:0/2814351262 shutdown_connections 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e390a5740 1 --2- 172.21.15.7:0/2814351262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d9bcb0 0x564d16d9e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e390a5740 1 --2- 172.21.15.7:0/2814351262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16d9b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e390a5740 1 --2- 172.21.15.7:0/2814351262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d90ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e390a5740 1 -- 172.21.15.7:0/2814351262 >> 172.21.15.7:0/2814351262 conn(0x564d16d8d060 msgr2=0x564d16d8d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e390a5740 1 -- 172.21.15.7:0/2814351262 shutdown_connections 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e390a5740 1 -- 172.21.15.7:0/2814351262 wait complete. 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.765+0000 7f1e390a5740 1 -- start start 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e390a5740 1 Processor -- start 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d98c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d91410 0x564d16d99150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d9bcb0 0x564d16d99690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e390a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564d16d9e880 con 0x564d16d91410 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e390a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564d16c7e290 con 0x564d16d90af0 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e390a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564d16d9ea00 con 0x564d16d9bcb0 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e390a5740 10 client.0 _ll_register_callbacks cb 0x564d16caf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d91410 0x564d16d99150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e348ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d98c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e348ae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d98c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:35656/0 (socket says 172.21.15.7:35656) 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e348ae640 1 -- 172.21.15.7:0/3253510452 learned_addr learned my addr 172.21.15.7:0/3253510452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e348ae640 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d9bcb0 0x564d16d99690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e348ae640 1 -- 172.21.15.7:0/3253510452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d9bcb0 msgr2=0x564d16d99690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e348ae640 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d9bcb0 0x564d16d99690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e348ae640 1 -- 172.21.15.7:0/3253510452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d91410 msgr2=0x564d16d99150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.766+0000 7f1e348ae640 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d91410 0x564d16d99150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e348ae640 1 -- 172.21.15.7:0/3253510452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d16a08210 con 0x564d16d90af0 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e348ae640 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d98c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1e28003e60 tx=0x7f1e28003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10fb03b4db750464 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/3253510452 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28056060 con 0x564d16d90af0 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e390a5740 1 -- 172.21.15.7:0/3253510452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e28036070 con 0x564d16d90af0 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/3253510452 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f1e280043e0 con 0x564d16d90af0 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/3253510452 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1e28048040 con 0x564d16d90af0 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e390a5740 10 client.25331 fetch_fsmap learned FSMap version 32 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e390a5740 1 -- 172.21.15.7:0/3253510452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d16ad4620 con 0x564d16d90af0 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/3253510452 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e28036070 con 0x564d16d90af0 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/3253510452 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1e2804e040 con 0x564d16d90af0 2024-09-10T10:51:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e390a5740 10 client.25331 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e390a5740 10 client.25331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e390a5740 20 client.25331 populate_metadata read hostname 'smithi007' 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e390a5740 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564d16dabb30 0x564d16dadf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.767+0000 7f1e390a5740 4 client.25331 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.768+0000 7f1e390a5740 1 -- 172.21.15.7:0/3253510452 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d16d02a20 con 0x564d16dabb30 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.768+0000 7f1e2f7fe640 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564d16dabb30 0x564d16dadf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.768+0000 7f1e0f7fe640 10 client.25331 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.768+0000 7f1e2f7fe640 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564d16dabb30 0x564d16dadf20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c30200fc68e99e server_cookie=c0356bf4e5b2a7d2 in_seq=0 out_seq=0 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/3253510452 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d16d02a20 con 0x564d16dabb30 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e0f7fe640 10 client.25331 handle_command_reply: tid=0 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 client.25331 shutdown 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 -- 172.21.15.7:0/3253510452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 msgr2=0x564d16d98c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d98c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1e28003e60 tx=0x7f1e28003f60 comp rx=0 tx=0).stop 2024-09-10T10:51:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 -- 172.21.15.7:0/3253510452 shutdown_connections 2024-09-10T10:51:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564d16dabb30 0x564d16dadf20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d9bcb0 0x564d16d99690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d91410 0x564d16d99150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 --2- 172.21.15.7:0/3253510452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d90af0 0x564d16d98c10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e0f7fe640 0 client.25331 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 -- 172.21.15.7:0/3253510452 >> 172.21.15.7:0/3253510452 conn(0x564d16d8d060 msgr2=0x564d16d92d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 -- 172.21.15.7:0/3253510452 shutdown_connections 2024-09-10T10:51:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 1 -- 172.21.15.7:0/3253510452 wait complete. 2024-09-10T10:51:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.769+0000 7f1e390a5740 20 client.25331 trim_cache size 0 max 16384 2024-09-10T10:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.784+0000 7f1e390a5740 1 Processor -- start 2024-09-10T10:51:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.784+0000 7f1e390a5740 1 -- start start 2024-09-10T10:51:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.784+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16d994c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16db0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d9bcb0 0x564d16db1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e390a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301b2240 con 0x564d16d90af0 2024-09-10T10:51:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e390a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301afb10 con 0x564d16d9bcb0 2024-09-10T10:51:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e390a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301ad960 con 0x564d16d91410 2024-09-10T10:51:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e348ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16d994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e348ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16d994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43344/0 (socket says 172.21.15.7:43344) 2024-09-10T10:51:01.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e348ae640 1 -- 172.21.15.7:0/651816293 learned_addr learned my addr 172.21.15.7:0/651816293 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e2f7fe640 1 --2- 172.21.15.7:0/651816293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16db0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e348ae640 1 --2- 172.21.15.7:0/651816293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d9bcb0 0x564d16db1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e348ae640 1 -- 172.21.15.7:0/651816293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 msgr2=0x564d16db0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e348ae640 1 --2- 172.21.15.7:0/651816293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16db0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e348ae640 1 -- 172.21.15.7:0/651816293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d9bcb0 msgr2=0x564d16db1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e348ae640 1 --2- 172.21.15.7:0/651816293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d9bcb0 0x564d16db1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e348ae640 1 -- 172.21.15.7:0/651816293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d16d92000 con 0x564d16d90af0 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e348ae640 1 --2- 172.21.15.7:0/651816293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16d994c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1e28030af0 tx=0x7f1e28009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393c46646372848f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e15ffb640 1 -- 172.21.15.7:0/651816293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2803f070 con 0x564d16d90af0 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e15ffb640 1 -- 172.21.15.7:0/651816293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e28036070 con 0x564d16d90af0 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.785+0000 7f1e15ffb640 1 -- 172.21.15.7:0/651816293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28056030 con 0x564d16d90af0 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.786+0000 7f1e390a5740 1 -- 172.21.15.7:0/651816293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 msgr2=0x564d16d994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:01.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.786+0000 7f1e390a5740 1 --2- 172.21.15.7:0/651816293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16d994c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1e28030af0 tx=0x7f1e28009a20 comp rx=0 tx=0).stop 2024-09-10T10:51:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.786+0000 7f1e390a5740 1 -- 172.21.15.7:0/651816293 shutdown_connections 2024-09-10T10:51:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.786+0000 7f1e390a5740 1 --2- 172.21.15.7:0/651816293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d9bcb0 0x564d16db1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.786+0000 7f1e390a5740 1 --2- 172.21.15.7:0/651816293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16db0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.786+0000 7f1e390a5740 1 --2- 172.21.15.7:0/651816293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16d994c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.786+0000 7f1e390a5740 1 -- 172.21.15.7:0/651816293 >> 172.21.15.7:0/651816293 conn(0x564d16d8d060 msgr2=0x564d16d9ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.786+0000 7f1e390a5740 1 -- 172.21.15.7:0/651816293 shutdown_connections 2024-09-10T10:51:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.786+0000 7f1e390a5740 1 -- 172.21.15.7:0/651816293 wait complete. 2024-09-10T10:51:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.786+0000 7f1e390a5740 1 -- start start 2024-09-10T10:51:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e390a5740 1 Processor -- start 2024-09-10T10:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16daef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16daf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e390a5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d9bcb0 0x564d16daf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e390a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301adfd0 con 0x564d16d90af0 2024-09-10T10:51:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e390a5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301af440 con 0x564d16d9bcb0 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e390a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301ae6a0 con 0x564d16d91410 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e390a5740 10 client.0 _ll_register_callbacks cb 0x7f1e28046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e348ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16daef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e348ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16daef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43352/0 (socket says 172.21.15.7:43352) 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e348ae640 1 -- 172.21.15.7:0/1134266422 learned_addr learned my addr 172.21.15.7:0/1134266422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e2f7fe640 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16daf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e348ae640 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d9bcb0 0x564d16daf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e348ae640 1 -- 172.21.15.7:0/1134266422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 msgr2=0x564d16daf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e348ae640 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16daf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e348ae640 1 -- 172.21.15.7:0/1134266422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d9bcb0 msgr2=0x564d16daf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e348ae640 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d9bcb0 0x564d16daf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e348ae640 1 -- 172.21.15.7:0/1134266422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d16e15670 con 0x564d16d90af0 2024-09-10T10:51:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e348ae640 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16daef00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1e28030810 tx=0x7f1e28003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1358aae3f7146767 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.787+0000 7f1e0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/1134266422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2803f040 con 0x564d16d90af0 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/1134266422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f1e28004330 con 0x564d16d90af0 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e390a5740 1 -- 172.21.15.7:0/1134266422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e28036070 con 0x564d16d90af0 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/1134266422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1e28009540 con 0x564d16d90af0 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e390a5740 10 client.15657 fetch_fsmap learned FSMap version 32 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e390a5740 1 -- 172.21.15.7:0/1134266422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d16e16940 con 0x564d16d90af0 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/1134266422 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e28036070 con 0x564d16d90af0 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/1134266422 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f1e28042030 con 0x564d16d90af0 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e390a5740 10 client.15657 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e390a5740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e390a5740 20 client.15657 populate_metadata read hostname 'smithi007' 2024-09-10T10:51:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e390a5740 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564d16e16e60 0x564d16e19250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e390a5740 4 client.15657 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:51:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e390a5740 1 -- 172.21.15.7:0/1134266422 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d16cfcd70 con 0x564d16e16e60 2024-09-10T10:51:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.788+0000 7f1e2f7fe640 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564d16e16e60 0x564d16e19250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.789+0000 7f1e2f7fe640 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564d16e16e60 0x564d16e19250 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703dc2fc0ca30c38 server_cookie=226a8966771bd1bb in_seq=0 out_seq=0 2024-09-10T10:51:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.789+0000 7f1e0f7fe640 10 client.15657 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e0f7fe640 1 -- 172.21.15.7:0/1134266422 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d16cfcd70 con 0x564d16e16e60 2024-09-10T10:51:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e0f7fe640 10 client.15657 handle_command_reply: tid=0 2024-09-10T10:51:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e390a5740 1 client.15657 shutdown 2024-09-10T10:51:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e390a5740 1 -- 172.21.15.7:0/1134266422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 msgr2=0x564d16daef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e390a5740 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16daef00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1e28030810 tx=0x7f1e28003b90 comp rx=0 tx=0).stop 2024-09-10T10:51:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e390a5740 1 -- 172.21.15.7:0/1134266422 shutdown_connections 2024-09-10T10:51:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e390a5740 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564d16e16e60 0x564d16e19250 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e390a5740 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564d16d9bcb0 0x564d16daf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e390a5740 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d16d91410 0x564d16daf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e0f7fe640 0 client.15657 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e390a5740 1 --2- 172.21.15.7:0/1134266422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d16d90af0 0x564d16daef00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.857+0000 7f1e390a5740 1 -- 172.21.15.7:0/1134266422 >> 172.21.15.7:0/1134266422 conn(0x564d16d8d060 msgr2=0x564d16da6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.858+0000 7f1e390a5740 1 -- 172.21.15.7:0/1134266422 shutdown_connections 2024-09-10T10:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.858+0000 7f1e390a5740 1 -- 172.21.15.7:0/1134266422 wait complete. 2024-09-10T10:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.858+0000 7f1e390a5740 20 client.15657 trim_cache size 0 max 16384 2024-09-10T10:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:51:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.858+0000 7f1e36311640 1 -- 172.21.15.7:0/3860672884 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1e180a2da0 msgr2=0x7f1e180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.858+0000 7f1e36311640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1e180a2da0 0x7f1e180a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x564d16d8dec0 tx=0x7f1e200d5530 comp rx=0 tx=0).stop 2024-09-10T10:51:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.858+0000 7f1e36311640 1 -- 172.21.15.7:0/3860672884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e30101ad0 msgr2=0x7f1e300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.858+0000 7f1e36311640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e30101ad0 0x7f1e300744f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000e9f0 tx=0x7f1e2000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.859+0000 7f1e36311640 1 -- 172.21.15.7:0/3860672884 shutdown_connections 2024-09-10T10:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.859+0000 7f1e36311640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30107b70 0x7f1e30074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.859+0000 7f1e36311640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e30102470 0x7f1e30074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.859+0000 7f1e36311640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1e180a2da0 0x7f1e180a5260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.859+0000 7f1e36311640 1 --2- 172.21.15.7:0/3860672884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e30101ad0 0x7f1e300744f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.859+0000 7f1e36311640 1 -- 172.21.15.7:0/3860672884 >> 172.21.15.7:0/3860672884 conn(0x7f1e300fd7d0 msgr2=0x7f1e300fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.859+0000 7f1e36311640 1 -- 172.21.15.7:0/3860672884 shutdown_connections 2024-09-10T10:51:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:01.859+0000 7f1e36311640 1 -- 172.21.15.7:0/3860672884 wait complete. 2024-09-10T10:51:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:01 smithi103 ceph-mon[35160]: pgmap v158: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 458 op/s 2024-09-10T10:51:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:01 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:51:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:01 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:51:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:01 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:51:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:01 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:51:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:01 smithi007 ceph-mon[30873]: pgmap v158: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 458 op/s 2024-09-10T10:51:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:01 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:51:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:01 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:51:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:01 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:51:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:01 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:51:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:01 smithi133 ceph-mon[35282]: pgmap v158: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 458 op/s 2024-09-10T10:51:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:01 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:51:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:01 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:51:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:01 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:51:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:01 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:51:02.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 569134 12.51 MB/sec execute 122 sec latency 24.266 ms 2024-09-10T10:51:03.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 572088 12.56 MB/sec execute 123 sec latency 19.186 ms 2024-09-10T10:51:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:03 smithi103 ceph-mon[35160]: pgmap v159: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-09-10T10:51:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:03 smithi007 ceph-mon[30873]: pgmap v159: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-09-10T10:51:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:03 smithi133 ceph-mon[35282]: pgmap v159: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-09-10T10:51:04.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 574148 12.51 MB/sec execute 124 sec latency 13.301 ms 2024-09-10T10:51:05.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 576119 12.52 MB/sec execute 125 sec latency 16.964 ms 2024-09-10T10:51:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:05 smithi103 ceph-mon[35160]: pgmap v160: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-09-10T10:51:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:05 smithi007 ceph-mon[30873]: pgmap v160: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-09-10T10:51:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:05 smithi133 ceph-mon[35282]: pgmap v160: 97 pgs: 97 active+clean; 158 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-09-10T10:51:06.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 579086 12.56 MB/sec execute 126 sec latency 13.259 ms 2024-09-10T10:51:07.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 580683 12.51 MB/sec execute 127 sec latency 52.820 ms 2024-09-10T10:51:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:07 smithi103 ceph-mon[35160]: pgmap v161: 97 pgs: 97 active+clean; 153 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.1 MiB/s wr, 478 op/s 2024-09-10T10:51:07.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:07 smithi007 ceph-mon[30873]: pgmap v161: 97 pgs: 97 active+clean; 153 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.1 MiB/s wr, 478 op/s 2024-09-10T10:51:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:07 smithi133 ceph-mon[35282]: pgmap v161: 97 pgs: 97 active+clean; 153 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.1 MiB/s wr, 478 op/s 2024-09-10T10:51:08.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 582964 12.52 MB/sec execute 128 sec latency 16.993 ms 2024-09-10T10:51:09.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 585904 12.54 MB/sec execute 129 sec latency 16.992 ms 2024-09-10T10:51:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:09 smithi103 ceph-mon[35160]: pgmap v162: 97 pgs: 97 active+clean; 153 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-09-10T10:51:09.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:09 smithi007 ceph-mon[30873]: pgmap v162: 97 pgs: 97 active+clean; 153 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-09-10T10:51:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:09 smithi133 ceph-mon[35282]: pgmap v162: 97 pgs: 97 active+clean; 153 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 341 op/s 2024-09-10T10:51:10.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 587766 12.52 MB/sec execute 130 sec latency 11.586 ms 2024-09-10T10:51:11.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 590084 12.53 MB/sec execute 131 sec latency 12.615 ms 2024-09-10T10:51:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:11 smithi103 ceph-mon[35160]: pgmap v163: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 472 op/s 2024-09-10T10:51:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:11 smithi007 ceph-mon[30873]: pgmap v163: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 472 op/s 2024-09-10T10:51:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:11 smithi133 ceph-mon[35282]: pgmap v163: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 472 op/s 2024-09-10T10:51:12.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 592794 12.53 MB/sec execute 132 sec latency 48.750 ms 2024-09-10T10:51:13.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 593897 12.48 MB/sec execute 133 sec latency 235.404 ms 2024-09-10T10:51:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:13 smithi103 ceph-mon[35160]: pgmap v164: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 335 op/s 2024-09-10T10:51:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:13 smithi007 ceph-mon[30873]: pgmap v164: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 335 op/s 2024-09-10T10:51:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:13 smithi133 ceph-mon[35282]: pgmap v164: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 335 op/s 2024-09-10T10:51:14.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 596827 12.52 MB/sec execute 134 sec latency 12.798 ms 2024-09-10T10:51:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.952+0000 7fdf89986640 1 -- 172.21.15.7:0/1728294123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf841009b0 msgr2=0x7fdf84100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.952+0000 7fdf89986640 1 --2- 172.21.15.7:0/1728294123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf841009b0 0x7fdf84100e10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009930 tx=0x7fdf6c02a420 comp rx=0 tx=0).stop 2024-09-10T10:51:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 -- 172.21.15.7:0/1728294123 shutdown_connections 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 --2- 172.21.15.7:0/1728294123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84101350 0x7fdf8410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 --2- 172.21.15.7:0/1728294123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf841009b0 0x7fdf84100e10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 --2- 172.21.15.7:0/1728294123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf841069c0 0x7fdf84106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 -- 172.21.15.7:0/1728294123 >> 172.21.15.7:0/1728294123 conn(0x7fdf84077990 msgr2=0x7fdf840fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 -- 172.21.15.7:0/1728294123 shutdown_connections 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 -- 172.21.15.7:0/1728294123 wait complete. 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 Processor -- start 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 -- start start 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf841009b0 0x7fdf8410e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84101350 0x7fdf8410e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf841069c0 0x7fdf8410ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84111eb0 con 0x7fdf841069c0 2024-09-10T10:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84111d30 con 0x7fdf841009b0 2024-09-10T10:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf89986640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf84112030 con 0x7fdf84101350 2024-09-10T10:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf841069c0 0x7fdf8410ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf841069c0 0x7fdf8410ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44300/0 (socket says 172.21.15.7:44300) 2024-09-10T10:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.953+0000 7fdf837fe640 1 -- 172.21.15.7:0/3939189812 learned_addr learned my addr 172.21.15.7:0/3939189812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84101350 0x7fdf8410e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf841009b0 0x7fdf8410e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf837fe640 1 -- 172.21.15.7:0/3939189812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84101350 msgr2=0x7fdf8410e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf837fe640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84101350 0x7fdf8410e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf837fe640 1 -- 172.21.15.7:0/3939189812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf841009b0 msgr2=0x7fdf8410e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf837fe640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf841009b0 0x7fdf8410e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf837fe640 1 -- 172.21.15.7:0/3939189812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf841a6860 con 0x7fdf841069c0 2024-09-10T10:51:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf837fe640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf841069c0 0x7fdf8410ed90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00eb10 tx=0x7fdf7c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf88984640 1 -- 172.21.15.7:0/3939189812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c004b20 con 0x7fdf841069c0 2024-09-10T10:51:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf88984640 1 -- 172.21.15.7:0/3939189812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf7c004cc0 con 0x7fdf841069c0 2024-09-10T10:51:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf89986640 1 -- 172.21.15.7:0/3939189812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf841a6b50 con 0x7fdf841069c0 2024-09-10T10:51:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf88984640 1 -- 172.21.15.7:0/3939189812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c010760 con 0x7fdf841069c0 2024-09-10T10:51:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.954+0000 7fdf89986640 1 -- 172.21.15.7:0/3939189812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf841a7070 con 0x7fdf841069c0 2024-09-10T10:51:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.957+0000 7fdf88984640 1 -- 172.21.15.7:0/3939189812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fdf7c005020 con 0x7fdf841069c0 2024-09-10T10:51:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.957+0000 7fdf8c7f9740 1 Processor -- start 2024-09-10T10:51:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.957+0000 7fdf8c7f9740 1 -- start start 2024-09-10T10:51:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 0x55d2d9d01ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d02410 0x55d2d9d0c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d0f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9c662f0 con 0x55d2d9d02410 2024-09-10T10:51:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9c0e270 con 0x55d2d9d0ccb0 2024-09-10T10:51:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9c622b0 con 0x55d2d9d01af0 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf82ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d02410 0x55d2d9d0c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf82ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d02410 0x55d2d9d0c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44314/0 (socket says 172.21.15.7:44314) 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf82ffd640 1 -- 172.21.15.7:0/2982129686 learned_addr learned my addr 172.21.15.7:0/2982129686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/2982129686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 0x55d2d9d01ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf82ffd640 1 -- 172.21.15.7:0/2982129686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 msgr2=0x55d2d9d01ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/2982129686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d0f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/2982129686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 0x55d2d9d01ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf82ffd640 1 -- 172.21.15.7:0/2982129686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d0ccb0 msgr2=0x55d2d9d0f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/2982129686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d0f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf82ffd640 1 -- 172.21.15.7:0/2982129686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d9c0d340 con 0x55d2d9d02410 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/2982129686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d0f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf88984640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fdf700a2e20 0x7fdf700a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fdf700a2e20 0x7fdf700a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/2982129686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d02410 0x55d2d9d0c770 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009bf0 tx=0x7fdf740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87366a446e1e1b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf88984640 1 -- 172.21.15.7:0/3939189812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fdf7c0cfa30 con 0x7fdf841069c0 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.958+0000 7fdf79ffb640 1 -- 172.21.15.7:0/2982129686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74012450 con 0x55d2d9d02410 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf79ffb640 1 -- 172.21.15.7:0/2982129686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf74012ad0 con 0x55d2d9d02410 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf79ffb640 1 -- 172.21.15.7:0/2982129686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400fba0 con 0x55d2d9d02410 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2982129686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d02410 msgr2=0x55d2d9d0c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/2982129686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d02410 0x55d2d9d0c770 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009bf0 tx=0x7fdf740117b0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fdf700a2e20 0x7fdf700a52e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009900 tx=0x7fdf6c0058d0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2982129686 shutdown_connections 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/2982129686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d0f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/2982129686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d02410 0x55d2d9d0c770 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/2982129686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 0x55d2d9d01ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2982129686 >> 172.21.15.7:0/2982129686 conn(0x55d2d9cfe060 msgr2=0x55d2d9cfe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2982129686 shutdown_connections 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.959+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2982129686 wait complete. 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf8c7f9740 1 -- start start 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf8c7f9740 1 Processor -- start 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 0x55d2d9d09b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 0x55d2d9d0a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d0a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9d0f880 con 0x55d2d9d0ccb0 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9bef290 con 0x55d2d9d02410 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d2d9d0fa00 con 0x55d2d9d01af0 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf8c7f9740 10 client.0 _ll_register_callbacks cb 0x55d2d9c20140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf82ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 0x55d2d9d09b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 0x55d2d9d0a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf82ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 0x55d2d9d09b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:60266/0 (socket says 172.21.15.7:60266) 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf7bfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 0x55d2d9d0a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:42502/0 (socket says 172.21.15.7:42502) 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf82ffd640 1 -- 172.21.15.7:0/1468387763 learned_addr learned my addr 172.21.15.7:0/1468387763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.960+0000 7fdf837fe640 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d0a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf82ffd640 1 -- 172.21.15.7:0/1468387763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 msgr2=0x55d2d9d0a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 0x55d2d9d0a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf82ffd640 1 -- 172.21.15.7:0/1468387763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 msgr2=0x55d2d9d0a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d0a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf82ffd640 1 -- 172.21.15.7:0/1468387763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d9979210 con 0x55d2d9d01af0 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf837fe640 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d0a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 0x55d2d9d09b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdf74005930 tx=0x7fdf74009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=744f7d254c061ce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf62ffd640 1 -- 172.21.15.7:0/1468387763 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74028020 con 0x55d2d9d01af0 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf62ffd640 1 -- 172.21.15.7:0/1468387763 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fdf7400f8a0 con 0x55d2d9d01af0 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/1468387763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf74012ad0 con 0x55d2d9d01af0 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf62ffd640 1 -- 172.21.15.7:0/1468387763 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdf74033050 con 0x55d2d9d01af0 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf8c7f9740 10 client.25357 fetch_fsmap learned FSMap version 32 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.961+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/1468387763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d2d9a45620 con 0x55d2d9d01af0 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf62ffd640 1 -- 172.21.15.7:0/1468387763 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf74012ad0 con 0x55d2d9d01af0 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf62ffd640 1 -- 172.21.15.7:0/1468387763 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdf740046c0 con 0x55d2d9d01af0 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf8c7f9740 10 client.25357 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf8c7f9740 10 client.25357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf8c7f9740 20 client.25357 populate_metadata read hostname 'smithi007' 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d2d9d1ca90 0x55d2d9d1ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf8c7f9740 4 client.25357 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/1468387763 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2d9c73a20 con 0x55d2d9d1ca90 2024-09-10T10:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d2d9d1ca90 0x55d2d9d1ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d2d9d1ca90 0x55d2d9d1ee80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd5b5de8bd7ad3e6 server_cookie=a39dcc8e0b4c8011 in_seq=0 out_seq=0 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.962+0000 7fdf62ffd640 10 client.25357 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf62ffd640 1 -- 172.21.15.7:0/1468387763 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2d9c73a20 con 0x55d2d9d1ca90 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf62ffd640 10 client.25357 handle_command_reply: tid=0 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 client.25357 shutdown 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/1468387763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 msgr2=0x55d2d9d09b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 0x55d2d9d09b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdf74005930 tx=0x7fdf74009470 comp rx=0 tx=0).stop 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/1468387763 shutdown_connections 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d2d9d1ca90 0x55d2d9d1ee80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d0a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 0x55d2d9d0a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/1468387763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d01af0 0x55d2d9d09b30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf62ffd640 0 client.25357 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/1468387763 >> 172.21.15.7:0/1468387763 conn(0x55d2d9cfe060 msgr2=0x55d2d9d03d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/1468387763 shutdown_connections 2024-09-10T10:51:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/1468387763 wait complete. 2024-09-10T10:51:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.964+0000 7fdf8c7f9740 20 client.25357 trim_cache size 0 max 16384 2024-09-10T10:51:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf8c7f9740 1 Processor -- start 2024-09-10T10:51:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf8c7f9740 1 -- start start 2024-09-10T10:51:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d01af0 0x55d2d9d0a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d02410 0x55d2d9d03c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d24450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841b2260 con 0x55d2d9d0ccb0 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841afb30 con 0x55d2d9d01af0 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841ad980 con 0x55d2d9d02410 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d24450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf7bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d02410 0x55d2d9d03c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d24450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44324/0 (socket says 172.21.15.7:44324) 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf837fe640 1 -- 172.21.15.7:0/553295831 learned_addr learned my addr 172.21.15.7:0/553295831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/553295831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d01af0 0x55d2d9d0a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf837fe640 1 -- 172.21.15.7:0/553295831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d02410 msgr2=0x55d2d9d03c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf837fe640 1 --2- 172.21.15.7:0/553295831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d02410 0x55d2d9d03c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf837fe640 1 -- 172.21.15.7:0/553295831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d01af0 msgr2=0x55d2d9d0a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf837fe640 1 --2- 172.21.15.7:0/553295831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d01af0 0x55d2d9d0a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf837fe640 1 -- 172.21.15.7:0/553295831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d9cfde90 con 0x55d2d9d0ccb0 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.979+0000 7fdf837fe640 1 --2- 172.21.15.7:0/553295831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d24450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c005880 tx=0x7fdf7c09bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a4beda174a0b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf797fa640 1 -- 172.21.15.7:0/553295831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c09c650 con 0x55d2d9d0ccb0 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf797fa640 1 -- 172.21.15.7:0/553295831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf7c09bf50 con 0x55d2d9d0ccb0 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf797fa640 1 -- 172.21.15.7:0/553295831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0a4990 con 0x55d2d9d0ccb0 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/553295831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 msgr2=0x55d2d9d24450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/553295831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d24450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c005880 tx=0x7fdf7c09bde0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/553295831 shutdown_connections 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/553295831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d24450 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/553295831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d02410 0x55d2d9d03c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/553295831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d01af0 0x55d2d9d0a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/553295831 >> 172.21.15.7:0/553295831 conn(0x55d2d9cfe060 msgr2=0x55d2d9d03550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/553295831 shutdown_connections 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.980+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/553295831 wait complete. 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf8c7f9740 1 -- start start 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf8c7f9740 1 Processor -- start 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d01af0 0x55d2d9d15360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 0x55d2d9d06700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf8c7f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d06c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841adff0 con 0x55d2d9d01af0 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841af460 con 0x55d2d9d02410 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf8c7f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf841ae6c0 con 0x55d2d9d0ccb0 2024-09-10T10:51:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf8c7f9740 10 client.0 _ll_register_callbacks cb 0x7fdf7c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf82ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d01af0 0x55d2d9d15360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf82ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d01af0 0x55d2d9d15360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44338/0 (socket says 172.21.15.7:44338) 2024-09-10T10:51:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf82ffd640 1 -- 172.21.15.7:0/2926346770 learned_addr learned my addr 172.21.15.7:0/2926346770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf837fe640 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d06c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 0x55d2d9d06700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf82ffd640 1 -- 172.21.15.7:0/2926346770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d0ccb0 msgr2=0x55d2d9d06c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d06c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf82ffd640 1 -- 172.21.15.7:0/2926346770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 msgr2=0x55d2d9d06700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 0x55d2d9d06700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.981+0000 7fdf82ffd640 1 -- 172.21.15.7:0/2926346770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d9d07440 con 0x55d2d9d01af0 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf82ffd640 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d01af0 0x55d2d9d15360 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdf74004ab0 tx=0x7fdf74005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78d05f4f8d38569 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf62ffd640 1 -- 172.21.15.7:0/2926346770 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74028040 con 0x55d2d9d01af0 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf62ffd640 1 -- 172.21.15.7:0/2926346770 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fdf74012cd0 con 0x55d2d9d01af0 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2926346770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf7c09bf50 con 0x55d2d9d01af0 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf62ffd640 1 -- 172.21.15.7:0/2926346770 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdf74048050 con 0x55d2d9d01af0 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf8c7f9740 10 client.15675 fetch_fsmap learned FSMap version 32 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2926346770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d2d9d87420 con 0x55d2d9d01af0 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf62ffd640 1 -- 172.21.15.7:0/2926346770 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf7c09bf50 con 0x55d2d9d01af0 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf62ffd640 1 -- 172.21.15.7:0/2926346770 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fdf74004d20 con 0x55d2d9d01af0 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf8c7f9740 10 client.15675 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:51:14.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf8c7f9740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf8c7f9740 20 client.15675 populate_metadata read hostname 'smithi007' 2024-09-10T10:51:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d2d9d87a80 0x55d2d9d89e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf8c7f9740 4 client.15675 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:51:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.982+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2926346770 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2d9c6dd70 con 0x55d2d9d87a80 2024-09-10T10:51:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.983+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d2d9d87a80 0x55d2d9d89e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.983+0000 7fdf62ffd640 10 client.15675 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:14.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:14.983+0000 7fdf7bfff640 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d2d9d87a80 0x55d2d9d89e70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e870378f9ce23e8a server_cookie=872a17839c356cb3 in_seq=0 out_seq=0 2024-09-10T10:51:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf62ffd640 1 -- 172.21.15.7:0/2926346770 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2d9c6dd70 con 0x55d2d9d87a80 2024-09-10T10:51:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf62ffd640 10 client.15675 handle_command_reply: tid=0 2024-09-10T10:51:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 client.15675 shutdown 2024-09-10T10:51:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2926346770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d01af0 msgr2=0x55d2d9d15360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d01af0 0x55d2d9d15360 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdf74004ab0 tx=0x7fdf74005f10 comp rx=0 tx=0).stop 2024-09-10T10:51:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2926346770 shutdown_connections 2024-09-10T10:51:15.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d2d9d87a80 0x55d2d9d89e70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf62ffd640 0 client.15675 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2d9d0ccb0 0x55d2d9d06c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2d9d02410 0x55d2d9d06700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 --2- 172.21.15.7:0/2926346770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2d9d01af0 0x55d2d9d15360 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2926346770 >> 172.21.15.7:0/2926346770 conn(0x55d2d9cfe060 msgr2=0x55d2d9cfd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2926346770 shutdown_connections 2024-09-10T10:51:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.056+0000 7fdf8c7f9740 1 -- 172.21.15.7:0/2926346770 wait complete. 2024-09-10T10:51:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf8c7f9740 20 client.15675 trim_cache size 0 max 16384 2024-09-10T10:51:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:51:15.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:51:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:51:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:51:15.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 -- 172.21.15.7:0/3939189812 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fdf700a2e20 msgr2=0x7fdf700a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fdf700a2e20 0x7fdf700a52e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009900 tx=0x7fdf6c0058d0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 -- 172.21.15.7:0/3939189812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf841069c0 msgr2=0x7fdf8410ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf841069c0 0x7fdf8410ed90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00eb10 tx=0x7fdf7c00c2d0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 -- 172.21.15.7:0/3939189812 shutdown_connections 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fdf700a2e20 0x7fdf700a52e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf841069c0 0x7fdf8410ed90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf84101350 0x7fdf8410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 --2- 172.21.15.7:0/3939189812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdf841009b0 0x7fdf8410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 -- 172.21.15.7:0/3939189812 >> 172.21.15.7:0/3939189812 conn(0x7fdf84077990 msgr2=0x7fdf84103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 -- 172.21.15.7:0/3939189812 shutdown_connections 2024-09-10T10:51:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:15.057+0000 7fdf89986640 1 -- 172.21.15.7:0/3939189812 wait complete. 2024-09-10T10:51:15.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 598487 12.48 MB/sec execute 135 sec latency 36.895 ms 2024-09-10T10:51:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:15 smithi103 ceph-mon[35160]: pgmap v165: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 335 op/s 2024-09-10T10:51:15.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:15 smithi007 ceph-mon[30873]: pgmap v165: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 335 op/s 2024-09-10T10:51:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:15 smithi133 ceph-mon[35282]: pgmap v165: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 335 op/s 2024-09-10T10:51:16.513 INFO:tasks.workunit.client.0.smithi007.stdout: 1 600708 12.49 MB/sec execute 136 sec latency 13.194 ms 2024-09-10T10:51:17.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 603426 12.49 MB/sec execute 137 sec latency 50.606 ms 2024-09-10T10:51:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:17 smithi103 ceph-mon[35160]: pgmap v166: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 462 op/s 2024-09-10T10:51:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:17 smithi007 ceph-mon[30873]: pgmap v166: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 462 op/s 2024-09-10T10:51:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:17 smithi133 ceph-mon[35282]: pgmap v166: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 462 op/s 2024-09-10T10:51:18.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 605256 12.48 MB/sec execute 138 sec latency 35.165 ms 2024-09-10T10:51:19.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 607772 12.49 MB/sec execute 139 sec latency 13.021 ms 2024-09-10T10:51:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:19 smithi103 ceph-mon[35160]: pgmap v167: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 258 op/s 2024-09-10T10:51:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:19 smithi007 ceph-mon[30873]: pgmap v167: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 258 op/s 2024-09-10T10:51:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:19 smithi133 ceph-mon[35282]: pgmap v167: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 258 op/s 2024-09-10T10:51:20.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 610703 12.51 MB/sec execute 140 sec latency 12.579 ms 2024-09-10T10:51:21.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 612400 12.49 MB/sec execute 141 sec latency 19.733 ms 2024-09-10T10:51:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:21 smithi103 ceph-mon[35160]: pgmap v168: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 441 op/s 2024-09-10T10:51:21.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:21 smithi007 ceph-mon[30873]: pgmap v168: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 441 op/s 2024-09-10T10:51:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:21 smithi133 ceph-mon[35282]: pgmap v168: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 441 op/s 2024-09-10T10:51:22.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 614678 12.50 MB/sec execute 142 sec latency 77.003 ms 2024-09-10T10:51:23.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 616433 12.45 MB/sec execute 143 sec latency 33.212 ms 2024-09-10T10:51:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:23 smithi103 ceph-mon[35160]: pgmap v169: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-10T10:51:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:23 smithi007 ceph-mon[30873]: pgmap v169: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-10T10:51:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:23 smithi133 ceph-mon[35282]: pgmap v169: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-10T10:51:24.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 618542 12.46 MB/sec execute 144 sec latency 9.441 ms 2024-09-10T10:51:25.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 621646 12.49 MB/sec execute 145 sec latency 11.878 ms 2024-09-10T10:51:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:25 smithi103 ceph-mon[35160]: pgmap v170: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-10T10:51:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:25 smithi007 ceph-mon[30873]: pgmap v170: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-10T10:51:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:25 smithi133 ceph-mon[35282]: pgmap v170: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-10T10:51:26.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 623944 12.47 MB/sec execute 146 sec latency 12.253 ms 2024-09-10T10:51:27.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 625798 12.46 MB/sec execute 147 sec latency 45.108 ms 2024-09-10T10:51:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:27 smithi103 ceph-mon[35160]: pgmap v171: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 438 op/s 2024-09-10T10:51:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:27 smithi007 ceph-mon[30873]: pgmap v171: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 438 op/s 2024-09-10T10:51:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:27 smithi133 ceph-mon[35282]: pgmap v171: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 438 op/s 2024-09-10T10:51:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.151+0000 7f8fd8178640 1 -- 172.21.15.7:0/4111083358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd0100890 msgr2=0x7f8fd0100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.151+0000 7f8fd8178640 1 --2- 172.21.15.7:0/4111083358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd0100890 0x7f8fd0100c70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009900 tx=0x7f8fcc02a410 comp rx=0 tx=0).stop 2024-09-10T10:51:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.151+0000 7f8fd8178640 1 -- 172.21.15.7:0/4111083358 shutdown_connections 2024-09-10T10:51:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.151+0000 7f8fd8178640 1 --2- 172.21.15.7:0/4111083358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd010e440 0x7f8fd0110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.151+0000 7f8fd8178640 1 --2- 172.21.15.7:0/4111083358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd0101240 0x7f8fd010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.151+0000 7f8fd8178640 1 --2- 172.21.15.7:0/4111083358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd0100890 0x7f8fd0100c70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.151+0000 7f8fd8178640 1 -- 172.21.15.7:0/4111083358 >> 172.21.15.7:0/4111083358 conn(0x7f8fd0077990 msgr2=0x7f8fd00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.151+0000 7f8fd8178640 1 -- 172.21.15.7:0/4111083358 shutdown_connections 2024-09-10T10:51:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.151+0000 7f8fd8178640 1 -- 172.21.15.7:0/4111083358 wait complete. 2024-09-10T10:51:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd8178640 1 Processor -- start 2024-09-10T10:51:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd8178640 1 -- start start 2024-09-10T10:51:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd8178640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd0100890 0x7f8fd019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd8178640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd0101240 0x7f8fd019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd8178640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd010e440 0x7f8fd019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd8178640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0114100 con 0x7f8fd0101240 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd8178640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0113f80 con 0x7f8fd0100890 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd8178640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0114280 con 0x7f8fd010e440 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd5713640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd0101240 0x7f8fd019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd5713640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd0101240 0x7f8fd019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33410/0 (socket says 172.21.15.7:33410) 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd5713640 1 -- 172.21.15.7:0/434959885 learned_addr learned my addr 172.21.15.7:0/434959885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd6715640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd010e440 0x7f8fd019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd0100890 0x7f8fd019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd5713640 1 -- 172.21.15.7:0/434959885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd010e440 msgr2=0x7f8fd019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd5713640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd010e440 0x7f8fd019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd5713640 1 -- 172.21.15.7:0/434959885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd0100890 msgr2=0x7f8fd019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd5713640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd0100890 0x7f8fd019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd5713640 1 -- 172.21.15.7:0/434959885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd019f5c0 con 0x7f8fd0101240 2024-09-10T10:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.152+0000 7f8fd6715640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd010e440 0x7f8fd019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.153+0000 7f8fd5713640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd0101240 0x7f8fd019ab10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000ea60 tx=0x7f8fc000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.153+0000 7f8fc6ffd640 1 -- 172.21.15.7:0/434959885 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000cde0 con 0x7f8fd0101240 2024-09-10T10:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.153+0000 7f8fc6ffd640 1 -- 172.21.15.7:0/434959885 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc00035f0 con 0x7f8fd0101240 2024-09-10T10:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.153+0000 7f8fd8178640 1 -- 172.21.15.7:0/434959885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd019f8b0 con 0x7f8fd0101240 2024-09-10T10:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.153+0000 7f8fc6ffd640 1 -- 172.21.15.7:0/434959885 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0010670 con 0x7f8fd0101240 2024-09-10T10:51:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.153+0000 7f8fd8178640 1 -- 172.21.15.7:0/434959885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a7140 con 0x7f8fd0101240 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fdaff2740 1 Processor -- start 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fc6ffd640 1 -- 172.21.15.7:0/434959885 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8fc00037a0 con 0x7f8fd0101240 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fdaff2740 1 -- start start 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab7af0 0x556356ab7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab8410 0x556356ac2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ac2cb0 0x556356ac50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556356a1c2f0 con 0x556356ab8410 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563569c4270 con 0x556356ac2cb0 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556356a182b0 con 0x556356ab7af0 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fd5f14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab8410 0x556356ac2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fd6715640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab7af0 0x556356ab7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fd5f14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab8410 0x556356ac2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33426/0 (socket says 172.21.15.7:33426) 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fd5f14640 1 -- 172.21.15.7:0/3552180770 learned_addr learned my addr 172.21.15.7:0/3552180770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fd6715640 1 --2- 172.21.15.7:0/3552180770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab7af0 0x556356ab7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:46530/0 (socket says 172.21.15.7:46530) 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.156+0000 7f8fd5713640 1 --2- 172.21.15.7:0/3552180770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ac2cb0 0x556356ac50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fd5f14640 1 -- 172.21.15.7:0/3552180770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab7af0 msgr2=0x556356ab7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/3552180770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab7af0 0x556356ab7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fd5f14640 1 -- 172.21.15.7:0/3552180770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ac2cb0 msgr2=0x556356ac50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/3552180770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ac2cb0 0x556356ac50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fd5f14640 1 -- 172.21.15.7:0/3552180770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563569c3340 con 0x556356ab8410 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fc6ffd640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8fbc0a2da0 0x7f8fbc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fd6715640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8fbc0a2da0 0x7f8fbc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/3552180770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab8410 0x556356ac2770 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009900 tx=0x7f8fcc005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca0f4e7e3db4b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fc6ffd640 1 -- 172.21.15.7:0/434959885 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f8fc00cf700 con 0x7f8fd0101240 2024-09-10T10:51:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fbbfff640 1 -- 172.21.15.7:0/3552180770 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc035040 con 0x556356ab8410 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fbbfff640 1 -- 172.21.15.7:0/3552180770 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fcc004400 con 0x556356ab8410 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.157+0000 7f8fbbfff640 1 -- 172.21.15.7:0/3552180770 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0035f0 con 0x556356ab8410 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fdaff2740 1 -- 172.21.15.7:0/3552180770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab8410 msgr2=0x556356ac2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/3552180770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab8410 0x556356ac2770 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009900 tx=0x7f8fcc005de0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fd6715640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8fbc0a2da0 0x7f8fbc0a5260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009fd0 tx=0x7f8fc8009500 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fdaff2740 1 -- 172.21.15.7:0/3552180770 shutdown_connections 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/3552180770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ac2cb0 0x556356ac50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/3552180770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab8410 0x556356ac2770 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/3552180770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab7af0 0x556356ab7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fdaff2740 1 -- 172.21.15.7:0/3552180770 >> 172.21.15.7:0/3552180770 conn(0x556356ab4060 msgr2=0x556356ab4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fdaff2740 1 -- 172.21.15.7:0/3552180770 shutdown_connections 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fdaff2740 1 -- 172.21.15.7:0/3552180770 wait complete. 2024-09-10T10:51:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.158+0000 7f8fdaff2740 1 -- start start 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fdaff2740 1 Processor -- start 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356abfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 0x556356ac00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 0x556356ac0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556356ac5880 con 0x556356ac2cb0 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563569a5290 con 0x556356ab7af0 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556356ac5a00 con 0x556356ab8410 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fdaff2740 10 client.0 _ll_register_callbacks cb 0x5563569d6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fd5713640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 0x556356ac00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fd5f14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356abfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fd5713640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 0x556356ac00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:46534/0 (socket says 172.21.15.7:46534) 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fd6715640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 0x556356ac0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.159+0000 7f8fd5713640 1 -- 172.21.15.7:0/1979150922 learned_addr learned my addr 172.21.15.7:0/1979150922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fd5713640 1 -- 172.21.15.7:0/1979150922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 msgr2=0x556356abfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fd5713640 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356abfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fd5713640 1 -- 172.21.15.7:0/1979150922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 msgr2=0x556356ac0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fd5713640 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 0x556356ac0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fd5713640 1 -- 172.21.15.7:0/1979150922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55635672f210 con 0x556356ab8410 2024-09-10T10:51:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356abfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fd5713640 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 0x556356ac00d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c760 tx=0x7f8fc0009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e35e1ef4e313e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:51:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fb97fa640 1 -- 172.21.15.7:0/1979150922 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc009b760 con 0x556356ab8410 2024-09-10T10:51:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fb97fa640 1 -- 172.21.15.7:0/1979150922 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f8fc0098290 con 0x556356ab8410 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fdaff2740 1 -- 172.21.15.7:0/1979150922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fcc004400 con 0x556356ab8410 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fb97fa640 1 -- 172.21.15.7:0/1979150922 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8fc009b580 con 0x556356ab8410 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fdaff2740 10 client.25375 fetch_fsmap learned FSMap version 32 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fdaff2740 1 -- 172.21.15.7:0/1979150922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5563567fb620 con 0x556356ab8410 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fb97fa640 1 -- 172.21.15.7:0/1979150922 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fcc004400 con 0x556356ab8410 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fb97fa640 1 -- 172.21.15.7:0/1979150922 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8fc00b4050 con 0x556356ab8410 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.160+0000 7f8fdaff2740 10 client.25375 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.161+0000 7f8fdaff2740 10 client.25375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.161+0000 7f8fdaff2740 20 client.25375 populate_metadata read hostname 'smithi007' 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.161+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556356ad2ba0 0x556356ad4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.161+0000 7f8fdaff2740 4 client.25375 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:28.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.161+0000 7f8fdaff2740 1 -- 172.21.15.7:0/1979150922 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556356a29a20 con 0x556356ad2ba0 2024-09-10T10:51:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.161+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556356ad2ba0 0x556356ad4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.161+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556356ad2ba0 0x556356ad4f90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c74aaa46d0bdfe4 server_cookie=9f0b4fedc32e537b in_seq=0 out_seq=0 2024-09-10T10:51:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.161+0000 7f8fb97fa640 10 client.25375 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fb97fa640 1 -- 172.21.15.7:0/1979150922 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556356a29a20 con 0x556356ad2ba0 2024-09-10T10:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fb97fa640 10 client.25375 handle_command_reply: tid=0 2024-09-10T10:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 client.25375 shutdown 2024-09-10T10:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 -- 172.21.15.7:0/1979150922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 msgr2=0x556356ac00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 0x556356ac00d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c760 tx=0x7f8fc0009550 comp rx=0 tx=0).stop 2024-09-10T10:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 -- 172.21.15.7:0/1979150922 shutdown_connections 2024-09-10T10:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556356ad2ba0 0x556356ad4f90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 0x556356ac0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fb97fa640 0 client.25375 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 0x556356ac00d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/1979150922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356abfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 -- 172.21.15.7:0/1979150922 >> 172.21.15.7:0/1979150922 conn(0x556356ab4060 msgr2=0x556356ab9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 -- 172.21.15.7:0/1979150922 shutdown_connections 2024-09-10T10:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 1 -- 172.21.15.7:0/1979150922 wait complete. 2024-09-10T10:51:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.163+0000 7f8fdaff2740 20 client.25375 trim_cache size 0 max 16384 2024-09-10T10:51:28.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.178+0000 7f8fdaff2740 1 Processor -- start 2024-09-10T10:51:28.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fdaff2740 1 -- start start 2024-09-10T10:51:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356aba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 0x556356ac0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 0x556356ada860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01b2280 con 0x556356ac2cb0 2024-09-10T10:51:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01afb50 con 0x556356ab7af0 2024-09-10T10:51:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01ad9a0 con 0x556356ab8410 2024-09-10T10:51:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd5f14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356aba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd6715640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 0x556356ada860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd5f14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356aba560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:53264/0 (socket says 172.21.15.7:53264) 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd5713640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 0x556356ac0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd5f14640 1 -- 172.21.15.7:0/3343149175 learned_addr learned my addr 172.21.15.7:0/3343149175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd6715640 1 -- 172.21.15.7:0/3343149175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 msgr2=0x556356ac0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd6715640 1 --2- 172.21.15.7:0/3343149175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 0x556356ac0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd6715640 1 -- 172.21.15.7:0/3343149175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 msgr2=0x556356aba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd6715640 1 --2- 172.21.15.7:0/3343149175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356aba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd6715640 1 -- 172.21.15.7:0/3343149175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556356ab8240 con 0x556356ac2cb0 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/3343149175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356aba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.179+0000 7f8fd6715640 1 --2- 172.21.15.7:0/3343149175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 0x556356ada860 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8013010 tx=0x7f8fc801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f37afd8652931a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.180+0000 7f8fbb7fe640 1 -- 172.21.15.7:0/3343149175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8003d60 con 0x556356ac2cb0 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.180+0000 7f8fbb7fe640 1 -- 172.21.15.7:0/3343149175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8004590 con 0x556356ac2cb0 2024-09-10T10:51:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.180+0000 7f8fbb7fe640 1 -- 172.21.15.7:0/3343149175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8005730 con 0x556356ac2cb0 2024-09-10T10:51:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.180+0000 7f8fdaff2740 1 -- 172.21.15.7:0/3343149175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 msgr2=0x556356ada860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.180+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/3343149175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 0x556356ada860 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8013010 tx=0x7f8fc801b000 comp rx=0 tx=0).stop 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 -- 172.21.15.7:0/3343149175 shutdown_connections 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/3343149175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ac2cb0 0x556356ada860 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/3343149175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ab8410 0x556356ac0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/3343149175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab7af0 0x556356aba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 -- 172.21.15.7:0/3343149175 >> 172.21.15.7:0/3343149175 conn(0x556356ab4060 msgr2=0x556356ac1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 -- 172.21.15.7:0/3343149175 shutdown_connections 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 -- 172.21.15.7:0/3343149175 wait complete. 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 -- start start 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 Processor -- start 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab7af0 0x556356acaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab8410 0x556356acb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ac2cb0 0x556356ac63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01ae010 con 0x556356ab7af0 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01af480 con 0x556356ab8410 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01ae6e0 con 0x556356ac2cb0 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.181+0000 7f8fdaff2740 10 client.0 _ll_register_callbacks cb 0x7f8fc000e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd5f14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab7af0 0x556356acaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd5f14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab7af0 0x556356acaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33434/0 (socket says 172.21.15.7:33434) 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd5f14640 1 -- 172.21.15.7:0/403414884 learned_addr learned my addr 172.21.15.7:0/403414884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd6715640 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ac2cb0 0x556356ac63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd5713640 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab8410 0x556356acb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd5f14640 1 -- 172.21.15.7:0/403414884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ac2cb0 msgr2=0x556356ac63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ac2cb0 0x556356ac63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd5f14640 1 -- 172.21.15.7:0/403414884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab8410 msgr2=0x556356acb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab8410 0x556356acb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd5f14640 1 -- 172.21.15.7:0/403414884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556356ac6bd0 con 0x556356ab7af0 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fd5f14640 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab7af0 0x556356acaf80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc031550 tx=0x7f8fcc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34aef9434422515a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fb97fa640 1 -- 172.21.15.7:0/403414884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc035070 con 0x556356ab7af0 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fb97fa640 1 -- 172.21.15.7:0/403414884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f8fcc0317b0 con 0x556356ab7af0 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fdaff2740 1 -- 172.21.15.7:0/403414884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc8004590 con 0x556356ab7af0 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fb97fa640 1 -- 172.21.15.7:0/403414884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8fcc0044a0 con 0x556356ab7af0 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fdaff2740 10 client.15693 fetch_fsmap learned FSMap version 32 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.182+0000 7f8fdaff2740 1 -- 172.21.15.7:0/403414884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556356b3d440 con 0x556356ab7af0 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fb97fa640 1 -- 172.21.15.7:0/403414884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8004590 con 0x556356ab7af0 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fb97fa640 1 -- 172.21.15.7:0/403414884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f8fcc003bf0 con 0x556356ab7af0 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fdaff2740 10 client.15693 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:51:28.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fdaff2740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fdaff2740 20 client.15693 populate_metadata read hostname 'smithi007' 2024-09-10T10:51:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556356b3d7a0 0x556356b3fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fdaff2740 4 client.15693 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:51:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fdaff2740 1 -- 172.21.15.7:0/403414884 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556356a23d70 con 0x556356b3d7a0 2024-09-10T10:51:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fd5713640 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556356b3d7a0 0x556356b3fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fb97fa640 10 client.15693 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:28.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.183+0000 7f8fd5713640 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556356b3d7a0 0x556356b3fb90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7eacb084fc7da65 server_cookie=181c3a9bea271f7a in_seq=0 out_seq=0 2024-09-10T10:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fb97fa640 1 -- 172.21.15.7:0/403414884 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556356a23d70 con 0x556356b3d7a0 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fb97fa640 10 client.15693 handle_command_reply: tid=0 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 client.15693 shutdown 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 -- 172.21.15.7:0/403414884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab7af0 msgr2=0x556356acaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab7af0 0x556356acaf80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc031550 tx=0x7f8fcc004ab0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 -- 172.21.15.7:0/403414884 shutdown_connections 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556356b3d7a0 0x556356b3fb90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556356ac2cb0 0x556356ac63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556356ab8410 0x556356acb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fb97fa640 0 client.15693 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 --2- 172.21.15.7:0/403414884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556356ab7af0 0x556356acaf80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 -- 172.21.15.7:0/403414884 >> 172.21.15.7:0/403414884 conn(0x556356ab4060 msgr2=0x556356ab29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 -- 172.21.15.7:0/403414884 shutdown_connections 2024-09-10T10:51:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 1 -- 172.21.15.7:0/403414884 wait complete. 2024-09-10T10:51:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.267+0000 7f8fdaff2740 20 client.15693 trim_cache size 0 max 16384 2024-09-10T10:51:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:51:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:51:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:51:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 -- 172.21.15.7:0/434959885 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8fbc0a2da0 msgr2=0x7f8fbc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8fbc0a2da0 0x7f8fbc0a5260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009fd0 tx=0x7f8fc8009500 comp rx=0 tx=0).stop 2024-09-10T10:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 -- 172.21.15.7:0/434959885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd0101240 msgr2=0x7f8fd019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd0101240 0x7f8fd019ab10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000ea60 tx=0x7f8fc000ef20 comp rx=0 tx=0).stop 2024-09-10T10:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 -- 172.21.15.7:0/434959885 shutdown_connections 2024-09-10T10:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fd010e440 0x7f8fd019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8fbc0a2da0 0x7f8fbc0a5260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fd0101240 0x7f8fd019ab10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 --2- 172.21.15.7:0/434959885 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8fd0100890 0x7f8fd019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 -- 172.21.15.7:0/434959885 >> 172.21.15.7:0/434959885 conn(0x7f8fd0077990 msgr2=0x7f8fd00ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 -- 172.21.15.7:0/434959885 shutdown_connections 2024-09-10T10:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:28.268+0000 7f8fd8178640 1 -- 172.21.15.7:0/434959885 wait complete. 2024-09-10T10:51:28.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 628563 12.50 MB/sec execute 148 sec latency 45.120 ms 2024-09-10T10:51:29.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 630425 12.46 MB/sec execute 149 sec latency 13.513 ms 2024-09-10T10:51:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:29 smithi007 ceph-mon[30873]: pgmap v172: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-10T10:51:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:29 smithi133 ceph-mon[35282]: pgmap v172: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-10T10:51:30.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:29 smithi103 ceph-mon[35160]: pgmap v172: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-09-10T10:51:30.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 632745 12.47 MB/sec execute 150 sec latency 12.950 ms 2024-09-10T10:51:31.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 635841 12.50 MB/sec execute 151 sec latency 14.457 ms 2024-09-10T10:51:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:31 smithi007 ceph-mon[30873]: pgmap v173: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 495 op/s 2024-09-10T10:51:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:31 smithi133 ceph-mon[35282]: pgmap v173: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 495 op/s 2024-09-10T10:51:32.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:31 smithi103 ceph-mon[35160]: pgmap v173: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 495 op/s 2024-09-10T10:51:32.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 637705 12.47 MB/sec execute 152 sec latency 32.452 ms 2024-09-10T10:51:33.514 INFO:tasks.workunit.client.0.smithi007.stdout: 1 639831 12.47 MB/sec execute 153 sec latency 14.361 ms 2024-09-10T10:51:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:33 smithi007 ceph-mon[30873]: pgmap v174: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-09-10T10:51:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:33 smithi133 ceph-mon[35282]: pgmap v174: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-09-10T10:51:34.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:33 smithi103 ceph-mon[35160]: pgmap v174: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-09-10T10:51:34.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 642808 12.51 MB/sec execute 154 sec latency 18.517 ms 2024-09-10T10:51:35.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 644715 12.47 MB/sec execute 155 sec latency 15.837 ms 2024-09-10T10:51:35.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:35 smithi007 ceph-mon[30873]: pgmap v175: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-09-10T10:51:35.973 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:35 smithi103 ceph-mon[35160]: pgmap v175: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-09-10T10:51:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:35 smithi133 ceph-mon[35282]: pgmap v175: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 312 op/s 2024-09-10T10:51:36.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 646815 12.48 MB/sec execute 156 sec latency 54.664 ms 2024-09-10T10:51:37.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 649516 12.48 MB/sec execute 157 sec latency 37.446 ms 2024-09-10T10:51:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:37 smithi007 ceph-mon[30873]: pgmap v176: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 443 op/s 2024-09-10T10:51:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:37 smithi133 ceph-mon[35282]: pgmap v176: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 443 op/s 2024-09-10T10:51:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:37 smithi103 ceph-mon[35160]: pgmap v176: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 443 op/s 2024-09-10T10:51:38.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 651294 12.47 MB/sec execute 158 sec latency 23.796 ms 2024-09-10T10:51:39.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 653820 12.48 MB/sec execute 159 sec latency 13.158 ms 2024-09-10T10:51:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:39 smithi133 ceph-mon[35282]: pgmap v177: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2024-09-10T10:51:40.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:39 smithi103 ceph-mon[35160]: pgmap v177: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2024-09-10T10:51:40.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:39 smithi007 ceph-mon[30873]: pgmap v177: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2024-09-10T10:51:40.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 656443 12.48 MB/sec execute 160 sec latency 13.945 ms 2024-09-10T10:51:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 -- 172.21.15.7:0/2961043838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c4102c80 msgr2=0x7f77c4103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 --2- 172.21.15.7:0/2961043838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c4102c80 0x7f77c4103060 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009ca0 tx=0x7f77b402a950 comp rx=0 tx=0).stop 2024-09-10T10:51:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 -- 172.21.15.7:0/2961043838 shutdown_connections 2024-09-10T10:51:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 --2- 172.21.15.7:0/2961043838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77c410c0d0 0x7f77c410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 --2- 172.21.15.7:0/2961043838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c41035a0 0x7f77c410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 --2- 172.21.15.7:0/2961043838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c4102c80 0x7f77c4103060 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 -- 172.21.15.7:0/2961043838 >> 172.21.15.7:0/2961043838 conn(0x7f77c4077990 msgr2=0x7f77c40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 -- 172.21.15.7:0/2961043838 shutdown_connections 2024-09-10T10:51:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 -- 172.21.15.7:0/2961043838 wait complete. 2024-09-10T10:51:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 Processor -- start 2024-09-10T10:51:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.361+0000 7f77cb5c4640 1 -- start start 2024-09-10T10:51:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77cb5c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77c4102c80 0x7f77c4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77cb5c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c41035a0 0x7f77c4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77cb5c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c410c0d0 0x7f77c419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77cb5c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4111eb0 con 0x7f77c41035a0 2024-09-10T10:51:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77cb5c4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4111d30 con 0x7f77c4102c80 2024-09-10T10:51:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77cb5c4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4112030 con 0x7f77c410c0d0 2024-09-10T10:51:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c9360640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77c4102c80 0x7f77c4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c8b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c41035a0 0x7f77c4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c9360640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77c4102c80 0x7f77c4198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:54000/0 (socket says 172.21.15.7:54000) 2024-09-10T10:51:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c9360640 1 -- 172.21.15.7:0/1432657782 learned_addr learned my addr 172.21.15.7:0/1432657782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c9b61640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c410c0d0 0x7f77c419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c9360640 1 -- 172.21.15.7:0/1432657782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c410c0d0 msgr2=0x7f77c419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c9360640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c410c0d0 0x7f77c419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c9360640 1 -- 172.21.15.7:0/1432657782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c41035a0 msgr2=0x7f77c4198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c9360640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c41035a0 0x7f77c4198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c9360640 1 -- 172.21.15.7:0/1432657782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c419d330 con 0x7f77c4102c80 2024-09-10T10:51:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c8b5f640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c41035a0 0x7f77c4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c9360640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77c4102c80 0x7f77c4198340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f77b4005cc0 tx=0x7f77b402f630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.362+0000 7f77c27fc640 1 -- 172.21.15.7:0/1432657782 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4034040 con 0x7f77c4102c80 2024-09-10T10:51:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.363+0000 7f77cb5c4640 1 -- 172.21.15.7:0/1432657782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c419d5c0 con 0x7f77c4102c80 2024-09-10T10:51:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.363+0000 7f77c27fc640 1 -- 172.21.15.7:0/1432657782 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b4030920 con 0x7f77c4102c80 2024-09-10T10:51:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.363+0000 7f77c27fc640 1 -- 172.21.15.7:0/1432657782 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4004780 con 0x7f77c4102c80 2024-09-10T10:51:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.363+0000 7f77cb5c4640 1 -- 172.21.15.7:0/1432657782 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c41a4eb0 con 0x7f77c4102c80 2024-09-10T10:51:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.365+0000 7f77c27fc640 1 -- 172.21.15.7:0/1432657782 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f77b4037020 con 0x7f77c4102c80 2024-09-10T10:51:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.365+0000 7f77ce43c740 1 Processor -- start 2024-09-10T10:51:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.365+0000 7f77ce43c740 1 -- start start 2024-09-10T10:51:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.365+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.365+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 0x563a6a9c1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.365+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.365+0000 7f77ce43c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563a6a91b2f0 con 0x563a6a9b6af0 2024-09-10T10:51:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.365+0000 7f77ce43c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563a6a8c3270 con 0x563a6a9b7410 2024-09-10T10:51:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.365+0000 7f77ce43c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563a6a9172b0 con 0x563a6a9c1cb0 2024-09-10T10:51:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9b61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9360640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 0x563a6a9c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9b61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49494/0 (socket says 172.21.15.7:49494) 2024-09-10T10:51:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c8b5f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9b61640 1 -- 172.21.15.7:0/3561092505 learned_addr learned my addr 172.21.15.7:0/3561092505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9b61640 1 -- 172.21.15.7:0/3561092505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 msgr2=0x563a6a9c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9b61640 1 --2- 172.21.15.7:0/3561092505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9b61640 1 -- 172.21.15.7:0/3561092505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 msgr2=0x563a6a9c1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9b61640 1 --2- 172.21.15.7:0/3561092505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 0x563a6a9c1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9b61640 1 -- 172.21.15.7:0/3561092505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6a8c2340 con 0x563a6a9b6af0 2024-09-10T10:51:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c27fc640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77ac0a2da0 0x7f77ac0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9360640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77ac0a2da0 0x7f77ac0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c9b61640 1 --2- 172.21.15.7:0/3561092505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b6ed0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009c20 tx=0x7f77bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d9c62f23601b3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77c27fc640 1 -- 172.21.15.7:0/1432657782 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f77b40f01e0 con 0x7f77c4102c80 2024-09-10T10:51:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77ab7fe640 1 -- 172.21.15.7:0/3561092505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc01e070 con 0x563a6a9b6af0 2024-09-10T10:51:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77ab7fe640 1 -- 172.21.15.7:0/3561092505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77bc019b10 con 0x563a6a9b6af0 2024-09-10T10:51:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.366+0000 7f77ab7fe640 1 -- 172.21.15.7:0/3561092505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc00ea80 con 0x563a6a9b6af0 2024-09-10T10:51:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77ce43c740 1 -- 172.21.15.7:0/3561092505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 msgr2=0x563a6a9b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77c9360640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77ac0a2da0 0x7f77ac0a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x563a6a9b3ec0 tx=0x7f77b40a7040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77ce43c740 1 --2- 172.21.15.7:0/3561092505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b6ed0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009c20 tx=0x7f77bc011810 comp rx=0 tx=0).stop 2024-09-10T10:51:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77ce43c740 1 -- 172.21.15.7:0/3561092505 shutdown_connections 2024-09-10T10:51:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77ce43c740 1 --2- 172.21.15.7:0/3561092505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77ce43c740 1 --2- 172.21.15.7:0/3561092505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 0x563a6a9c1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77ce43c740 1 --2- 172.21.15.7:0/3561092505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b6ed0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77ce43c740 1 -- 172.21.15.7:0/3561092505 >> 172.21.15.7:0/3561092505 conn(0x563a6a9b3060 msgr2=0x563a6a9b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77ce43c740 1 -- 172.21.15.7:0/3561092505 shutdown_connections 2024-09-10T10:51:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77ce43c740 1 -- 172.21.15.7:0/3561092505 wait complete. 2024-09-10T10:51:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.367+0000 7f77ce43c740 1 -- start start 2024-09-10T10:51:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77ce43c740 1 Processor -- start 2024-09-10T10:51:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 0x563a6a9bebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9bf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77ce43c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563a6a9c4880 con 0x563a6a9b7410 2024-09-10T10:51:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77ce43c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563a6a8a4290 con 0x563a6a9b6af0 2024-09-10T10:51:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77ce43c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563a6a9c4a00 con 0x563a6a9c1cb0 2024-09-10T10:51:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77ce43c740 10 client.0 _ll_register_callbacks cb 0x563a6a8d5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77c8b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9bf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77c8b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9bf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49508/0 (socket says 172.21.15.7:49508) 2024-09-10T10:51:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77c8b5f640 1 -- 172.21.15.7:0/1110587973 learned_addr learned my addr 172.21.15.7:0/1110587973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77c9b61640 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9bf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77c9b61640 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 0x563a6a9bebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77c8b5f640 1 -- 172.21.15.7:0/1110587973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 msgr2=0x563a6a9bf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77c8b5f640 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9bf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77c8b5f640 1 -- 172.21.15.7:0/1110587973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 msgr2=0x563a6a9bebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77c8b5f640 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 0x563a6a9bebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.368+0000 7f77c8b5f640 1 -- 172.21.15.7:0/1110587973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6a62e210 con 0x563a6a9b7410 2024-09-10T10:51:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77c8b5f640 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9bf100 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f77b80116e0 tx=0x7f77b8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1f0739f5ad2217 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:51:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1110587973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8010a50 con 0x563a6a9b7410 2024-09-10T10:51:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1110587973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f77b8010bf0 con 0x563a6a9b7410 2024-09-10T10:51:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77ce43c740 1 -- 172.21.15.7:0/1110587973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77bc019b10 con 0x563a6a9b7410 2024-09-10T10:51:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1110587973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f77b8010df0 con 0x563a6a9b7410 2024-09-10T10:51:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77ce43c740 10 client.15705 fetch_fsmap learned FSMap version 32 2024-09-10T10:51:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77ce43c740 1 -- 172.21.15.7:0/1110587973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a6a6fa620 con 0x563a6a9b7410 2024-09-10T10:51:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1110587973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77bc019b10 con 0x563a6a9b7410 2024-09-10T10:51:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1110587973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f77b800f560 con 0x563a6a9b7410 2024-09-10T10:51:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77ce43c740 10 client.15705 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:51:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77ce43c740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77ce43c740 20 client.15705 populate_metadata read hostname 'smithi007' 2024-09-10T10:51:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563a6a9d1c50 0x563a6a9d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77ce43c740 4 client.15705 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.369+0000 7f77ce43c740 1 -- 172.21.15.7:0/1110587973 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a6a928a20 con 0x563a6a9d1c50 2024-09-10T10:51:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.370+0000 7f77c9b61640 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563a6a9d1c50 0x563a6a9d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.370+0000 7f77c9b61640 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563a6a9d1c50 0x563a6a9d4080 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d457d16651b0875 server_cookie=8f66198432944aca in_seq=0 out_seq=0 2024-09-10T10:51:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.370+0000 7f77a8ff9640 10 client.15705 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1110587973 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a6a928a20 con 0x563a6a9d1c50 2024-09-10T10:51:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77a8ff9640 10 client.15705 handle_command_reply: tid=0 2024-09-10T10:51:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77ce43c740 1 client.15705 shutdown 2024-09-10T10:51:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77ce43c740 1 -- 172.21.15.7:0/1110587973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 msgr2=0x563a6a9bf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9bf100 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f77b80116e0 tx=0x7f77b8011ba0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77ce43c740 1 -- 172.21.15.7:0/1110587973 shutdown_connections 2024-09-10T10:51:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563a6a9d1c50 0x563a6a9d4080 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9bf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9bf100 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77a8ff9640 0 client.15705 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1110587973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 0x563a6a9bebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.372+0000 7f77ce43c740 1 -- 172.21.15.7:0/1110587973 >> 172.21.15.7:0/1110587973 conn(0x563a6a9b3060 msgr2=0x563a6a9b8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.373+0000 7f77ce43c740 1 -- 172.21.15.7:0/1110587973 shutdown_connections 2024-09-10T10:51:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.373+0000 7f77ce43c740 1 -- 172.21.15.7:0/1110587973 wait complete. 2024-09-10T10:51:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.373+0000 7f77ce43c740 20 client.15705 trim_cache size 0 max 16384 2024-09-10T10:51:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77ce43c740 1 Processor -- start 2024-09-10T10:51:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77ce43c740 1 -- start start 2024-09-10T10:51:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 0x563a6a9bf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9d9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77ce43c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b0070 con 0x563a6a9b6af0 2024-09-10T10:51:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77ce43c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41ad940 con 0x563a6a9b7410 2024-09-10T10:51:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77ce43c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41ab790 con 0x563a6a9c1cb0 2024-09-10T10:51:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c9b61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c9b61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49520/0 (socket says 172.21.15.7:49520) 2024-09-10T10:51:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c9b61640 1 -- 172.21.15.7:0/3930998960 learned_addr learned my addr 172.21.15.7:0/3930998960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c9b61640 1 --2- 172.21.15.7:0/3930998960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9d9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c8b5f640 1 --2- 172.21.15.7:0/3930998960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 0x563a6a9bf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c9b61640 1 -- 172.21.15.7:0/3930998960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 msgr2=0x563a6a9d9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c9b61640 1 --2- 172.21.15.7:0/3930998960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9d9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c9b61640 1 -- 172.21.15.7:0/3930998960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 msgr2=0x563a6a9bf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c9b61640 1 --2- 172.21.15.7:0/3930998960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 0x563a6a9bf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c9b61640 1 -- 172.21.15.7:0/3930998960 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a6a9b7240 con 0x563a6a9b6af0 2024-09-10T10:51:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.389+0000 7f77c8b5f640 1 --2- 172.21.15.7:0/3930998960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 0x563a6a9bf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.390+0000 7f77c9b61640 1 --2- 172.21.15.7:0/3930998960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b95b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009fd0 tx=0x7f77bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705ca46362e82ac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.390+0000 7f77aaffd640 1 -- 172.21.15.7:0/3930998960 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc01e040 con 0x563a6a9b6af0 2024-09-10T10:51:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.390+0000 7f77aaffd640 1 -- 172.21.15.7:0/3930998960 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77bc01f040 con 0x563a6a9b6af0 2024-09-10T10:51:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.390+0000 7f77aaffd640 1 -- 172.21.15.7:0/3930998960 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc003850 con 0x563a6a9b6af0 2024-09-10T10:51:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 -- 172.21.15.7:0/3930998960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 msgr2=0x563a6a9b95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 --2- 172.21.15.7:0/3930998960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b95b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009fd0 tx=0x7f77bc011810 comp rx=0 tx=0).stop 2024-09-10T10:51:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 -- 172.21.15.7:0/3930998960 shutdown_connections 2024-09-10T10:51:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 --2- 172.21.15.7:0/3930998960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9d9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 --2- 172.21.15.7:0/3930998960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b7410 0x563a6a9bf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 --2- 172.21.15.7:0/3930998960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b6af0 0x563a6a9b95b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 -- 172.21.15.7:0/3930998960 >> 172.21.15.7:0/3930998960 conn(0x563a6a9b3060 msgr2=0x563a6a9c0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 -- 172.21.15.7:0/3930998960 shutdown_connections 2024-09-10T10:51:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 -- 172.21.15.7:0/3930998960 wait complete. 2024-09-10T10:51:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 -- start start 2024-09-10T10:51:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 Processor -- start 2024-09-10T10:51:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 0x563a6a9ca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.391+0000 7f77ce43c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77ce43c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41abe00 con 0x563a6a9b7410 2024-09-10T10:51:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77ce43c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41ad270 con 0x563a6a9b6af0 2024-09-10T10:51:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77ce43c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41ac4d0 con 0x563a6a9c1cb0 2024-09-10T10:51:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77ce43c740 10 client.0 _ll_register_callbacks cb 0x7f77b800bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c8b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c9b61640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 0x563a6a9ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c8b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49536/0 (socket says 172.21.15.7:49536) 2024-09-10T10:51:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c9b61640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 0x563a6a9ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:54026/0 (socket says 172.21.15.7:54026) 2024-09-10T10:51:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c8b5f640 1 -- 172.21.15.7:0/1428150726 learned_addr learned my addr 172.21.15.7:0/1428150726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c9b61640 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c8b5f640 1 -- 172.21.15.7:0/1428150726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 msgr2=0x563a6a9c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c8b5f640 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c8b5f640 1 -- 172.21.15.7:0/1428150726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 msgr2=0x563a6a9ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c8b5f640 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 0x563a6a9ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c8b5f640 1 -- 172.21.15.7:0/1428150726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6aa3fd70 con 0x563a6a9b7410 2024-09-10T10:51:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c9b61640 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 0x563a6a9ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77c8b5f640 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9c5880 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000c00 tx=0x7f77b8005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f76cfb944cfe1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:51:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1428150726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8000e60 con 0x563a6a9b7410 2024-09-10T10:51:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1428150726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7f77b800f3c0 con 0x563a6a9b7410 2024-09-10T10:51:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77ce43c740 1 -- 172.21.15.7:0/1428150726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77bc01f040 con 0x563a6a9b7410 2024-09-10T10:51:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1428150726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f77b800fe00 con 0x563a6a9b7410 2024-09-10T10:51:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77ce43c740 10 client.15711 fetch_fsmap learned FSMap version 32 2024-09-10T10:51:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.392+0000 7f77ce43c740 1 -- 172.21.15.7:0/1428150726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a6aa40e70 con 0x563a6a9b7410 2024-09-10T10:51:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1428150726 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77bc01f040 con 0x563a6a9b7410 2024-09-10T10:51:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1428150726 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f77b80053b0 con 0x563a6a9b7410 2024-09-10T10:51:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77ce43c740 10 client.15711 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:51:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77ce43c740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77ce43c740 20 client.15711 populate_metadata read hostname 'smithi007' 2024-09-10T10:51:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563a6aa415f0 0x563a6aa43a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77ce43c740 4 client.15711 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:51:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77ce43c740 1 -- 172.21.15.7:0/1428150726 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a6a922d70 con 0x563a6aa415f0 2024-09-10T10:51:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77c9b61640 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563a6aa415f0 0x563a6aa43a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77c9b61640 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563a6aa415f0 0x563a6aa43a10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8793787fa74c69f7 server_cookie=22e9869d97642923 in_seq=0 out_seq=0 2024-09-10T10:51:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.393+0000 7f77a8ff9640 10 client.15711 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77a8ff9640 1 -- 172.21.15.7:0/1428150726 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a6a922d70 con 0x563a6aa415f0 2024-09-10T10:51:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77a8ff9640 10 client.15711 handle_command_reply: tid=0 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77ce43c740 1 client.15711 shutdown 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77ce43c740 1 -- 172.21.15.7:0/1428150726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 msgr2=0x563a6a9c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9c5880 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f77b8000c00 tx=0x7f77b8005da0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77ce43c740 1 -- 172.21.15.7:0/1428150726 shutdown_connections 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563a6aa415f0 0x563a6aa43a10 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a6a9c1cb0 0x563a6a9c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563a6a9b7410 0x563a6a9c5880 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77a8ff9640 0 client.15711 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77ce43c740 1 --2- 172.21.15.7:0/1428150726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563a6a9b6af0 0x563a6a9ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.474+0000 7f77ce43c740 1 -- 172.21.15.7:0/1428150726 >> 172.21.15.7:0/1428150726 conn(0x563a6a9b3060 msgr2=0x563a6a9b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.475+0000 7f77ce43c740 1 -- 172.21.15.7:0/1428150726 shutdown_connections 2024-09-10T10:51:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.475+0000 7f77ce43c740 1 -- 172.21.15.7:0/1428150726 wait complete. 2024-09-10T10:51:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.475+0000 7f77ce43c740 20 client.15711 trim_cache size 0 max 16384 2024-09-10T10:51:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:51:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:51:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:51:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:51:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 -- 172.21.15.7:0/1432657782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77ac0a2da0 msgr2=0x7f77ac0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77ac0a2da0 0x7f77ac0a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x563a6a9b3ec0 tx=0x7f77b40a7040 comp rx=0 tx=0).stop 2024-09-10T10:51:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 -- 172.21.15.7:0/1432657782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77c4102c80 msgr2=0x7f77c4198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77c4102c80 0x7f77c4198340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f77b4005cc0 tx=0x7f77b402f630 comp rx=0 tx=0).stop 2024-09-10T10:51:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 -- 172.21.15.7:0/1432657782 shutdown_connections 2024-09-10T10:51:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77c410c0d0 0x7f77c419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77ac0a2da0 0x7f77ac0a5260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c41035a0 0x7f77c4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 --2- 172.21.15.7:0/1432657782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77c4102c80 0x7f77c4198340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 -- 172.21.15.7:0/1432657782 >> 172.21.15.7:0/1432657782 conn(0x7f77c4077990 msgr2=0x7f77c40fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 -- 172.21.15.7:0/1432657782 shutdown_connections 2024-09-10T10:51:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:41.476+0000 7f77cb5c4640 1 -- 172.21.15.7:0/1432657782 wait complete. 2024-09-10T10:51:41.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 658138 12.46 MB/sec execute 161 sec latency 42.351 ms 2024-09-10T10:51:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:41 smithi133 ceph-mon[35282]: pgmap v178: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 445 op/s 2024-09-10T10:51:42.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:41 smithi103 ceph-mon[35160]: pgmap v178: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 445 op/s 2024-09-10T10:51:42.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:41 smithi007 ceph-mon[30873]: pgmap v178: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 445 op/s 2024-09-10T10:51:42.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 660739 12.49 MB/sec execute 162 sec latency 66.566 ms 2024-09-10T10:51:43.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 662445 12.45 MB/sec execute 163 sec latency 34.360 ms 2024-09-10T10:51:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:43 smithi133 ceph-mon[35282]: pgmap v179: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-09-10T10:51:44.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:43 smithi103 ceph-mon[35160]: pgmap v179: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-09-10T10:51:44.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:43 smithi007 ceph-mon[30873]: pgmap v179: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-09-10T10:51:44.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 664539 12.45 MB/sec execute 164 sec latency 11.949 ms 2024-09-10T10:51:45.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 667266 12.46 MB/sec execute 165 sec latency 24.352 ms 2024-09-10T10:51:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:45 smithi133 ceph-mon[35282]: pgmap v180: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-09-10T10:51:46.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:45 smithi103 ceph-mon[35160]: pgmap v180: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-09-10T10:51:46.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:45 smithi007 ceph-mon[30873]: pgmap v180: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-09-10T10:51:46.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 669181 12.45 MB/sec execute 166 sec latency 12.385 ms 2024-09-10T10:51:47.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 671621 12.46 MB/sec execute 167 sec latency 12.932 ms 2024-09-10T10:51:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:47 smithi133 ceph-mon[35282]: pgmap v181: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 439 op/s 2024-09-10T10:51:48.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:47 smithi103 ceph-mon[35160]: pgmap v181: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 439 op/s 2024-09-10T10:51:48.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:47 smithi007 ceph-mon[30873]: pgmap v181: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 439 op/s 2024-09-10T10:51:48.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 674198 12.46 MB/sec execute 168 sec latency 47.565 ms 2024-09-10T10:51:49.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 676191 12.45 MB/sec execute 169 sec latency 11.754 ms 2024-09-10T10:51:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:49 smithi133 ceph-mon[35282]: pgmap v182: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 308 op/s 2024-09-10T10:51:50.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:49 smithi103 ceph-mon[35160]: pgmap v182: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 308 op/s 2024-09-10T10:51:50.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:49 smithi007 ceph-mon[30873]: pgmap v182: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 308 op/s 2024-09-10T10:51:50.515 INFO:tasks.workunit.client.0.smithi007.stdout: 1 678754 12.46 MB/sec execute 170 sec latency 11.722 ms 2024-09-10T10:51:51.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 681678 12.48 MB/sec execute 171 sec latency 14.939 ms 2024-09-10T10:51:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:51 smithi133 ceph-mon[35282]: pgmap v183: 97 pgs: 97 active+clean; 179 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 438 op/s 2024-09-10T10:51:52.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:51 smithi103 ceph-mon[35160]: pgmap v183: 97 pgs: 97 active+clean; 179 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 438 op/s 2024-09-10T10:51:52.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:51 smithi007 ceph-mon[30873]: pgmap v183: 97 pgs: 97 active+clean; 179 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 438 op/s 2024-09-10T10:51:52.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 683561 12.46 MB/sec execute 172 sec latency 17.726 ms 2024-09-10T10:51:53.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 685806 12.47 MB/sec execute 173 sec latency 23.646 ms 2024-09-10T10:51:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:53 smithi133 ceph-mon[35282]: pgmap v184: 97 pgs: 97 active+clean; 179 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 309 op/s 2024-09-10T10:51:54.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:53 smithi103 ceph-mon[35160]: pgmap v184: 97 pgs: 97 active+clean; 179 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 309 op/s 2024-09-10T10:51:54.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:53 smithi007 ceph-mon[30873]: pgmap v184: 97 pgs: 97 active+clean; 179 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 309 op/s 2024-09-10T10:51:54.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 688314 12.47 MB/sec execute 174 sec latency 23.377 ms 2024-09-10T10:51:54.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.567+0000 7fb06b227640 1 -- 172.21.15.7:0/3232822608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb064102470 msgr2=0x7fb06410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.567+0000 7fb06b227640 1 --2- 172.21.15.7:0/3232822608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb064102470 0x7fb06410e9a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009cd0 tx=0x7fb05c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.567+0000 7fb06b227640 1 -- 172.21.15.7:0/3232822608 shutdown_connections 2024-09-10T10:51:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.567+0000 7fb06b227640 1 --2- 172.21.15.7:0/3232822608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb064102470 0x7fb06410e9a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.567+0000 7fb06b227640 1 --2- 172.21.15.7:0/3232822608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb064101ad0 0x7fb064101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.567+0000 7fb06b227640 1 --2- 172.21.15.7:0/3232822608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb064107b70 0x7fb064107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.567+0000 7fb06b227640 1 -- 172.21.15.7:0/3232822608 >> 172.21.15.7:0/3232822608 conn(0x7fb0640fd7d0 msgr2=0x7fb0640ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.567+0000 7fb06b227640 1 -- 172.21.15.7:0/3232822608 shutdown_connections 2024-09-10T10:51:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.567+0000 7fb06b227640 1 -- 172.21.15.7:0/3232822608 wait complete. 2024-09-10T10:51:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb06b227640 1 Processor -- start 2024-09-10T10:51:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb06b227640 1 -- start start 2024-09-10T10:51:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb06b227640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb064101ad0 0x7fb0641982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb06b227640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb064102470 0x7fb064198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb06b227640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb064107b70 0x7fb06419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb06b227640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb064111ee0 con 0x7fb064101ad0 2024-09-10T10:51:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb06b227640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb064111d60 con 0x7fb064102470 2024-09-10T10:51:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb06b227640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb064112060 con 0x7fb064107b70 2024-09-10T10:51:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb068fc3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb064101ad0 0x7fb0641982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb068fc3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb064101ad0 0x7fb0641982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59716/0 (socket says 172.21.15.7:59716) 2024-09-10T10:51:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb068fc3640 1 -- 172.21.15.7:0/11959462 learned_addr learned my addr 172.21.15.7:0/11959462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb0697c4640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb064107b70 0x7fb06419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb063fff640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb064102470 0x7fb064198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.568+0000 7fb068fc3640 1 -- 172.21.15.7:0/11959462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb064107b70 msgr2=0x7fb06419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.569+0000 7fb068fc3640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb064107b70 0x7fb06419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.569+0000 7fb068fc3640 1 -- 172.21.15.7:0/11959462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb064102470 msgr2=0x7fb064198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.569+0000 7fb068fc3640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb064102470 0x7fb064198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.569+0000 7fb068fc3640 1 -- 172.21.15.7:0/11959462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06419d2c0 con 0x7fb064101ad0 2024-09-10T10:51:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.569+0000 7fb068fc3640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb064101ad0 0x7fb0641982d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb05400e9f0 tx=0x7fb05400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.569+0000 7fb061ffb640 1 -- 172.21.15.7:0/11959462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05400cd80 con 0x7fb064101ad0 2024-09-10T10:51:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.569+0000 7fb061ffb640 1 -- 172.21.15.7:0/11959462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb054010910 con 0x7fb064101ad0 2024-09-10T10:51:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.569+0000 7fb06b227640 1 -- 172.21.15.7:0/11959462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06419d5b0 con 0x7fb064101ad0 2024-09-10T10:51:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.569+0000 7fb061ffb640 1 -- 172.21.15.7:0/11959462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054018990 con 0x7fb064101ad0 2024-09-10T10:51:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.569+0000 7fb06b227640 1 -- 172.21.15.7:0/11959462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0641a4e40 con 0x7fb064101ad0 2024-09-10T10:51:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb061ffb640 1 -- 172.21.15.7:0/11959462 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb054010ac0 con 0x7fb064101ad0 2024-09-10T10:51:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb06dfc3740 1 Processor -- start 2024-09-10T10:51:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb06dfc3740 1 -- start start 2024-09-10T10:51:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ed9b0 0x558f3b2edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b2ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558f3b2522f0 con 0x558f3b2fd0c0 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558f3b1fa270 con 0x558f3b2ed9b0 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558f3b24e2b0 con 0x558f3b2ee2d0 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb061ffb640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb0500a2da0 0x7fb0500a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb063fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b2ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb068fc3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb063fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b2ff4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59718/0 (socket says 172.21.15.7:59718) 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb068fc3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:50100/0 (socket says 172.21.15.7:50100) 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.572+0000 7fb063fff640 1 -- 172.21.15.7:0/444924186 learned_addr learned my addr 172.21.15.7:0/444924186 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb0697c4640 1 --2- 172.21.15.7:0/444924186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ed9b0 0x558f3b2edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb0697c4640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb0500a2da0 0x7fb0500a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb061ffb640 1 -- 172.21.15.7:0/11959462 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fb0540cf6e0 con 0x7fb064101ad0 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb063fff640 1 -- 172.21.15.7:0/444924186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 msgr2=0x558f3b2fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb063fff640 1 --2- 172.21.15.7:0/444924186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb063fff640 1 -- 172.21.15.7:0/444924186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ed9b0 msgr2=0x558f3b2edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb063fff640 1 --2- 172.21.15.7:0/444924186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ed9b0 0x558f3b2edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb063fff640 1 -- 172.21.15.7:0/444924186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3b1f9340 con 0x558f3b2fd0c0 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb068fc3640 1 --2- 172.21.15.7:0/444924186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb063fff640 1 --2- 172.21.15.7:0/444924186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b2ff4b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009ca0 tx=0x7fb04c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b0cba3715240f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb0697c4640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb0500a2da0 0x7fb0500a5260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb05c004ab0 tx=0x7fb05c004a40 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb05affd640 1 -- 172.21.15.7:0/444924186 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c01e070 con 0x558f3b2fd0c0 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb05affd640 1 -- 172.21.15.7:0/444924186 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c009e50 con 0x558f3b2fd0c0 2024-09-10T10:51:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.573+0000 7fb05affd640 1 -- 172.21.15.7:0/444924186 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00e9a0 con 0x558f3b2fd0c0 2024-09-10T10:51:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.574+0000 7fb06dfc3740 1 -- 172.21.15.7:0/444924186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 msgr2=0x558f3b2ff4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.574+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/444924186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b2ff4b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009ca0 tx=0x7fb04c011810 comp rx=0 tx=0).stop 2024-09-10T10:51:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.574+0000 7fb06dfc3740 1 -- 172.21.15.7:0/444924186 shutdown_connections 2024-09-10T10:51:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.574+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/444924186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b2ff4b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.574+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/444924186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.574+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/444924186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ed9b0 0x558f3b2edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.574+0000 7fb06dfc3740 1 -- 172.21.15.7:0/444924186 >> 172.21.15.7:0/444924186 conn(0x558f3b2ea060 msgr2=0x558f3b2ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.574+0000 7fb06dfc3740 1 -- 172.21.15.7:0/444924186 shutdown_connections 2024-09-10T10:51:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.574+0000 7fb06dfc3740 1 -- 172.21.15.7:0/444924186 wait complete. 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.574+0000 7fb06dfc3740 1 -- start start 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb06dfc3740 1 Processor -- start 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ed9b0 0x558f3b2fa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2fd0c0 0x558f3b2faaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558f3b2ffc90 con 0x558f3b2ee2d0 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558f3b1db290 con 0x558f3b2ed9b0 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558f3b2ffe10 con 0x558f3b2fd0c0 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb06dfc3740 10 client.0 _ll_register_callbacks cb 0x558f3b20c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb063fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb063fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fa560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59728/0 (socket says 172.21.15.7:59728) 2024-09-10T10:51:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb063fff640 1 -- 172.21.15.7:0/2869936625 learned_addr learned my addr 172.21.15.7:0/2869936625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb0697c4640 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2fd0c0 0x558f3b2faaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb068fc3640 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ed9b0 0x558f3b2fa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb063fff640 1 -- 172.21.15.7:0/2869936625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2fd0c0 msgr2=0x558f3b2faaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb063fff640 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2fd0c0 0x558f3b2faaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb063fff640 1 -- 172.21.15.7:0/2869936625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ed9b0 msgr2=0x558f3b2fa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb063fff640 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ed9b0 0x558f3b2fa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.575+0000 7fb063fff640 1 -- 172.21.15.7:0/2869936625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3af65210 con 0x558f3b2ee2d0 2024-09-10T10:51:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb063fff640 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fa560 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00e520 tx=0x7fb04c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e71cdaa494dbf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb043fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb043fff640 1 -- 172.21.15.7:0/2869936625 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c027020 con 0x558f3b2ee2d0 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb06dfc3740 1 -- 172.21.15.7:0/2869936625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb04c009e50 con 0x558f3b2ee2d0 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb043fff640 1 -- 172.21.15.7:0/2869936625 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fb04c00eb30 con 0x558f3b2ee2d0 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb043fff640 1 -- 172.21.15.7:0/2869936625 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb04c032050 con 0x558f3b2ee2d0 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb06dfc3740 10 client.15723 fetch_fsmap learned FSMap version 32 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb06dfc3740 1 -- 172.21.15.7:0/2869936625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558f3b031620 con 0x558f3b2ee2d0 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb043fff640 1 -- 172.21.15.7:0/2869936625 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c009e50 con 0x558f3b2ee2d0 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb043fff640 1 -- 172.21.15.7:0/2869936625 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb04c019550 con 0x558f3b2ee2d0 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb06dfc3740 10 client.15723 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb06dfc3740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb06dfc3740 20 client.15723 populate_metadata read hostname 'smithi007' 2024-09-10T10:51:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558f3b30cfa0 0x558f3b30f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb06dfc3740 4 client.15723 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.576+0000 7fb06dfc3740 1 -- 172.21.15.7:0/2869936625 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f3b25fa20 con 0x558f3b30cfa0 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.577+0000 7fb068fc3640 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558f3b30cfa0 0x558f3b30f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.577+0000 7fb068fc3640 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558f3b30cfa0 0x558f3b30f390 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95875f277f0d05 server_cookie=5ac1c8f165d90e00 in_seq=0 out_seq=0 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.577+0000 7fb043fff640 10 client.15723 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb043fff640 1 -- 172.21.15.7:0/2869936625 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f3b25fa20 con 0x558f3b30cfa0 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb043fff640 10 client.15723 handle_command_reply: tid=0 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 client.15723 shutdown 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 -- 172.21.15.7:0/2869936625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ee2d0 msgr2=0x558f3b2fa560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fa560 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00e520 tx=0x7fb04c005740 comp rx=0 tx=0).stop 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 -- 172.21.15.7:0/2869936625 shutdown_connections 2024-09-10T10:51:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558f3b30cfa0 0x558f3b30f390 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2fd0c0 0x558f3b2faaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ee2d0 0x558f3b2fa560 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/2869936625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ed9b0 0x558f3b2fa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb043fff640 0 client.15723 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 -- 172.21.15.7:0/2869936625 >> 172.21.15.7:0/2869936625 conn(0x558f3b2ea060 msgr2=0x558f3b2efdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 -- 172.21.15.7:0/2869936625 shutdown_connections 2024-09-10T10:51:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 1 -- 172.21.15.7:0/2869936625 wait complete. 2024-09-10T10:51:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.578+0000 7fb06dfc3740 20 client.15723 trim_cache size 0 max 16384 2024-09-10T10:51:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.593+0000 7fb06dfc3740 1 Processor -- start 2024-09-10T10:51:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.593+0000 7fb06dfc3740 1 -- start start 2024-09-10T10:51:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.593+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ed9b0 0x558f3b2fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.593+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ee2d0 0x558f3b314ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.593+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b315010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.593+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641adde0 con 0x558f3b2fd0c0 2024-09-10T10:51:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.593+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641ab6b0 con 0x558f3b2ee2d0 2024-09-10T10:51:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.593+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641a9520 con 0x558f3b2ed9b0 2024-09-10T10:51:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb0697c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b315010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb068fc3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ed9b0 0x558f3b2fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb0697c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b315010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59738/0 (socket says 172.21.15.7:59738) 2024-09-10T10:51:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb0697c4640 1 -- 172.21.15.7:0/3605237041 learned_addr learned my addr 172.21.15.7:0/3605237041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb063fff640 1 --2- 172.21.15.7:0/3605237041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ee2d0 0x558f3b314ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb0697c4640 1 -- 172.21.15.7:0/3605237041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ed9b0 msgr2=0x558f3b2fbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb0697c4640 1 --2- 172.21.15.7:0/3605237041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ed9b0 0x558f3b2fbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb0697c4640 1 -- 172.21.15.7:0/3605237041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ee2d0 msgr2=0x558f3b314ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb0697c4640 1 --2- 172.21.15.7:0/3605237041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ee2d0 0x558f3b314ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb0697c4640 1 -- 172.21.15.7:0/3605237041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3b2eef80 con 0x558f3b2fd0c0 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb0697c4640 1 --2- 172.21.15.7:0/3605237041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b315010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009cd0 tx=0x7fb05c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b93884d979d230 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb05a7fc640 1 -- 172.21.15.7:0/3605237041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c058070 con 0x558f3b2fd0c0 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb05a7fc640 1 -- 172.21.15.7:0/3605237041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05c036070 con 0x558f3b2fd0c0 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.594+0000 7fb05a7fc640 1 -- 172.21.15.7:0/3605237041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c041050 con 0x558f3b2fd0c0 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 -- 172.21.15.7:0/3605237041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 msgr2=0x558f3b315010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/3605237041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b315010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009cd0 tx=0x7fb05c009360 comp rx=0 tx=0).stop 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 -- 172.21.15.7:0/3605237041 shutdown_connections 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/3605237041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2fd0c0 0x558f3b315010 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/3605237041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2ee2d0 0x558f3b314ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/3605237041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ed9b0 0x558f3b2fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 -- 172.21.15.7:0/3605237041 >> 172.21.15.7:0/3605237041 conn(0x558f3b2ea060 msgr2=0x558f3b2fb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 -- 172.21.15.7:0/3605237041 shutdown_connections 2024-09-10T10:51:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 -- 172.21.15.7:0/3605237041 wait complete. 2024-09-10T10:51:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 -- start start 2024-09-10T10:51:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.595+0000 7fb06dfc3740 1 Processor -- start 2024-09-10T10:51:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ed9b0 0x558f3b305910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 0x558f3b2f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb06dfc3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2fd0c0 0x558f3b2f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641a9b70 con 0x558f3b2ed9b0 2024-09-10T10:51:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641aafe0 con 0x558f3b2fd0c0 2024-09-10T10:51:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb06dfc3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb0641aa240 con 0x558f3b2ee2d0 2024-09-10T10:51:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb06dfc3740 10 client.0 _ll_register_callbacks cb 0x7fb04c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb068fc3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ed9b0 0x558f3b305910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb068fc3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ed9b0 0x558f3b305910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59746/0 (socket says 172.21.15.7:59746) 2024-09-10T10:51:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb068fc3640 1 -- 172.21.15.7:0/1952987428 learned_addr learned my addr 172.21.15.7:0/1952987428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:51:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb0697c4640 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2fd0c0 0x558f3b2f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb063fff640 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 0x558f3b2f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb068fc3640 1 -- 172.21.15.7:0/1952987428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 msgr2=0x558f3b2f6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb068fc3640 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 0x558f3b2f6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb068fc3640 1 -- 172.21.15.7:0/1952987428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2fd0c0 msgr2=0x558f3b2f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb068fc3640 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2fd0c0 0x558f3b2f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.596+0000 7fb068fc3640 1 -- 172.21.15.7:0/1952987428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3b2f7bf0 con 0x558f3b2ed9b0 2024-09-10T10:51:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb068fc3640 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ed9b0 0x558f3b305910 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb0540cce90 tx=0x7fb0540d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb18b3923911db2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb043fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:51:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb043fff640 1 -- 172.21.15.7:0/1952987428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05409a070 con 0x558f3b2ed9b0 2024-09-10T10:51:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb043fff640 1 -- 172.21.15.7:0/1952987428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8520+0+0 (secure 0 0 0) 0x7fb0540cd0b0 con 0x558f3b2ed9b0 2024-09-10T10:51:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb06dfc3740 1 -- 172.21.15.7:0/1952987428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c036070 con 0x558f3b2ed9b0 2024-09-10T10:51:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb043fff640 1 -- 172.21.15.7:0/1952987428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb0540cd2b0 con 0x558f3b2ed9b0 2024-09-10T10:51:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb06dfc3740 10 client.15729 fetch_fsmap learned FSMap version 32 2024-09-10T10:51:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb06dfc3740 1 -- 172.21.15.7:0/1952987428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558f3b377b90 con 0x558f3b2ed9b0 2024-09-10T10:51:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb043fff640 1 -- 172.21.15.7:0/1952987428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05c036070 con 0x558f3b2ed9b0 2024-09-10T10:51:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb043fff640 1 -- 172.21.15.7:0/1952987428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7fb054018b30 con 0x558f3b2ed9b0 2024-09-10T10:51:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb06dfc3740 10 client.15729 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:51:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb06dfc3740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:51:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb06dfc3740 20 client.15729 populate_metadata read hostname 'smithi007' 2024-09-10T10:51:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558f3b30cfa0 0x558f3b37a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb06dfc3740 4 client.15729 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:51:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb06dfc3740 1 -- 172.21.15.7:0/1952987428 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f3b259d70 con 0x558f3b30cfa0 2024-09-10T10:51:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.597+0000 7fb063fff640 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558f3b30cfa0 0x558f3b37a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.598+0000 7fb063fff640 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558f3b30cfa0 0x558f3b37a270 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295394663719c52c server_cookie=5beb02b75454146d in_seq=0 out_seq=0 2024-09-10T10:51:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.598+0000 7fb043fff640 10 client.15729 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb043fff640 1 -- 172.21.15.7:0/1952987428 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f3b259d70 con 0x558f3b30cfa0 2024-09-10T10:51:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb043fff640 10 client.15729 handle_command_reply: tid=0 2024-09-10T10:51:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 client.15729 shutdown 2024-09-10T10:51:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 -- 172.21.15.7:0/1952987428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ed9b0 msgr2=0x558f3b305910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ed9b0 0x558f3b305910 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb0540cce90 tx=0x7fb0540d1040 comp rx=0 tx=0).stop 2024-09-10T10:51:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 -- 172.21.15.7:0/1952987428 shutdown_connections 2024-09-10T10:51:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558f3b30cfa0 0x558f3b37a270 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3b2fd0c0 0x558f3b2f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb043fff640 0 client.15729 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:51:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558f3b2ee2d0 0x558f3b2f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 --2- 172.21.15.7:0/1952987428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558f3b2ed9b0 0x558f3b305910 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 -- 172.21.15.7:0/1952987428 >> 172.21.15.7:0/1952987428 conn(0x558f3b2ea060 msgr2=0x558f3b2e91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 -- 172.21.15.7:0/1952987428 shutdown_connections 2024-09-10T10:51:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.674+0000 7fb06dfc3740 1 -- 172.21.15.7:0/1952987428 wait complete. 2024-09-10T10:51:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06dfc3740 20 client.15729 trim_cache size 0 max 16384 2024-09-10T10:51:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:51:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:51:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:51:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:51:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 -- 172.21.15.7:0/11959462 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb0500a2da0 msgr2=0x7fb0500a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb0500a2da0 0x7fb0500a5260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb05c004ab0 tx=0x7fb05c004a40 comp rx=0 tx=0).stop 2024-09-10T10:51:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 -- 172.21.15.7:0/11959462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb064101ad0 msgr2=0x7fb0641982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb064101ad0 0x7fb0641982d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb05400e9f0 tx=0x7fb05400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 -- 172.21.15.7:0/11959462 shutdown_connections 2024-09-10T10:51:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb064107b70 0x7fb06419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb064102470 0x7fb064198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb0500a2da0 0x7fb0500a5260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 --2- 172.21.15.7:0/11959462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb064101ad0 0x7fb0641982d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 -- 172.21.15.7:0/11959462 >> 172.21.15.7:0/11959462 conn(0x7fb0640fd7d0 msgr2=0x7fb0640fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 -- 172.21.15.7:0/11959462 shutdown_connections 2024-09-10T10:51:54.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:51:54.675+0000 7fb06b227640 1 -- 172.21.15.7:0/11959462 wait complete. 2024-09-10T10:51:55.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 689862 12.44 MB/sec execute 175 sec latency 37.862 ms 2024-09-10T10:51:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:55 smithi133 ceph-mon[35282]: pgmap v185: 97 pgs: 97 active+clean; 179 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 309 op/s 2024-09-10T10:51:56.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:55 smithi103 ceph-mon[35160]: pgmap v185: 97 pgs: 97 active+clean; 179 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 309 op/s 2024-09-10T10:51:56.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:55 smithi007 ceph-mon[30873]: pgmap v185: 97 pgs: 97 active+clean; 179 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 309 op/s 2024-09-10T10:51:56.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 692716 12.47 MB/sec execute 176 sec latency 15.587 ms 2024-09-10T10:51:57.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 694909 12.44 MB/sec execute 177 sec latency 18.921 ms 2024-09-10T10:51:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:57 smithi133 ceph-mon[35282]: pgmap v186: 97 pgs: 97 active+clean; 174 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-09-10T10:51:58.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:57 smithi103 ceph-mon[35160]: pgmap v186: 97 pgs: 97 active+clean; 174 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-09-10T10:51:58.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:57 smithi007 ceph-mon[30873]: pgmap v186: 97 pgs: 97 active+clean; 174 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 486 op/s 2024-09-10T10:51:58.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 696638 12.44 MB/sec execute 178 sec latency 35.540 ms 2024-09-10T10:51:59.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 699666 12.47 MB/sec execute 179 sec latency 12.375 ms 2024-09-10T10:51:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:51:59 smithi133 ceph-mon[35282]: pgmap v187: 97 pgs: 97 active+clean; 174 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 307 op/s 2024-09-10T10:52:00.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:51:59 smithi103 ceph-mon[35160]: pgmap v187: 97 pgs: 97 active+clean; 174 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 307 op/s 2024-09-10T10:52:00.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:51:59 smithi007 ceph-mon[30873]: pgmap v187: 97 pgs: 97 active+clean; 174 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 307 op/s 2024-09-10T10:52:00.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 701429 12.44 MB/sec execute 180 sec latency 12.782 ms 2024-09-10T10:52:01.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 703356 12.44 MB/sec execute 181 sec latency 23.213 ms 2024-09-10T10:52:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:01 smithi133 ceph-mon[35282]: pgmap v188: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 439 op/s 2024-09-10T10:52:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:01 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:52:02.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:01 smithi103 ceph-mon[35160]: pgmap v188: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 439 op/s 2024-09-10T10:52:02.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:01 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:52:02.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:01 smithi007 ceph-mon[30873]: pgmap v188: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 439 op/s 2024-09-10T10:52:02.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:01 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:52:02.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 705490 12.42 MB/sec execute 182 sec latency 23.669 ms 2024-09-10T10:52:02.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:02 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:52:02.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:02 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:52:02.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:02 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:52:03.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:02 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:52:03.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:02 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:52:03.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:02 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:52:03.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:02 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:52:03.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:02 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:52:03.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:02 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:52:03.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 707271 12.41 MB/sec execute 183 sec latency 29.978 ms 2024-09-10T10:52:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:03 smithi133 ceph-mon[35282]: pgmap v189: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-09-10T10:52:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:03 smithi103 ceph-mon[35160]: pgmap v189: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-09-10T10:52:04.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:03 smithi007 ceph-mon[30873]: pgmap v189: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 308 op/s 2024-09-10T10:52:04.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 709068 12.39 MB/sec execute 184 sec latency 31.081 ms 2024-09-10T10:52:05.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:04 smithi103 ceph-mon[35160]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T10:52:05.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:04 smithi103 ceph-mon[35160]: pgmap v191: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 370 op/s 2024-09-10T10:52:05.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:04 smithi007 ceph-mon[30873]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T10:52:05.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:04 smithi007 ceph-mon[30873]: pgmap v191: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 370 op/s 2024-09-10T10:52:05.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:04 smithi133 ceph-mon[35282]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T10:52:05.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:04 smithi133 ceph-mon[35282]: pgmap v191: 97 pgs: 97 active+clean; 181 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 370 op/s 2024-09-10T10:52:05.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 710945 12.38 MB/sec execute 185 sec latency 17.174 ms 2024-09-10T10:52:06.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 713988 12.42 MB/sec execute 186 sec latency 18.678 ms 2024-09-10T10:52:07.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:06 smithi133 ceph-mon[35282]: pgmap v192: 97 pgs: 97 active+clean; 188 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-09-10T10:52:07.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:06 smithi103 ceph-mon[35160]: pgmap v192: 97 pgs: 97 active+clean; 188 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-09-10T10:52:07.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:06 smithi007 ceph-mon[30873]: pgmap v192: 97 pgs: 97 active+clean; 188 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-09-10T10:52:07.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 716201 12.39 MB/sec execute 187 sec latency 20.980 ms 2024-09-10T10:52:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.769+0000 7f4b4176f640 1 -- 172.21.15.7:0/112086520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b3c10c0d0 msgr2=0x7f4b3c10e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.769+0000 7f4b4176f640 1 --2- 172.21.15.7:0/112086520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b3c10c0d0 0x7f4b3c10e570 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009900 tx=0x7f4b3402a410 comp rx=0 tx=0).stop 2024-09-10T10:52:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.769+0000 7f4b4176f640 1 -- 172.21.15.7:0/112086520 shutdown_connections 2024-09-10T10:52:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.769+0000 7f4b4176f640 1 --2- 172.21.15.7:0/112086520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b3c10c0d0 0x7f4b3c10e570 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.769+0000 7f4b4176f640 1 --2- 172.21.15.7:0/112086520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b3c1035c0 0x7f4b3c10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.769+0000 7f4b4176f640 1 --2- 172.21.15.7:0/112086520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b3c102ca0 0x7f4b3c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.769+0000 7f4b4176f640 1 -- 172.21.15.7:0/112086520 >> 172.21.15.7:0/112086520 conn(0x7f4b3c0fc6b0 msgr2=0x7f4b3c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.769+0000 7f4b4176f640 1 -- 172.21.15.7:0/112086520 shutdown_connections 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.769+0000 7f4b4176f640 1 -- 172.21.15.7:0/112086520 wait complete. 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.770+0000 7f4b4176f640 1 Processor -- start 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.770+0000 7f4b4176f640 1 -- start start 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.770+0000 7f4b4176f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b3c102ca0 0x7f4b3c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.770+0000 7f4b4176f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b3c1035c0 0x7f4b3c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.770+0000 7f4b4176f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b3c10c0d0 0x7f4b3c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.770+0000 7f4b4176f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c111ed0 con 0x7f4b3c10c0d0 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.770+0000 7f4b4176f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c111d50 con 0x7f4b3c1035c0 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.770+0000 7f4b4176f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c112050 con 0x7f4b3c102ca0 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b3c10c0d0 0x7f4b3c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b3c10c0d0 0x7f4b3c19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35826/0 (socket says 172.21.15.7:35826) 2024-09-10T10:52:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3b7fe640 1 -- 172.21.15.7:0/1078609782 learned_addr learned my addr 172.21.15.7:0/1078609782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3affd640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b3c102ca0 0x7f4b3c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b3c1035c0 0x7f4b3c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3b7fe640 1 -- 172.21.15.7:0/1078609782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b3c102ca0 msgr2=0x7f4b3c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3b7fe640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b3c102ca0 0x7f4b3c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3b7fe640 1 -- 172.21.15.7:0/1078609782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b3c1035c0 msgr2=0x7f4b3c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3b7fe640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b3c1035c0 0x7f4b3c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3b7fe640 1 -- 172.21.15.7:0/1078609782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c19d310 con 0x7f4b3c10c0d0 2024-09-10T10:52:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b3c1035c0 0x7f4b3c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b3b7fe640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b3c10c0d0 0x7f4b3c19cbe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005ab0 tx=0x7f4b340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b23fff640 1 -- 172.21.15.7:0/1078609782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3403f030 con 0x7f4b3c10c0d0 2024-09-10T10:52:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b23fff640 1 -- 172.21.15.7:0/1078609782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b340044c0 con 0x7f4b3c10c0d0 2024-09-10T10:52:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b4176f640 1 -- 172.21.15.7:0/1078609782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c19d5a0 con 0x7f4b3c10c0d0 2024-09-10T10:52:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b23fff640 1 -- 172.21.15.7:0/1078609782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34035070 con 0x7f4b3c10c0d0 2024-09-10T10:52:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.771+0000 7f4b4176f640 1 -- 172.21.15.7:0/1078609782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c1a4e90 con 0x7f4b3c10c0d0 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.774+0000 7f4b445e2740 1 Processor -- start 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.774+0000 7f4b23fff640 1 -- 172.21.15.7:0/1078609782 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4b34042030 con 0x7f4b3c10c0d0 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.774+0000 7f4b445e2740 1 -- start start 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf624af0 0x55d0cf624ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf625410 0x55d0cf62f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf6320a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b445e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d0cf5892f0 con 0x55d0cf62fcb0 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b445e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d0cf531270 con 0x55d0cf624af0 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b445e2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d0cf5852b0 con 0x55d0cf625410 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf625410 0x55d0cf62f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf625410 0x55d0cf62f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:35728/0 (socket says 172.21.15.7:35728) 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3affd640 1 -- 172.21.15.7:0/2180350678 learned_addr learned my addr 172.21.15.7:0/2180350678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/2180350678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf624af0 0x55d0cf624ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/2180350678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf6320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b23fff640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4b240a2da0 0x7f4b240a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3affd640 1 -- 172.21.15.7:0/2180350678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf624af0 msgr2=0x55d0cf624ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3affd640 1 --2- 172.21.15.7:0/2180350678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf624af0 0x55d0cf624ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3affd640 1 -- 172.21.15.7:0/2180350678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 msgr2=0x55d0cf6320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3affd640 1 --2- 172.21.15.7:0/2180350678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf6320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3affd640 1 -- 172.21.15.7:0/2180350678 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0cf530340 con 0x55d0cf625410 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/2180350678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf6320a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3b7fe640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4b240a2da0 0x7f4b240a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b23fff640 1 -- 172.21.15.7:0/1078609782 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8608+0+0 (secure 0 0 0) 0x7f4b340ef770 con 0x7f4b3c10c0d0 2024-09-10T10:52:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.775+0000 7f4b3affd640 1 --2- 172.21.15.7:0/2180350678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf625410 0x55d0cf62f770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009cf0 tx=0x7f4b2c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d8bad93044ba56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b217fa640 1 -- 172.21.15.7:0/2180350678 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c004230 con 0x55d0cf625410 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b217fa640 1 -- 172.21.15.7:0/2180350678 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c0043d0 con 0x55d0cf625410 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b217fa640 1 -- 172.21.15.7:0/2180350678 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c019900 con 0x55d0cf625410 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b445e2740 1 -- 172.21.15.7:0/2180350678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf625410 msgr2=0x55d0cf62f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b3b7fe640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4b240a2da0 0x7f4b240a5260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b340bb940 tx=0x7f4b340b10d0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b445e2740 1 --2- 172.21.15.7:0/2180350678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf625410 0x55d0cf62f770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009cf0 tx=0x7f4b2c011810 comp rx=0 tx=0).stop 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b445e2740 1 -- 172.21.15.7:0/2180350678 shutdown_connections 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b445e2740 1 --2- 172.21.15.7:0/2180350678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf6320a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b445e2740 1 --2- 172.21.15.7:0/2180350678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf625410 0x55d0cf62f770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b445e2740 1 --2- 172.21.15.7:0/2180350678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf624af0 0x55d0cf624ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b445e2740 1 -- 172.21.15.7:0/2180350678 >> 172.21.15.7:0/2180350678 conn(0x55d0cf621060 msgr2=0x55d0cf621440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.776+0000 7f4b445e2740 1 -- 172.21.15.7:0/2180350678 shutdown_connections 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b445e2740 1 -- 172.21.15.7:0/2180350678 wait complete. 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b445e2740 1 -- start start 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b445e2740 1 Processor -- start 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf62cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf62d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf62fcb0 0x55d0cf62d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b445e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d0cf632880 con 0x55d0cf624af0 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b445e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d0cf512290 con 0x55d0cf625410 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b445e2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d0cf632a00 con 0x55d0cf62fcb0 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b445e2740 10 client.0 _ll_register_callbacks cb 0x55d0cf543140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b3a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf62cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b3a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf62cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35842/0 (socket says 172.21.15.7:35842) 2024-09-10T10:52:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b3a7fc640 1 -- 172.21.15.7:0/2227237977 learned_addr learned my addr 172.21.15.7:0/2227237977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b3affd640 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf62d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.777+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf62fcb0 0x55d0cf62d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b3a7fc640 1 -- 172.21.15.7:0/2227237977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf62fcb0 msgr2=0x55d0cf62d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf62fcb0 0x55d0cf62d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b3a7fc640 1 -- 172.21.15.7:0/2227237977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 msgr2=0x55d0cf62d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf62d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b3a7fc640 1 -- 172.21.15.7:0/2227237977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0cf29c210 con 0x55d0cf624af0 2024-09-10T10:52:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf62cbf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4b30005fd0 tx=0x7f4b30004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32139d3e0322e496 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b1eff5640 1 -- 172.21.15.7:0/2227237977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30013d10 con 0x55d0cf624af0 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b1eff5640 1 -- 172.21.15.7:0/2227237977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8608+0+0 (secure 0 0 0) 0x7f4b30004b20 con 0x55d0cf624af0 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b445e2740 1 -- 172.21.15.7:0/2227237977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c0043d0 con 0x55d0cf624af0 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b1eff5640 1 -- 172.21.15.7:0/2227237977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4b30004d20 con 0x55d0cf624af0 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b445e2740 10 client.15741 fetch_fsmap learned FSMap version 32 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b445e2740 1 -- 172.21.15.7:0/2227237977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0cf368620 con 0x55d0cf624af0 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.778+0000 7f4b1eff5640 1 -- 172.21.15.7:0/2227237977 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c0043d0 con 0x55d0cf624af0 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.779+0000 7f4b1eff5640 1 -- 172.21.15.7:0/2227237977 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4b3001e030 con 0x55d0cf624af0 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.779+0000 7f4b445e2740 10 client.15741 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.779+0000 7f4b445e2740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.779+0000 7f4b445e2740 20 client.15741 populate_metadata read hostname 'smithi007' 2024-09-10T10:52:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.779+0000 7f4b445e2740 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d0cf63fc80 0x55d0cf642070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.779+0000 7f4b445e2740 4 client.15741 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.779+0000 7f4b445e2740 1 -- 172.21.15.7:0/2227237977 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0cf596a20 con 0x55d0cf63fc80 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.779+0000 7f4b3affd640 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d0cf63fc80 0x55d0cf642070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.779+0000 7f4b1eff5640 10 client.15741 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.779+0000 7f4b3affd640 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d0cf63fc80 0x55d0cf642070 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9028cbd48c4723e server_cookie=d81899600d9be3e6 in_seq=0 out_seq=0 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b1eff5640 1 -- 172.21.15.7:0/2227237977 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0cf596a20 con 0x55d0cf63fc80 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b1eff5640 10 client.15741 handle_command_reply: tid=0 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 client.15741 shutdown 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 -- 172.21.15.7:0/2227237977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 msgr2=0x55d0cf62cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf62cbf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4b30005fd0 tx=0x7f4b30004ab0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 -- 172.21.15.7:0/2227237977 shutdown_connections 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d0cf63fc80 0x55d0cf642070 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf62fcb0 0x55d0cf62d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf62d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 --2- 172.21.15.7:0/2227237977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf62cbf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 -- 172.21.15.7:0/2227237977 >> 172.21.15.7:0/2227237977 conn(0x55d0cf621060 msgr2=0x55d0cf626d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b1eff5640 0 client.15741 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 -- 172.21.15.7:0/2227237977 shutdown_connections 2024-09-10T10:52:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 1 -- 172.21.15.7:0/2227237977 wait complete. 2024-09-10T10:52:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.780+0000 7f4b445e2740 20 client.15741 trim_cache size 0 max 16384 2024-09-10T10:52:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.796+0000 7f4b445e2740 1 Processor -- start 2024-09-10T10:52:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.796+0000 7f4b445e2740 1 -- start start 2024-09-10T10:52:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.796+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf627560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.796+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf62d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.796+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf62fcb0 0x55d0cf647960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.796+0000 7f4b445e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1b0180 con 0x55d0cf624af0 2024-09-10T10:52:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.796+0000 7f4b445e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1ada50 con 0x55d0cf625410 2024-09-10T10:52:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.796+0000 7f4b445e2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1ab8a0 con 0x55d0cf62fcb0 2024-09-10T10:52:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf627560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf62d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf62d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:49866/0 (socket says 172.21.15.7:49866) 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf627560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35854/0 (socket says 172.21.15.7:35854) 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3affd640 1 -- 172.21.15.7:0/1990420451 learned_addr learned my addr 172.21.15.7:0/1990420451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/1990420451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf62fcb0 0x55d0cf647960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3affd640 1 -- 172.21.15.7:0/1990420451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf62fcb0 msgr2=0x55d0cf647960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3affd640 1 --2- 172.21.15.7:0/1990420451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf62fcb0 0x55d0cf647960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3affd640 1 -- 172.21.15.7:0/1990420451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 msgr2=0x55d0cf627560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3affd640 1 --2- 172.21.15.7:0/1990420451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf627560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3affd640 1 -- 172.21.15.7:0/1990420451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0cf624920 con 0x55d0cf625410 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b3affd640 1 --2- 172.21.15.7:0/1990420451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf62d4a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009cf0 tx=0x7f4b2c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f2ab90ebf0d417e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.797+0000 7f4b20ff9640 1 -- 172.21.15.7:0/1990420451 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c004180 con 0x55d0cf625410 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b20ff9640 1 -- 172.21.15.7:0/1990420451 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c003ad0 con 0x55d0cf625410 2024-09-10T10:52:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b20ff9640 1 -- 172.21.15.7:0/1990420451 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c004680 con 0x55d0cf625410 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b445e2740 1 -- 172.21.15.7:0/1990420451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 msgr2=0x55d0cf62d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b445e2740 1 --2- 172.21.15.7:0/1990420451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf62d4a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009cf0 tx=0x7f4b2c011810 comp rx=0 tx=0).stop 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b445e2740 1 -- 172.21.15.7:0/1990420451 shutdown_connections 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b445e2740 1 --2- 172.21.15.7:0/1990420451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf62fcb0 0x55d0cf647960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b445e2740 1 --2- 172.21.15.7:0/1990420451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf62d4a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b445e2740 1 --2- 172.21.15.7:0/1990420451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf624af0 0x55d0cf627560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b445e2740 1 -- 172.21.15.7:0/1990420451 >> 172.21.15.7:0/1990420451 conn(0x55d0cf621060 msgr2=0x55d0cf62ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b445e2740 1 -- 172.21.15.7:0/1990420451 shutdown_connections 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.798+0000 7f4b445e2740 1 -- 172.21.15.7:0/1990420451 wait complete. 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b445e2740 1 -- start start 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b445e2740 1 Processor -- start 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf624af0 0x55d0cf6382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf6335f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b445e2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf633b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b445e2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1abf10 con 0x55d0cf62fcb0 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b445e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1ad380 con 0x55d0cf625410 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b445e2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1ac5e0 con 0x55d0cf624af0 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b445e2740 10 client.0 _ll_register_callbacks cb 0x7f4b30004270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b3a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf633b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf6335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b3a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf633b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35868/0 (socket says 172.21.15.7:35868) 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b3a7fc640 1 -- 172.21.15.7:0/1439303031 learned_addr learned my addr 172.21.15.7:0/1439303031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.799+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf624af0 0x55d0cf6382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b3affd640 1 -- 172.21.15.7:0/1439303031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf624af0 msgr2=0x55d0cf6382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b3affd640 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf624af0 0x55d0cf6382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b3affd640 1 -- 172.21.15.7:0/1439303031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 msgr2=0x55d0cf633b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b3affd640 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf633b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b3affd640 1 -- 172.21.15.7:0/1439303031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0cf634220 con 0x55d0cf625410 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf633b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b3affd640 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf6335f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55d0cf62b710 tx=0x7f4b2c011e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aac2e98519ee530 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b1eff5640 1 -- 172.21.15.7:0/1439303031 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c005570 con 0x55d0cf625410 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b445e2740 1 -- 172.21.15.7:0/1439303031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c003ad0 con 0x55d0cf625410 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.800+0000 7f4b1eff5640 1 -- 172.21.15.7:0/1439303031 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8608+0+0 (secure 0 0 0) 0x7f4b2c0035f0 con 0x55d0cf625410 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b1eff5640 1 -- 172.21.15.7:0/1439303031 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4b2c003830 con 0x55d0cf625410 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b445e2740 10 client.25421 fetch_fsmap learned FSMap version 32 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b445e2740 1 -- 172.21.15.7:0/1439303031 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d0cf6aa800 con 0x55d0cf625410 2024-09-10T10:52:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b1eff5640 1 -- 172.21.15.7:0/1439303031 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c003ad0 con 0x55d0cf625410 2024-09-10T10:52:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b1eff5640 1 -- 172.21.15.7:0/1439303031 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 32) ==== 8346+0+0 (secure 0 0 0) 0x7f4b2c02b340 con 0x55d0cf625410 2024-09-10T10:52:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b445e2740 10 client.25421 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:52:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b445e2740 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b445e2740 20 client.25421 populate_metadata read hostname 'smithi007' 2024-09-10T10:52:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b445e2740 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d0cf6aad50 0x55d0cf6ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b445e2740 4 client.25421 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:52:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b445e2740 1 -- 172.21.15.7:0/1439303031 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0cf590d70 con 0x55d0cf6aad50 2024-09-10T10:52:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.801+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d0cf6aad50 0x55d0cf6ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.802+0000 7f4b1eff5640 10 client.25421 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.802+0000 7f4b3a7fc640 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d0cf6aad50 0x55d0cf6ad170 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9124ea15d8774cc9 server_cookie=de656c9dafe9b449 in_seq=0 out_seq=0 2024-09-10T10:52:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.871+0000 7f4b1eff5640 1 -- 172.21.15.7:0/1439303031 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0cf590d70 con 0x55d0cf6aad50 2024-09-10T10:52:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.871+0000 7f4b1eff5640 10 client.25421 handle_command_reply: tid=0 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.871+0000 7f4b445e2740 1 client.25421 shutdown 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 1 -- 172.21.15.7:0/1439303031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 msgr2=0x55d0cf6335f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf6335f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55d0cf62b710 tx=0x7f4b2c011e20 comp rx=0 tx=0).stop 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 1 -- 172.21.15.7:0/1439303031 shutdown_connections 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55d0cf6aad50 0x55d0cf6ad170 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d0cf62fcb0 0x55d0cf633b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d0cf625410 0x55d0cf6335f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b1eff5640 0 client.25421 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 1 --2- 172.21.15.7:0/1439303031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0cf624af0 0x55d0cf6382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 1 -- 172.21.15.7:0/1439303031 >> 172.21.15.7:0/1439303031 conn(0x55d0cf621060 msgr2=0x55d0cf61f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 1 -- 172.21.15.7:0/1439303031 shutdown_connections 2024-09-10T10:52:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 1 -- 172.21.15.7:0/1439303031 wait complete. 2024-09-10T10:52:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.872+0000 7f4b445e2740 20 client.25421 trim_cache size 0 max 16384 2024-09-10T10:52:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:52:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:52:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:52:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:52:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 -- 172.21.15.7:0/1078609782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4b240a2da0 msgr2=0x7f4b240a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4b240a2da0 0x7f4b240a5260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b340bb940 tx=0x7f4b340b10d0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 -- 172.21.15.7:0/1078609782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b3c10c0d0 msgr2=0x7f4b3c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b3c10c0d0 0x7f4b3c19cbe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005ab0 tx=0x7f4b340040c0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 -- 172.21.15.7:0/1078609782 shutdown_connections 2024-09-10T10:52:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4b240a2da0 0x7f4b240a5260 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b3c10c0d0 0x7f4b3c19cbe0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b3c1035c0 0x7f4b3c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 --2- 172.21.15.7:0/1078609782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b3c102ca0 0x7f4b3c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 -- 172.21.15.7:0/1078609782 >> 172.21.15.7:0/1078609782 conn(0x7f4b3c0fc6b0 msgr2=0x7f4b3c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 -- 172.21.15.7:0/1078609782 shutdown_connections 2024-09-10T10:52:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:07.873+0000 7f4b4176f640 1 -- 172.21.15.7:0/1078609782 wait complete. 2024-09-10T10:52:08.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 717859 12.39 MB/sec execute 188 sec latency 59.877 ms 2024-09-10T10:52:09.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 720615 12.40 MB/sec execute 189 sec latency 18.250 ms 2024-09-10T10:52:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:09 smithi103 ceph-mon[35160]: pgmap v193: 97 pgs: 97 active+clean; 188 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-09-10T10:52:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:09 smithi103 ceph-mon[35160]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T10:52:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:09 smithi007 ceph-mon[30873]: pgmap v193: 97 pgs: 97 active+clean; 188 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-09-10T10:52:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:09 smithi007 ceph-mon[30873]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T10:52:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:09 smithi133 ceph-mon[35282]: pgmap v193: 97 pgs: 97 active+clean; 188 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-09-10T10:52:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:09 smithi133 ceph-mon[35282]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T10:52:10.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 722379 12.39 MB/sec execute 190 sec latency 14.273 ms 2024-09-10T10:52:11.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 724791 12.39 MB/sec execute 191 sec latency 13.361 ms 2024-09-10T10:52:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:11 smithi103 ceph-mon[35160]: pgmap v195: 97 pgs: 97 active+clean; 185 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-09-10T10:52:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:11 smithi007 ceph-mon[30873]: pgmap v195: 97 pgs: 97 active+clean; 185 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-09-10T10:52:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:11 smithi133 ceph-mon[35282]: pgmap v195: 97 pgs: 97 active+clean; 185 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 501 op/s 2024-09-10T10:52:12.516 INFO:tasks.workunit.client.0.smithi007.stdout: 1 727395 12.40 MB/sec execute 192 sec latency 13.306 ms 2024-09-10T10:52:13.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 728818 12.37 MB/sec execute 193 sec latency 58.337 ms 2024-09-10T10:52:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:13 smithi103 ceph-mon[35160]: pgmap v196: 97 pgs: 97 active+clean; 185 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 491 op/s 2024-09-10T10:52:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:13 smithi007 ceph-mon[30873]: pgmap v196: 97 pgs: 97 active+clean; 185 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 491 op/s 2024-09-10T10:52:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:13 smithi133 ceph-mon[35282]: pgmap v196: 97 pgs: 97 active+clean; 185 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 491 op/s 2024-09-10T10:52:14.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 731296 12.38 MB/sec execute 194 sec latency 160.095 ms 2024-09-10T10:52:14.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:14 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:52:14.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:14 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:52:14.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:14 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:52:15.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 731296 12.31 MB/sec execute 195 sec latency 1160.193 ms 2024-09-10T10:52:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:15 smithi103 ceph-mon[35160]: pgmap v197: 97 pgs: 97 active+clean; 185 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 400 op/s 2024-09-10T10:52:15.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:15 smithi007 ceph-mon[30873]: pgmap v197: 97 pgs: 97 active+clean; 185 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 400 op/s 2024-09-10T10:52:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:15 smithi133 ceph-mon[35282]: pgmap v197: 97 pgs: 97 active+clean; 185 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 400 op/s 2024-09-10T10:52:16.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 731322 12.25 MB/sec execute 196 sec latency 1895.540 ms 2024-09-10T10:52:17.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 731322 12.19 MB/sec execute 197 sec latency 1224.761 ms 2024-09-10T10:52:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:17 smithi103 ceph-mon[35160]: pgmap v198: 97 pgs: 97 active+clean; 194 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-09-10T10:52:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:17 smithi007 ceph-mon[30873]: pgmap v198: 97 pgs: 97 active+clean; 194 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-09-10T10:52:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:17 smithi133 ceph-mon[35282]: pgmap v198: 97 pgs: 97 active+clean; 194 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-09-10T10:52:18.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 731322 12.13 MB/sec execute 198 sec latency 2224.892 ms 2024-09-10T10:52:19.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 731322 12.07 MB/sec execute 199 sec latency 3225.006 ms 2024-09-10T10:52:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:19 smithi103 ceph-mon[35160]: pgmap v199: 97 pgs: 97 active+clean; 194 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-09-10T10:52:19.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:19 smithi007 ceph-mon[30873]: pgmap v199: 97 pgs: 97 active+clean; 194 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-09-10T10:52:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:19 smithi133 ceph-mon[35282]: pgmap v199: 97 pgs: 97 active+clean; 194 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 354 op/s 2024-09-10T10:52:20.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 731322 12.01 MB/sec execute 200 sec latency 4225.114 ms 2024-09-10T10:52:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.973+0000 7f0d42f11640 1 -- 172.21.15.7:0/1455838284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c1020c0 msgr2=0x7f0d3c10d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.973+0000 7f0d42f11640 1 --2- 172.21.15.7:0/1455838284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c1020c0 0x7f0d3c10d8f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009ca0 tx=0x7f0d3402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.974+0000 7f0d42f11640 1 -- 172.21.15.7:0/1455838284 shutdown_connections 2024-09-10T10:52:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.974+0000 7f0d42f11640 1 --2- 172.21.15.7:0/1455838284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c1020c0 0x7f0d3c10d8f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.974+0000 7f0d42f11640 1 --2- 172.21.15.7:0/1455838284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c101740 0x7f0d3c101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.974+0000 7f0d42f11640 1 --2- 172.21.15.7:0/1455838284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d3c106b70 0x7f0d3c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.974+0000 7f0d42f11640 1 -- 172.21.15.7:0/1455838284 >> 172.21.15.7:0/1455838284 conn(0x7f0d3c075ef0 msgr2=0x7f0d3c076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.974+0000 7f0d42f11640 1 -- 172.21.15.7:0/1455838284 shutdown_connections 2024-09-10T10:52:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.974+0000 7f0d42f11640 1 -- 172.21.15.7:0/1455838284 wait complete. 2024-09-10T10:52:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.974+0000 7f0d42f11640 1 Processor -- start 2024-09-10T10:52:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.974+0000 7f0d42f11640 1 -- start start 2024-09-10T10:52:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d42f11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c101740 0x7f0d3c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d42f11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c1020c0 0x7f0d3c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d42f11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d3c106b70 0x7f0d3c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d42f11640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c110ed0 con 0x7f0d3c101740 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d42f11640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c110d50 con 0x7f0d3c1020c0 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d42f11640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c111050 con 0x7f0d3c106b70 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d40cad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c101740 0x7f0d3c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d40cad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c101740 0x7f0d3c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55446/0 (socket says 172.21.15.7:55446) 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d40cad640 1 -- 172.21.15.7:0/2436263605 learned_addr learned my addr 172.21.15.7:0/2436263605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d414ae640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d3c106b70 0x7f0d3c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d3bfff640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c1020c0 0x7f0d3c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d40cad640 1 -- 172.21.15.7:0/2436263605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d3c106b70 msgr2=0x7f0d3c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d40cad640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d3c106b70 0x7f0d3c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d40cad640 1 -- 172.21.15.7:0/2436263605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c1020c0 msgr2=0x7f0d3c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d40cad640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c1020c0 0x7f0d3c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d40cad640 1 -- 172.21.15.7:0/2436263605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3c19f520 con 0x7f0d3c101740 2024-09-10T10:52:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d40cad640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c101740 0x7f0d3c19a530 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000e9f0 tx=0x7f0d3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d39ffb640 1 -- 172.21.15.7:0/2436263605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3000cd80 con 0x7f0d3c101740 2024-09-10T10:52:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.975+0000 7f0d39ffb640 1 -- 172.21.15.7:0/2436263605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d30005500 con 0x7f0d3c101740 2024-09-10T10:52:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.976+0000 7f0d42f11640 1 -- 172.21.15.7:0/2436263605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d3c19f810 con 0x7f0d3c101740 2024-09-10T10:52:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.976+0000 7f0d39ffb640 1 -- 172.21.15.7:0/2436263605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30010690 con 0x7f0d3c101740 2024-09-10T10:52:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.976+0000 7f0d42f11640 1 -- 172.21.15.7:0/2436263605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d3c1a70a0 con 0x7f0d3c101740 2024-09-10T10:52:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.978+0000 7f0d39ffb640 1 -- 172.21.15.7:0/2436263605 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0d300046c0 con 0x7f0d3c101740 2024-09-10T10:52:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.978+0000 7f0d45ca6740 1 Processor -- start 2024-09-10T10:52:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d45ca6740 1 -- start start 2024-09-10T10:52:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d39ffb640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0d2c0a2da0 0x7f0d2c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a7f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a7fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a8000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564a2a7572f0 con 0x564a2a7fdcb0 2024-09-10T10:52:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a2a6ff270 con 0x564a2a7f3410 2024-09-10T10:52:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564a2a7532b0 con 0x564a2a7f2af0 2024-09-10T10:52:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d39ffb640 1 -- 172.21.15.7:0/2436263605 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7f0d300cf900 con 0x7f0d3c101740 2024-09-10T10:52:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d414ae640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0d2c0a2da0 0x7f0d2c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d414ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a8000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d414ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a8000a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55454/0 (socket says 172.21.15.7:55454) 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d414ae640 1 -- 172.21.15.7:0/2912515843 learned_addr learned my addr 172.21.15.7:0/2912515843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d40cad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a7f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d3bfff640 1 --2- 172.21.15.7:0/2912515843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a7fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d40cad640 1 -- 172.21.15.7:0/2912515843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 msgr2=0x564a2a7fd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d40cad640 1 --2- 172.21.15.7:0/2912515843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a7fd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d40cad640 1 -- 172.21.15.7:0/2912515843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 msgr2=0x564a2a8000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d40cad640 1 --2- 172.21.15.7:0/2912515843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a8000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d40cad640 1 -- 172.21.15.7:0/2912515843 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2a6fe340 con 0x564a2a7f2af0 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.979+0000 7f0d414ae640 1 --2- 172.21.15.7:0/2912515843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a8000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.980+0000 7f0d40cad640 1 --2- 172.21.15.7:0/2912515843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a7f2ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d300ccfc0 tx=0x7f0d30098160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb2b08b7000249e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.980+0000 7f0d414ae640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0d2c0a2da0 0x7f0d2c0a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x564a2a7f03e0 tx=0x7f0d3402a8c0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.980+0000 7f0d26ffd640 1 -- 172.21.15.7:0/2912515843 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d300aeab0 con 0x564a2a7f2af0 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.980+0000 7f0d26ffd640 1 -- 172.21.15.7:0/2912515843 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3009c490 con 0x564a2a7f2af0 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.980+0000 7f0d26ffd640 1 -- 172.21.15.7:0/2912515843 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d300ae8d0 con 0x564a2a7f2af0 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 -- 172.21.15.7:0/2912515843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 msgr2=0x564a2a7f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/2912515843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a7f2ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d300ccfc0 tx=0x7f0d30098160 comp rx=0 tx=0).stop 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 -- 172.21.15.7:0/2912515843 shutdown_connections 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/2912515843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a8000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/2912515843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a7fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/2912515843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a7f2ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 -- 172.21.15.7:0/2912515843 >> 172.21.15.7:0/2912515843 conn(0x564a2a7ef060 msgr2=0x564a2a7ef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 -- 172.21.15.7:0/2912515843 shutdown_connections 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 -- 172.21.15.7:0/2912515843 wait complete. 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 -- start start 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.981+0000 7f0d45ca6740 1 Processor -- start 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7f2af0 0x564a2a7fac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f3410 0x564a2a7fb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7fdcb0 0x564a2a7fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564a2a800880 con 0x564a2a7f2af0 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564a2a6e0290 con 0x564a2a7fdcb0 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564a2a800a00 con 0x564a2a7f3410 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d45ca6740 10 client.0 _ll_register_callbacks cb 0x564a2a711140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d40cad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7f2af0 0x564a2a7fac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d40cad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7f2af0 0x564a2a7fac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55464/0 (socket says 172.21.15.7:55464) 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d40cad640 1 -- 172.21.15.7:0/1318713827 learned_addr learned my addr 172.21.15.7:0/1318713827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d3bfff640 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f3410 0x564a2a7fb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d414ae640 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7fdcb0 0x564a2a7fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d40cad640 1 -- 172.21.15.7:0/1318713827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f3410 msgr2=0x564a2a7fb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d40cad640 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f3410 0x564a2a7fb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d40cad640 1 -- 172.21.15.7:0/1318713827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7fdcb0 msgr2=0x564a2a7fb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d40cad640 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7fdcb0 0x564a2a7fb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d40cad640 1 -- 172.21.15.7:0/1318713827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2a46a210 con 0x564a2a7f2af0 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d40cad640 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7f2af0 0x564a2a7fac20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0d300a44d0 tx=0x7f0d300b00d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af14b804364ff1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d1bfff640 1 -- 172.21.15.7:0/1318713827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d300cd440 con 0x564a2a7f2af0 2024-09-10T10:52:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d1bfff640 1 -- 172.21.15.7:0/1318713827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7f0d300ae890 con 0x564a2a7f2af0 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.982+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1318713827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3009c490 con 0x564a2a7f2af0 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d1bfff640 1 -- 172.21.15.7:0/1318713827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d300b9050 con 0x564a2a7f2af0 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d45ca6740 10 client.15759 fetch_fsmap learned FSMap version 33 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1318713827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a2a536620 con 0x564a2a7f2af0 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d1bfff640 1 -- 172.21.15.7:0/1318713827 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3009c490 con 0x564a2a7f2af0 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d1bfff640 1 -- 172.21.15.7:0/1318713827 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f0d3009b1e0 con 0x564a2a7f2af0 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d45ca6740 10 client.15759 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d45ca6740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d45ca6740 20 client.15759 populate_metadata read hostname 'smithi007' 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564a2a80db00 0x564a2a80fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d45ca6740 4 client.15759 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1318713827 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a2a764a20 con 0x564a2a80db00 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.983+0000 7f0d3bfff640 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564a2a80db00 0x564a2a80fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.984+0000 7f0d1bfff640 10 client.15759 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.984+0000 7f0d3bfff640 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564a2a80db00 0x564a2a80fef0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a855daa217b4d2b0 server_cookie=bdd8f3226523bf66 in_seq=0 out_seq=0 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.984+0000 7f0d1bfff640 1 -- 172.21.15.7:0/1318713827 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a2a764a20 con 0x564a2a80db00 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.984+0000 7f0d1bfff640 10 client.15759 handle_command_reply: tid=0 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.984+0000 7f0d45ca6740 1 client.15759 shutdown 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1318713827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7f2af0 msgr2=0x564a2a7fac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7f2af0 0x564a2a7fac20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0d300a44d0 tx=0x7f0d300b00d0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1318713827 shutdown_connections 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564a2a80db00 0x564a2a80fef0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7fdcb0 0x564a2a7fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f3410 0x564a2a7fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/1318713827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7f2af0 0x564a2a7fac20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1318713827 >> 172.21.15.7:0/1318713827 conn(0x564a2a7ef060 msgr2=0x564a2a7f4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d1bfff640 0 client.15759 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1318713827 shutdown_connections 2024-09-10T10:52:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1318713827 wait complete. 2024-09-10T10:52:20.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:20.985+0000 7f0d45ca6740 20 client.15759 trim_cache size 0 max 16384 2024-09-10T10:52:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d45ca6740 1 Processor -- start 2024-09-10T10:52:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d45ca6740 1 -- start start 2024-09-10T10:52:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a7fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a7fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a815550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1b2350 con 0x564a2a7fdcb0 2024-09-10T10:52:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1afbf0 con 0x564a2a7f3410 2024-09-10T10:52:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ada60 con 0x564a2a7f2af0 2024-09-10T10:52:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d414ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a815550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d414ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a815550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55474/0 (socket says 172.21.15.7:55474) 2024-09-10T10:52:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d414ae640 1 -- 172.21.15.7:0/1603778167 learned_addr learned my addr 172.21.15.7:0/1603778167 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d40cad640 1 --2- 172.21.15.7:0/1603778167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a7fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d3bfff640 1 --2- 172.21.15.7:0/1603778167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a7fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d414ae640 1 -- 172.21.15.7:0/1603778167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 msgr2=0x564a2a7fb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d414ae640 1 --2- 172.21.15.7:0/1603778167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a7fb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d414ae640 1 -- 172.21.15.7:0/1603778167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 msgr2=0x564a2a7fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d414ae640 1 --2- 172.21.15.7:0/1603778167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a7fcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.000+0000 7f0d414ae640 1 -- 172.21.15.7:0/1603778167 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2a7f0110 con 0x564a2a7fdcb0 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d414ae640 1 --2- 172.21.15.7:0/1603778167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a815550 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0d3402ade0 tx=0x7f0d34036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a202e2a851e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d267fc640 1 -- 172.21.15.7:0/1603778167 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34044040 con 0x564a2a7fdcb0 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d267fc640 1 -- 172.21.15.7:0/1603778167 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d34036070 con 0x564a2a7fdcb0 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d267fc640 1 -- 172.21.15.7:0/1603778167 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34030d90 con 0x564a2a7fdcb0 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1603778167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 msgr2=0x564a2a815550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/1603778167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a815550 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0d3402ade0 tx=0x7f0d34036000 comp rx=0 tx=0).stop 2024-09-10T10:52:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1603778167 shutdown_connections 2024-09-10T10:52:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/1603778167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a815550 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/1603778167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a7fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/1603778167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a7fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1603778167 >> 172.21.15.7:0/1603778167 conn(0x564a2a7ef060 msgr2=0x564a2a7fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1603778167 shutdown_connections 2024-09-10T10:52:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.001+0000 7f0d45ca6740 1 -- 172.21.15.7:0/1603778167 wait complete. 2024-09-10T10:52:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d45ca6740 1 -- start start 2024-09-10T10:52:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d45ca6740 1 Processor -- start 2024-09-10T10:52:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a806470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a8017e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d45ca6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a801d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ae0b0 con 0x564a2a7fdcb0 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1af520 con 0x564a2a7f3410 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d45ca6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3c1ae780 con 0x564a2a7f2af0 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d45ca6740 10 client.0 _ll_register_callbacks cb 0x7f0d30005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d414ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a801d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d414ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a801d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55478/0 (socket says 172.21.15.7:55478) 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.002+0000 7f0d414ae640 1 -- 172.21.15.7:0/553920121 learned_addr learned my addr 172.21.15.7:0/553920121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d40cad640 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a806470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d3bfff640 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a8017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d414ae640 1 -- 172.21.15.7:0/553920121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 msgr2=0x564a2a806470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d414ae640 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a806470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d414ae640 1 -- 172.21.15.7:0/553920121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 msgr2=0x564a2a8017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d414ae640 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a8017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d414ae640 1 -- 172.21.15.7:0/553920121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2a7f2920 con 0x564a2a7fdcb0 2024-09-10T10:52:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d414ae640 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a801d40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0d340093d0 tx=0x7f0d340307f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c613bdb75bccd790 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:52:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d1bfff640 1 -- 172.21.15.7:0/553920121 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34060060 con 0x564a2a7fdcb0 2024-09-10T10:52:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d1bfff640 1 -- 172.21.15.7:0/553920121 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7f0d3402fe30 con 0x564a2a7fdcb0 2024-09-10T10:52:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d45ca6740 1 -- 172.21.15.7:0/553920121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d34036070 con 0x564a2a7fdcb0 2024-09-10T10:52:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d1bfff640 1 -- 172.21.15.7:0/553920121 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d34030b30 con 0x564a2a7fdcb0 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d45ca6740 10 client.15765 fetch_fsmap learned FSMap version 33 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.003+0000 7f0d45ca6740 1 -- 172.21.15.7:0/553920121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a2a8783e0 con 0x564a2a7fdcb0 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d1bfff640 1 -- 172.21.15.7:0/553920121 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d34036070 con 0x564a2a7fdcb0 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d1bfff640 1 -- 172.21.15.7:0/553920121 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f0d3405b420 con 0x564a2a7fdcb0 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d45ca6740 10 client.15765 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d45ca6740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d45ca6740 20 client.15765 populate_metadata read hostname 'smithi007' 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564a2a878a20 0x564a2a87ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d45ca6740 4 client.15765 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d45ca6740 1 -- 172.21.15.7:0/553920121 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a2a75ed70 con 0x564a2a878a20 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d40cad640 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564a2a878a20 0x564a2a87ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d40cad640 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564a2a878a20 0x564a2a87ae40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c553ef787dd13d3b server_cookie=803c9c85159f9c87 in_seq=0 out_seq=0 2024-09-10T10:52:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.004+0000 7f0d1bfff640 10 client.15765 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d1bfff640 1 -- 172.21.15.7:0/553920121 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a2a75ed70 con 0x564a2a878a20 2024-09-10T10:52:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d1bfff640 10 client.15765 handle_command_reply: tid=0 2024-09-10T10:52:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 client.15765 shutdown 2024-09-10T10:52:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 -- 172.21.15.7:0/553920121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 msgr2=0x564a2a801d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a801d40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0d340093d0 tx=0x7f0d340307f0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 -- 172.21.15.7:0/553920121 shutdown_connections 2024-09-10T10:52:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x564a2a878a20 0x564a2a87ae40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a2a7fdcb0 0x564a2a801d40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564a2a7f3410 0x564a2a8017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d1bfff640 0 client.15765 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 --2- 172.21.15.7:0/553920121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a2a7f2af0 0x564a2a806470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 -- 172.21.15.7:0/553920121 >> 172.21.15.7:0/553920121 conn(0x564a2a7ef060 msgr2=0x564a2a7fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 -- 172.21.15.7:0/553920121 shutdown_connections 2024-09-10T10:52:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 1 -- 172.21.15.7:0/553920121 wait complete. 2024-09-10T10:52:21.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.082+0000 7f0d45ca6740 20 client.15765 trim_cache size 0 max 16384 2024-09-10T10:52:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:52:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:52:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:52:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 -- 172.21.15.7:0/2436263605 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0d2c0a2da0 msgr2=0x7f0d2c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0d2c0a2da0 0x7f0d2c0a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x564a2a7f03e0 tx=0x7f0d3402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 -- 172.21.15.7:0/2436263605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c101740 msgr2=0x7f0d3c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c101740 0x7f0d3c19a530 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000e9f0 tx=0x7f0d3000edf0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 -- 172.21.15.7:0/2436263605 shutdown_connections 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d3c106b70 0x7f0d3c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0d3c1020c0 0x7f0d3c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0d2c0a2da0 0x7f0d2c0a5260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 --2- 172.21.15.7:0/2436263605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d3c101740 0x7f0d3c19a530 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 -- 172.21.15.7:0/2436263605 >> 172.21.15.7:0/2436263605 conn(0x7f0d3c075ef0 msgr2=0x7f0d3c10bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 -- 172.21.15.7:0/2436263605 shutdown_connections 2024-09-10T10:52:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:21.083+0000 7f0d42f11640 1 -- 172.21.15.7:0/2436263605 wait complete. 2024-09-10T10:52:21.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 731743 11.97 MB/sec execute 201 sec latency 4957.500 ms 2024-09-10T10:52:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:21 smithi103 ceph-mon[35160]: pgmap v200: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 338 op/s 2024-09-10T10:52:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:21 smithi007 ceph-mon[30873]: pgmap v200: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 338 op/s 2024-09-10T10:52:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:21 smithi133 ceph-mon[35282]: pgmap v200: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 338 op/s 2024-09-10T10:52:22.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 733891 11.94 MB/sec execute 202 sec latency 21.325 ms 2024-09-10T10:52:23.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 735473 11.94 MB/sec execute 203 sec latency 60.249 ms 2024-09-10T10:52:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:23 smithi103 ceph-mon[35160]: pgmap v201: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.5 MiB/s wr, 143 op/s 2024-09-10T10:52:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:23 smithi007 ceph-mon[30873]: pgmap v201: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.5 MiB/s wr, 143 op/s 2024-09-10T10:52:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:23 smithi133 ceph-mon[35282]: pgmap v201: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.5 MiB/s wr, 143 op/s 2024-09-10T10:52:24.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 738042 11.95 MB/sec execute 204 sec latency 19.681 ms 2024-09-10T10:52:25.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 739773 11.93 MB/sec execute 205 sec latency 11.502 ms 2024-09-10T10:52:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:25 smithi103 ceph-mon[35160]: pgmap v202: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.5 MiB/s wr, 143 op/s 2024-09-10T10:52:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:25 smithi007 ceph-mon[30873]: pgmap v202: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.5 MiB/s wr, 143 op/s 2024-09-10T10:52:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:25 smithi133 ceph-mon[35282]: pgmap v202: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.5 MiB/s wr, 143 op/s 2024-09-10T10:52:26.517 INFO:tasks.workunit.client.0.smithi007.stdout: 1 742387 11.95 MB/sec execute 206 sec latency 14.376 ms 2024-09-10T10:52:27.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 744737 11.94 MB/sec execute 207 sec latency 11.788 ms 2024-09-10T10:52:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:27 smithi103 ceph-mon[35160]: pgmap v203: 97 pgs: 97 active+clean; 190 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 284 op/s 2024-09-10T10:52:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:27 smithi007 ceph-mon[30873]: pgmap v203: 97 pgs: 97 active+clean; 190 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 284 op/s 2024-09-10T10:52:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:27 smithi133 ceph-mon[35282]: pgmap v203: 97 pgs: 97 active+clean; 190 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 284 op/s 2024-09-10T10:52:28.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 746457 11.94 MB/sec execute 208 sec latency 39.173 ms 2024-09-10T10:52:29.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 749218 11.96 MB/sec execute 209 sec latency 20.178 ms 2024-09-10T10:52:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:29 smithi103 ceph-mon[35160]: pgmap v204: 97 pgs: 97 active+clean; 190 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-09-10T10:52:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:29 smithi007 ceph-mon[30873]: pgmap v204: 97 pgs: 97 active+clean; 190 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-09-10T10:52:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:29 smithi133 ceph-mon[35282]: pgmap v204: 97 pgs: 97 active+clean; 190 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-09-10T10:52:30.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 751111 11.94 MB/sec execute 210 sec latency 13.690 ms 2024-09-10T10:52:31.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 753322 11.94 MB/sec execute 211 sec latency 12.493 ms 2024-09-10T10:52:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:31 smithi103 ceph-mon[35160]: pgmap v205: 97 pgs: 97 active+clean; 198 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 293 op/s 2024-09-10T10:52:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:31 smithi007 ceph-mon[30873]: pgmap v205: 97 pgs: 97 active+clean; 198 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 293 op/s 2024-09-10T10:52:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:31 smithi133 ceph-mon[35282]: pgmap v205: 97 pgs: 97 active+clean; 198 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 293 op/s 2024-09-10T10:52:32.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 756283 11.97 MB/sec execute 212 sec latency 17.702 ms 2024-09-10T10:52:33.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 758012 11.95 MB/sec execute 213 sec latency 11.652 ms 2024-09-10T10:52:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:33 smithi103 ceph-mon[35160]: pgmap v206: 97 pgs: 97 active+clean; 198 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-09-10T10:52:33.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:33 smithi007 ceph-mon[30873]: pgmap v206: 97 pgs: 97 active+clean; 198 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-09-10T10:52:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:33 smithi133 ceph-mon[35282]: pgmap v206: 97 pgs: 97 active+clean; 198 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-09-10T10:52:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.176+0000 7fe16a5e8640 1 -- 172.21.15.7:0/1953538238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe164102cc0 msgr2=0x7fe1641030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.176+0000 7fe16a5e8640 1 --2- 172.21.15.7:0/1953538238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe164102cc0 0x7fe1641030a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe154009900 tx=0x7fe15402a480 comp rx=0 tx=0).stop 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.176+0000 7fe16a5e8640 1 -- 172.21.15.7:0/1953538238 shutdown_connections 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.176+0000 7fe16a5e8640 1 --2- 172.21.15.7:0/1953538238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe16410c110 0x7fe16410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.176+0000 7fe16a5e8640 1 --2- 172.21.15.7:0/1953538238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1641035e0 0x7fe16410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.176+0000 7fe16a5e8640 1 --2- 172.21.15.7:0/1953538238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe164102cc0 0x7fe1641030a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.176+0000 7fe16a5e8640 1 -- 172.21.15.7:0/1953538238 >> 172.21.15.7:0/1953538238 conn(0x7fe1640fc6b0 msgr2=0x7fe1640fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.176+0000 7fe16a5e8640 1 -- 172.21.15.7:0/1953538238 shutdown_connections 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.176+0000 7fe16a5e8640 1 -- 172.21.15.7:0/1953538238 wait complete. 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.176+0000 7fe16a5e8640 1 Processor -- start 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe16a5e8640 1 -- start start 2024-09-10T10:52:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe16a5e8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe164102cc0 0x7fe164100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe16a5e8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1641035e0 0x7fe164100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe16a5e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16410c110 0x7fe164101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe16a5e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe164111ef0 con 0x7fe16410c110 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe16a5e8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe164111d70 con 0x7fe164102cc0 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe16a5e8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe164112070 con 0x7fe1641035e0 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe168b85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16410c110 0x7fe164101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe15bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1641035e0 0x7fe164100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe15bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1641035e0 0x7fe164100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:59956/0 (socket says 172.21.15.7:59956) 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe168b85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16410c110 0x7fe164101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47246/0 (socket says 172.21.15.7:47246) 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe15bfff640 1 -- 172.21.15.7:0/3076590222 learned_addr learned my addr 172.21.15.7:0/3076590222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe163fff640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe164102cc0 0x7fe164100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe168b85640 1 -- 172.21.15.7:0/3076590222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1641035e0 msgr2=0x7fe164100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe168b85640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1641035e0 0x7fe164100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe168b85640 1 -- 172.21.15.7:0/3076590222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe164102cc0 msgr2=0x7fe164100910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe168b85640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe164102cc0 0x7fe164100910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe168b85640 1 -- 172.21.15.7:0/3076590222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1641a68e0 con 0x7fe16410c110 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.177+0000 7fe15bfff640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1641035e0 0x7fe164100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.178+0000 7fe168b85640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16410c110 0x7fe164101390 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00e940 tx=0x7fe15c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.178+0000 7fe161ffb640 1 -- 172.21.15.7:0/3076590222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c004d20 con 0x7fe16410c110 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.178+0000 7fe16a5e8640 1 -- 172.21.15.7:0/3076590222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1641a6bd0 con 0x7fe16410c110 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.178+0000 7fe161ffb640 1 -- 172.21.15.7:0/3076590222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c005020 con 0x7fe16410c110 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.178+0000 7fe161ffb640 1 -- 172.21.15.7:0/3076590222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0116b0 con 0x7fe16410c110 2024-09-10T10:52:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.178+0000 7fe16a5e8640 1 -- 172.21.15.7:0/3076590222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1641a7120 con 0x7fe16410c110 2024-09-10T10:52:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.180+0000 7fe161ffb640 1 -- 172.21.15.7:0/3076590222 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe15c0054d0 con 0x7fe16410c110 2024-09-10T10:52:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.180+0000 7fe16d45f740 1 Processor -- start 2024-09-10T10:52:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.180+0000 7fe16d45f740 1 -- start start 2024-09-10T10:52:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.180+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0e9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe16d45f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5642cd04e2f0 con 0x5642cd0f4cb0 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe16d45f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5642ccff6270 con 0x5642cd0e9af0 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe16d45f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5642cd04a2b0 con 0x5642cd0ea410 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe15bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe163fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe15bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47258/0 (socket says 172.21.15.7:47258) 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe15bfff640 1 -- 172.21.15.7:0/4169428684 learned_addr learned my addr 172.21.15.7:0/4169428684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe15bfff640 1 --2- 172.21.15.7:0/4169428684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0e9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe161ffb640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe1500a2da0 0x7fe1500a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe163fff640 1 -- 172.21.15.7:0/4169428684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 msgr2=0x5642cd0e9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe163fff640 1 --2- 172.21.15.7:0/4169428684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0e9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe163fff640 1 -- 172.21.15.7:0/4169428684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 msgr2=0x5642cd0f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe163fff640 1 --2- 172.21.15.7:0/4169428684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe163fff640 1 -- 172.21.15.7:0/4169428684 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642ccff5340 con 0x5642cd0ea410 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe168b85640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe1500a2da0 0x7fe1500a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe161ffb640 1 -- 172.21.15.7:0/3076590222 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7fe15c0cf470 con 0x7fe16410c110 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.181+0000 7fe163fff640 1 --2- 172.21.15.7:0/4169428684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f4770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe154005940 tx=0x7fe154005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b32ad1cab3be4aa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe15a7fc640 1 -- 172.21.15.7:0/4169428684 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154040070 con 0x5642cd0ea410 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe15a7fc640 1 -- 172.21.15.7:0/4169428684 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe154031870 con 0x5642cd0ea410 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe15a7fc640 1 -- 172.21.15.7:0/4169428684 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154035040 con 0x5642cd0ea410 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe16d45f740 1 -- 172.21.15.7:0/4169428684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 msgr2=0x5642cd0f4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe168b85640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe1500a2da0 0x7fe1500a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0ccc40 tx=0x7fe15c086040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe16d45f740 1 --2- 172.21.15.7:0/4169428684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f4770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe154005940 tx=0x7fe154005e00 comp rx=0 tx=0).stop 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe16d45f740 1 -- 172.21.15.7:0/4169428684 shutdown_connections 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe16d45f740 1 --2- 172.21.15.7:0/4169428684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe16d45f740 1 --2- 172.21.15.7:0/4169428684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f4770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe16d45f740 1 --2- 172.21.15.7:0/4169428684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0e9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe16d45f740 1 -- 172.21.15.7:0/4169428684 >> 172.21.15.7:0/4169428684 conn(0x5642cd0e6060 msgr2=0x5642cd0e6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe16d45f740 1 -- 172.21.15.7:0/4169428684 shutdown_connections 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.182+0000 7fe16d45f740 1 -- 172.21.15.7:0/4169428684 wait complete. 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe16d45f740 1 -- start start 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe16d45f740 1 Processor -- start 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0e9af0 0x5642cd0f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe16d45f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5642cd0f7880 con 0x5642cd0e9af0 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe16d45f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5642ccfd7290 con 0x5642cd0f4cb0 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe16d45f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5642cd0f7a00 con 0x5642cd0ea410 2024-09-10T10:52:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe15bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0e9af0 0x5642cd0f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe16d45f740 10 client.0 _ll_register_callbacks cb 0x5642cd008140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe15bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0e9af0 0x5642cd0f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47264/0 (socket says 172.21.15.7:47264) 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe15bfff640 1 -- 172.21.15.7:0/1787491565 learned_addr learned my addr 172.21.15.7:0/1787491565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe163fff640 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.183+0000 7fe15bfff640 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe15bfff640 1 -- 172.21.15.7:0/1787491565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 msgr2=0x5642cd0f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe15bfff640 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe15bfff640 1 -- 172.21.15.7:0/1787491565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0f4cb0 msgr2=0x5642cd0f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe15bfff640 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe15bfff640 1 -- 172.21.15.7:0/1787491565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ccd61210 con 0x5642cd0e9af0 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe15bfff640 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0e9af0 0x5642cd0f1b80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe14c011e80 tx=0x7fe14c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f305df4a411a498 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe1437fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:52:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe1437fe640 1 -- 172.21.15.7:0/1787491565 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c00e9c0 con 0x5642cd0e9af0 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe1437fe640 1 -- 172.21.15.7:0/1787491565 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7fe14c00eb60 con 0x5642cd0e9af0 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe16d45f740 1 -- 172.21.15.7:0/1787491565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe154031870 con 0x5642cd0e9af0 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe1437fe640 1 -- 172.21.15.7:0/1787491565 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe14c00ed60 con 0x5642cd0e9af0 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe16d45f740 10 client.15777 fetch_fsmap learned FSMap version 33 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe16d45f740 1 -- 172.21.15.7:0/1787491565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5642cce2d620 con 0x5642cd0e9af0 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.184+0000 7fe1437fe640 1 -- 172.21.15.7:0/1787491565 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe154031870 con 0x5642cd0e9af0 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.185+0000 7fe1437fe640 1 -- 172.21.15.7:0/1787491565 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fe14c01d030 con 0x5642cd0e9af0 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.185+0000 7fe16d45f740 10 client.15777 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.185+0000 7fe16d45f740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.185+0000 7fe16d45f740 20 client.15777 populate_metadata read hostname 'smithi007' 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.185+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5642cd104c20 0x5642cd107010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.185+0000 7fe16d45f740 4 client.15777 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.185+0000 7fe16d45f740 1 -- 172.21.15.7:0/1787491565 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642cd05ba20 con 0x5642cd104c20 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.185+0000 7fe163fff640 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5642cd104c20 0x5642cd107010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.185+0000 7fe163fff640 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5642cd104c20 0x5642cd107010 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a63325d73728acd1 server_cookie=e3055ee75f215ec6 in_seq=0 out_seq=0 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.185+0000 7fe1437fe640 10 client.15777 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe1437fe640 1 -- 172.21.15.7:0/1787491565 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642cd05ba20 con 0x5642cd104c20 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe1437fe640 10 client.15777 handle_command_reply: tid=0 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 client.15777 shutdown 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 -- 172.21.15.7:0/1787491565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0e9af0 msgr2=0x5642cd0f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0e9af0 0x5642cd0f1b80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe14c011e80 tx=0x7fe14c013040 comp rx=0 tx=0).stop 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 -- 172.21.15.7:0/1787491565 shutdown_connections 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5642cd104c20 0x5642cd107010 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1787491565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0e9af0 0x5642cd0f1b80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe1437fe640 0 client.15777 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 -- 172.21.15.7:0/1787491565 >> 172.21.15.7:0/1787491565 conn(0x5642cd0e6060 msgr2=0x5642cd0ebd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 -- 172.21.15.7:0/1787491565 shutdown_connections 2024-09-10T10:52:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 1 -- 172.21.15.7:0/1787491565 wait complete. 2024-09-10T10:52:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.186+0000 7fe16d45f740 20 client.15777 trim_cache size 0 max 16384 2024-09-10T10:52:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.202+0000 7fe16d45f740 1 Processor -- start 2024-09-10T10:52:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.202+0000 7fe16d45f740 1 -- start start 2024-09-10T10:52:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0ea410 0x5642cd10c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0f4cb0 0x5642cd10ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe16d45f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641b2370 con 0x5642cd0ea410 2024-09-10T10:52:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe16d45f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641afc10 con 0x5642cd0e9af0 2024-09-10T10:52:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe16d45f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641ada80 con 0x5642cd0f4cb0 2024-09-10T10:52:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe163fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0ea410 0x5642cd10c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe163fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0ea410 0x5642cd10c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47278/0 (socket says 172.21.15.7:47278) 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe163fff640 1 -- 172.21.15.7:0/674856766 learned_addr learned my addr 172.21.15.7:0/674856766 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe15bfff640 1 --2- 172.21.15.7:0/674856766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0f4cb0 0x5642cd10ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe15bfff640 1 --2- 172.21.15.7:0/674856766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe163fff640 1 -- 172.21.15.7:0/674856766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0f4cb0 msgr2=0x5642cd10ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe163fff640 1 --2- 172.21.15.7:0/674856766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0f4cb0 0x5642cd10ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe163fff640 1 -- 172.21.15.7:0/674856766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 msgr2=0x5642cd0f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe163fff640 1 --2- 172.21.15.7:0/674856766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe163fff640 1 -- 172.21.15.7:0/674856766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cd0f46d0 con 0x5642cd0ea410 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.203+0000 7fe163fff640 1 --2- 172.21.15.7:0/674856766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0ea410 0x5642cd10c8e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x5642cd0e6ec0 tx=0x7fe1540058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cf04345efe699f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe159ffb640 1 -- 172.21.15.7:0/674856766 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154040050 con 0x5642cd0ea410 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe159ffb640 1 -- 172.21.15.7:0/674856766 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1540035f0 con 0x5642cd0ea410 2024-09-10T10:52:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe159ffb640 1 -- 172.21.15.7:0/674856766 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154035040 con 0x5642cd0ea410 2024-09-10T10:52:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe16d45f740 1 -- 172.21.15.7:0/674856766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0ea410 msgr2=0x5642cd10c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe16d45f740 1 --2- 172.21.15.7:0/674856766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0ea410 0x5642cd10c8e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x5642cd0e6ec0 tx=0x7fe1540058d0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe16d45f740 1 -- 172.21.15.7:0/674856766 shutdown_connections 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe16d45f740 1 --2- 172.21.15.7:0/674856766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0f4cb0 0x5642cd10ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe16d45f740 1 --2- 172.21.15.7:0/674856766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0ea410 0x5642cd10c8e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe16d45f740 1 --2- 172.21.15.7:0/674856766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe16d45f740 1 -- 172.21.15.7:0/674856766 >> 172.21.15.7:0/674856766 conn(0x5642cd0e6060 msgr2=0x5642cd0f3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe16d45f740 1 -- 172.21.15.7:0/674856766 shutdown_connections 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.204+0000 7fe16d45f740 1 -- 172.21.15.7:0/674856766 wait complete. 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe16d45f740 1 -- start start 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe16d45f740 1 Processor -- start 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0fd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe16d45f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe16d45f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641ae0d0 con 0x5642cd0f4cb0 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe16d45f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641af540 con 0x5642cd0e9af0 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe16d45f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641ae7a0 con 0x5642cd0ea410 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe16d45f740 10 client.0 _ll_register_callbacks cb 0x7fe14c0116b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe15bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe163fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe15bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47282/0 (socket says 172.21.15.7:47282) 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe15bfff640 1 -- 172.21.15.7:0/1235439210 learned_addr learned my addr 172.21.15.7:0/1235439210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.205+0000 7fe15bfff640 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe15bfff640 1 -- 172.21.15.7:0/1235439210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 msgr2=0x5642cd0f8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe15bfff640 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe15bfff640 1 -- 172.21.15.7:0/1235439210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 msgr2=0x5642cd0fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe15bfff640 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe15bfff640 1 -- 172.21.15.7:0/1235439210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cd0f93c0 con 0x5642cd0f4cb0 2024-09-10T10:52:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe15bfff640 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f8b90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00e780 tx=0x7fe14c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a5334cd30310c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe1437fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:52:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe1437fe640 1 -- 172.21.15.7:0/1235439210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c02f040 con 0x5642cd0f4cb0 2024-09-10T10:52:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe16d45f740 1 -- 172.21.15.7:0/1235439210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1540035f0 con 0x5642cd0f4cb0 2024-09-10T10:52:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe1437fe640 1 -- 172.21.15.7:0/1235439210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7fe14c00e9c0 con 0x5642cd0f4cb0 2024-09-10T10:52:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe1437fe640 1 -- 172.21.15.7:0/1235439210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe14c00ebc0 con 0x5642cd0f4cb0 2024-09-10T10:52:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe16d45f740 10 client.15783 fetch_fsmap learned FSMap version 33 2024-09-10T10:52:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe16d45f740 1 -- 172.21.15.7:0/1235439210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5642cd16f7f0 con 0x5642cd0f4cb0 2024-09-10T10:52:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe1437fe640 1 -- 172.21.15.7:0/1235439210 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1540035f0 con 0x5642cd0f4cb0 2024-09-10T10:52:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.206+0000 7fe1437fe640 1 -- 172.21.15.7:0/1235439210 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fe14c03f070 con 0x5642cd0f4cb0 2024-09-10T10:52:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.207+0000 7fe16d45f740 10 client.15783 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:52:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.207+0000 7fe16d45f740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.207+0000 7fe16d45f740 20 client.15783 populate_metadata read hostname 'smithi007' 2024-09-10T10:52:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.207+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5642cd16fdd0 0x5642cd1721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.207+0000 7fe16d45f740 4 client.15783 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:52:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.207+0000 7fe16d45f740 1 -- 172.21.15.7:0/1235439210 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642cd055d70 con 0x5642cd16fdd0 2024-09-10T10:52:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.207+0000 7fe163fff640 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5642cd16fdd0 0x5642cd1721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.207+0000 7fe163fff640 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5642cd16fdd0 0x5642cd1721f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9c83e532c2e53 server_cookie=c1eda7e2ab9ec662 in_seq=0 out_seq=0 2024-09-10T10:52:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.207+0000 7fe1437fe640 10 client.15783 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.271+0000 7fe1437fe640 1 -- 172.21.15.7:0/1235439210 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642cd055d70 con 0x5642cd16fdd0 2024-09-10T10:52:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.271+0000 7fe1437fe640 10 client.15783 handle_command_reply: tid=0 2024-09-10T10:52:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.271+0000 7fe16d45f740 1 client.15783 shutdown 2024-09-10T10:52:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:52:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:52:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:52:34.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:52:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe16d45f740 1 -- 172.21.15.7:0/1235439210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 msgr2=0x5642cd0f8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f8b90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00e780 tx=0x7fe14c013040 comp rx=0 tx=0).stop 2024-09-10T10:52:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe16d45f740 1 -- 172.21.15.7:0/1235439210 shutdown_connections 2024-09-10T10:52:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5642cd16fdd0 0x5642cd1721f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe1437fe640 0 client.15783 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642cd0f4cb0 0x5642cd0f8b90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642cd0ea410 0x5642cd0f8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe16d45f740 1 --2- 172.21.15.7:0/1235439210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5642cd0e9af0 0x5642cd0fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe16d45f740 1 -- 172.21.15.7:0/1235439210 >> 172.21.15.7:0/1235439210 conn(0x5642cd0e6060 msgr2=0x5642cd0eb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe16d45f740 1 -- 172.21.15.7:0/1235439210 shutdown_connections 2024-09-10T10:52:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.272+0000 7fe16d45f740 1 -- 172.21.15.7:0/1235439210 wait complete. 2024-09-10T10:52:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.273+0000 7fe16d45f740 20 client.15783 trim_cache size 0 max 16384 2024-09-10T10:52:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 -- 172.21.15.7:0/3076590222 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe1500a2da0 msgr2=0x7fe1500a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe1500a2da0 0x7fe1500a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0ccc40 tx=0x7fe15c086040 comp rx=0 tx=0).stop 2024-09-10T10:52:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 -- 172.21.15.7:0/3076590222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16410c110 msgr2=0x7fe164101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16410c110 0x7fe164101390 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00e940 tx=0x7fe15c00ee00 comp rx=0 tx=0).stop 2024-09-10T10:52:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 -- 172.21.15.7:0/3076590222 shutdown_connections 2024-09-10T10:52:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fe1500a2da0 0x7fe1500a5260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16410c110 0x7fe164101390 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1641035e0 0x7fe164100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 --2- 172.21.15.7:0/3076590222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe164102cc0 0x7fe164100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 -- 172.21.15.7:0/3076590222 >> 172.21.15.7:0/3076590222 conn(0x7fe1640fc6b0 msgr2=0x7fe1640fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 -- 172.21.15.7:0/3076590222 shutdown_connections 2024-09-10T10:52:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:34.274+0000 7fe16a5e8640 1 -- 172.21.15.7:0/3076590222 wait complete. 2024-09-10T10:52:34.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 760171 11.95 MB/sec execute 214 sec latency 50.798 ms 2024-09-10T10:52:35.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 762743 11.95 MB/sec execute 215 sec latency 13.793 ms 2024-09-10T10:52:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:35 smithi103 ceph-mon[35160]: pgmap v207: 97 pgs: 97 active+clean; 198 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-09-10T10:52:35.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:35 smithi007 ceph-mon[30873]: pgmap v207: 97 pgs: 97 active+clean; 198 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-09-10T10:52:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:35 smithi133 ceph-mon[35282]: pgmap v207: 97 pgs: 97 active+clean; 198 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-09-10T10:52:36.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 764627 11.94 MB/sec execute 216 sec latency 10.383 ms 2024-09-10T10:52:37.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 767264 11.97 MB/sec execute 217 sec latency 25.886 ms 2024-09-10T10:52:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:37 smithi103 ceph-mon[35160]: pgmap v208: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 433 op/s 2024-09-10T10:52:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:37 smithi133 ceph-mon[35282]: pgmap v208: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 433 op/s 2024-09-10T10:52:37.976 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:37 smithi007 ceph-mon[30873]: pgmap v208: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 433 op/s 2024-09-10T10:52:38.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 769873 11.97 MB/sec execute 218 sec latency 12.433 ms 2024-09-10T10:52:39.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 771095 11.95 MB/sec execute 219 sec latency 195.018 ms 2024-09-10T10:52:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:39 smithi103 ceph-mon[35160]: pgmap v209: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 292 op/s 2024-09-10T10:52:39.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:39 smithi007 ceph-mon[30873]: pgmap v209: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 292 op/s 2024-09-10T10:52:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:39 smithi133 ceph-mon[35282]: pgmap v209: 97 pgs: 97 active+clean; 190 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 292 op/s 2024-09-10T10:52:40.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 771095 11.89 MB/sec execute 220 sec latency 1195.140 ms 2024-09-10T10:52:41.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 771095 11.84 MB/sec execute 221 sec latency 2195.238 ms 2024-09-10T10:52:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:41 smithi103 ceph-mon[35160]: pgmap v210: 97 pgs: 97 active+clean; 197 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 415 op/s 2024-09-10T10:52:41.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:41 smithi007 ceph-mon[30873]: pgmap v210: 97 pgs: 97 active+clean; 197 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 415 op/s 2024-09-10T10:52:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:41 smithi133 ceph-mon[35282]: pgmap v210: 97 pgs: 97 active+clean; 197 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 415 op/s 2024-09-10T10:52:42.518 INFO:tasks.workunit.client.0.smithi007.stdout: 1 771095 11.78 MB/sec execute 222 sec latency 3195.361 ms 2024-09-10T10:52:43.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 771095 11.73 MB/sec execute 223 sec latency 4195.479 ms 2024-09-10T10:52:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:43 smithi103 ceph-mon[35160]: pgmap v211: 97 pgs: 97 active+clean; 197 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-09-10T10:52:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:43 smithi007 ceph-mon[30873]: pgmap v211: 97 pgs: 97 active+clean; 197 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-09-10T10:52:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:43 smithi133 ceph-mon[35282]: pgmap v211: 97 pgs: 97 active+clean; 197 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-09-10T10:52:44.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 771095 11.68 MB/sec execute 224 sec latency 5195.617 ms 2024-09-10T10:52:45.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 771095 11.63 MB/sec execute 225 sec latency 6195.729 ms 2024-09-10T10:52:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:45 smithi103 ceph-mon[35160]: pgmap v212: 97 pgs: 97 active+clean; 197 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-09-10T10:52:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:45 smithi007 ceph-mon[30873]: pgmap v212: 97 pgs: 97 active+clean; 197 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-09-10T10:52:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:45 smithi133 ceph-mon[35282]: pgmap v212: 97 pgs: 97 active+clean; 197 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 287 op/s 2024-09-10T10:52:46.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 771687 11.58 MB/sec execute 226 sec latency 6957.072 ms 2024-09-10T10:52:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.371+0000 7fb758ff6640 1 -- 172.21.15.7:0/2192755407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100860 msgr2=0x7fb754100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.371+0000 7fb758ff6640 1 --2- 172.21.15.7:0/2192755407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100860 0x7fb754100c40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb744009ca0 tx=0x7fb74402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.371+0000 7fb758ff6640 1 -- 172.21.15.7:0/2192755407 shutdown_connections 2024-09-10T10:52:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.371+0000 7fb758ff6640 1 --2- 172.21.15.7:0/2192755407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb75410e410 0x7fb754110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.371+0000 7fb758ff6640 1 --2- 172.21.15.7:0/2192755407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb754101210 0x7fb75410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.371+0000 7fb758ff6640 1 --2- 172.21.15.7:0/2192755407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754100860 0x7fb754100c40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.371+0000 7fb758ff6640 1 -- 172.21.15.7:0/2192755407 >> 172.21.15.7:0/2192755407 conn(0x7fb754077990 msgr2=0x7fb7540fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.371+0000 7fb758ff6640 1 -- 172.21.15.7:0/2192755407 shutdown_connections 2024-09-10T10:52:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.371+0000 7fb758ff6640 1 -- 172.21.15.7:0/2192755407 wait complete. 2024-09-10T10:52:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb758ff6640 1 Processor -- start 2024-09-10T10:52:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb758ff6640 1 -- start start 2024-09-10T10:52:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb758ff6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754100860 0x7fb75419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb758ff6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754101210 0x7fb75419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb758ff6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb75410e410 0x7fb75419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb758ff6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb754114080 con 0x7fb754101210 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb758ff6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb754113f00 con 0x7fb754100860 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb758ff6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb754114200 con 0x7fb75410e410 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb751d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754101210 0x7fb75419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb751d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754101210 0x7fb75419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43536/0 (socket says 172.21.15.7:43536) 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb751d9b640 1 -- 172.21.15.7:0/1147444014 learned_addr learned my addr 172.21.15.7:0/1147444014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb752d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb75410e410 0x7fb75419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.372+0000 7fb75259c640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754100860 0x7fb75419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb751d9b640 1 -- 172.21.15.7:0/1147444014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb75410e410 msgr2=0x7fb75419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb751d9b640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb75410e410 0x7fb75419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb751d9b640 1 -- 172.21.15.7:0/1147444014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754100860 msgr2=0x7fb75419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb751d9b640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754100860 0x7fb75419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb751d9b640 1 -- 172.21.15.7:0/1147444014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75419f580 con 0x7fb754101210 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb752d9d640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb75410e410 0x7fb75419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb75259c640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754100860 0x7fb75419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb751d9b640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754101210 0x7fb75419aad0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00e9f0 tx=0x7fb73c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb74b7fe640 1 -- 172.21.15.7:0/1147444014 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c00cd80 con 0x7fb754101210 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb74b7fe640 1 -- 172.21.15.7:0/1147444014 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb73c005500 con 0x7fb754101210 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb758ff6640 1 -- 172.21.15.7:0/1147444014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75419f870 con 0x7fb754101210 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb74b7fe640 1 -- 172.21.15.7:0/1147444014 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c010690 con 0x7fb754101210 2024-09-10T10:52:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.373+0000 7fb758ff6640 1 -- 172.21.15.7:0/1147444014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7541a7100 con 0x7fb754101210 2024-09-10T10:52:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb75be6c740 1 Processor -- start 2024-09-10T10:52:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb74b7fe640 1 -- 172.21.15.7:0/1147444014 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb73c0046c0 con 0x7fb754101210 2024-09-10T10:52:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb75be6c740 1 -- start start 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb517caf0 0x55aeb517ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 0x55aeb5187770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb5187cb0 0x55aeb518a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb75be6c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aeb50e12f0 con 0x55aeb517caf0 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb75be6c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aeb5089270 con 0x55aeb5187cb0 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb75be6c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55aeb50dd2b0 con 0x55aeb517d410 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb75259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 0x55aeb5187770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb751d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb5187cb0 0x55aeb518a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb75259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 0x55aeb5187770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:47662/0 (socket says 172.21.15.7:47662) 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.376+0000 7fb752d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb517caf0 0x55aeb517ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb75259c640 1 -- 172.21.15.7:0/1214602705 learned_addr learned my addr 172.21.15.7:0/1214602705 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb75259c640 1 -- 172.21.15.7:0/1214602705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb5187cb0 msgr2=0x55aeb518a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb75259c640 1 --2- 172.21.15.7:0/1214602705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb5187cb0 0x55aeb518a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb75259c640 1 -- 172.21.15.7:0/1214602705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb517caf0 msgr2=0x55aeb517ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb75259c640 1 --2- 172.21.15.7:0/1214602705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb517caf0 0x55aeb517ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb75259c640 1 -- 172.21.15.7:0/1214602705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeb5088340 con 0x55aeb517d410 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb752d9d640 1 --2- 172.21.15.7:0/1214602705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb517caf0 0x55aeb517ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb74b7fe640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb7400a2e20 0x7fb7400a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb75259c640 1 --2- 172.21.15.7:0/1214602705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 0x55aeb5187770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb744009bf0 tx=0x7fb744005f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d04876d149d421b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb752d9d640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb7400a2e20 0x7fb7400a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb74b7fe640 1 -- 172.21.15.7:0/1147444014 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7fb73c0cf800 con 0x7fb754101210 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb748ff9640 1 -- 172.21.15.7:0/1214602705 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744049070 con 0x55aeb517d410 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb748ff9640 1 -- 172.21.15.7:0/1214602705 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb744036040 con 0x55aeb517d410 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.377+0000 7fb748ff9640 1 -- 172.21.15.7:0/1214602705 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74402fe40 con 0x55aeb517d410 2024-09-10T10:52:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.378+0000 7fb75be6c740 1 -- 172.21.15.7:0/1214602705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 msgr2=0x55aeb5187770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.378+0000 7fb75be6c740 1 --2- 172.21.15.7:0/1214602705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 0x55aeb5187770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb744009bf0 tx=0x7fb744005f60 comp rx=0 tx=0).stop 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.378+0000 7fb752d9d640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb7400a2e20 0x7fb7400a52e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009fd0 tx=0x7fb74c005f90 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.378+0000 7fb75be6c740 1 -- 172.21.15.7:0/1214602705 shutdown_connections 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.378+0000 7fb75be6c740 1 --2- 172.21.15.7:0/1214602705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb5187cb0 0x55aeb518a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.378+0000 7fb75be6c740 1 --2- 172.21.15.7:0/1214602705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 0x55aeb5187770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.378+0000 7fb75be6c740 1 --2- 172.21.15.7:0/1214602705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb517caf0 0x55aeb517ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.378+0000 7fb75be6c740 1 -- 172.21.15.7:0/1214602705 >> 172.21.15.7:0/1214602705 conn(0x55aeb5179060 msgr2=0x55aeb5179440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.378+0000 7fb75be6c740 1 -- 172.21.15.7:0/1214602705 shutdown_connections 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.378+0000 7fb75be6c740 1 -- 172.21.15.7:0/1214602705 wait complete. 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75be6c740 1 -- start start 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75be6c740 1 Processor -- start 2024-09-10T10:52:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 0x55aeb5184be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 0x55aeb5185120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb5185660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75be6c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55aeb518a700 con 0x55aeb5187cb0 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75be6c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aeb506a290 con 0x55aeb517d410 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75be6c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55aeb518a880 con 0x55aeb517caf0 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75be6c740 10 client.0 _ll_register_callbacks cb 0x55aeb509b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 0x55aeb5184be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb752d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb5185660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 0x55aeb5184be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:47678/0 (socket says 172.21.15.7:47678) 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb751d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 0x55aeb5185120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.379+0000 7fb75259c640 1 -- 172.21.15.7:0/3015351519 learned_addr learned my addr 172.21.15.7:0/3015351519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb752d9d640 1 -- 172.21.15.7:0/3015351519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 msgr2=0x55aeb5184be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb752d9d640 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 0x55aeb5184be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb752d9d640 1 -- 172.21.15.7:0/3015351519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 msgr2=0x55aeb5185120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb752d9d640 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 0x55aeb5185120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb752d9d640 1 -- 172.21.15.7:0/3015351519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeb4df4210 con 0x55aeb5187cb0 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb751d9b640 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 0x55aeb5185120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb752d9d640 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb5185660 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb74c012500 tx=0x7fb74c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405587c45d33d33e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb739ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb739ffb640 1 -- 172.21.15.7:0/3015351519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c004030 con 0x55aeb5187cb0 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb739ffb640 1 -- 172.21.15.7:0/3015351519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7fb74c0041d0 con 0x55aeb5187cb0 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb75be6c740 1 -- 172.21.15.7:0/3015351519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb744036040 con 0x55aeb5187cb0 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb739ffb640 1 -- 172.21.15.7:0/3015351519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb74c0043d0 con 0x55aeb5187cb0 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb75be6c740 10 client.15795 fetch_fsmap learned FSMap version 33 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb75be6c740 1 -- 172.21.15.7:0/3015351519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aeb4ec0620 con 0x55aeb5187cb0 2024-09-10T10:52:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.380+0000 7fb739ffb640 1 -- 172.21.15.7:0/3015351519 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb744036040 con 0x55aeb5187cb0 2024-09-10T10:52:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.381+0000 7fb739ffb640 1 -- 172.21.15.7:0/3015351519 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fb74c003b20 con 0x55aeb5187cb0 2024-09-10T10:52:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.381+0000 7fb75be6c740 10 client.15795 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:52:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.381+0000 7fb75be6c740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.381+0000 7fb75be6c740 20 client.15795 populate_metadata read hostname 'smithi007' 2024-09-10T10:52:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.381+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55aeb5197b20 0x55aeb5199f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.381+0000 7fb75be6c740 4 client.15795 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.381+0000 7fb75be6c740 1 -- 172.21.15.7:0/3015351519 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aeb50eea20 con 0x55aeb5197b20 2024-09-10T10:52:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.381+0000 7fb75259c640 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55aeb5197b20 0x55aeb5199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.381+0000 7fb75259c640 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55aeb5197b20 0x55aeb5199f50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d5223cb421d934 server_cookie=7847e2cd0dd839e1 in_seq=0 out_seq=0 2024-09-10T10:52:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.381+0000 7fb739ffb640 10 client.15795 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb739ffb640 1 -- 172.21.15.7:0/3015351519 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aeb50eea20 con 0x55aeb5197b20 2024-09-10T10:52:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb739ffb640 10 client.15795 handle_command_reply: tid=0 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 client.15795 shutdown 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 -- 172.21.15.7:0/3015351519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 msgr2=0x55aeb5185660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb5185660 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb74c012500 tx=0x7fb74c004730 comp rx=0 tx=0).stop 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 -- 172.21.15.7:0/3015351519 shutdown_connections 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55aeb5197b20 0x55aeb5199f50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb5185660 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb739ffb640 0 client.15795 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 0x55aeb5185120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3015351519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 0x55aeb5184be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 -- 172.21.15.7:0/3015351519 >> 172.21.15.7:0/3015351519 conn(0x55aeb5179060 msgr2=0x55aeb517edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 -- 172.21.15.7:0/3015351519 shutdown_connections 2024-09-10T10:52:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 1 -- 172.21.15.7:0/3015351519 wait complete. 2024-09-10T10:52:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.382+0000 7fb75be6c740 20 client.15795 trim_cache size 0 max 16384 2024-09-10T10:52:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.398+0000 7fb75be6c740 1 Processor -- start 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.398+0000 7fb75be6c740 1 -- start start 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.398+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517caf0 0x55aeb517f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 0x55aeb5185490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb519f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb75be6c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541b21d0 con 0x55aeb5187cb0 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb75be6c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541afa70 con 0x55aeb517caf0 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb75be6c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541ad8e0 con 0x55aeb517d410 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb752d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb519f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb752d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb519f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43562/0 (socket says 172.21.15.7:43562) 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb752d9d640 1 -- 172.21.15.7:0/1575435478 learned_addr learned my addr 172.21.15.7:0/1575435478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb751d9b640 1 --2- 172.21.15.7:0/1575435478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 0x55aeb5185490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb75259c640 1 --2- 172.21.15.7:0/1575435478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517caf0 0x55aeb517f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb752d9d640 1 -- 172.21.15.7:0/1575435478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 msgr2=0x55aeb5185490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb752d9d640 1 --2- 172.21.15.7:0/1575435478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 0x55aeb5185490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb752d9d640 1 -- 172.21.15.7:0/1575435478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517caf0 msgr2=0x55aeb517f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb752d9d640 1 --2- 172.21.15.7:0/1575435478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517caf0 0x55aeb517f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb752d9d640 1 -- 172.21.15.7:0/1575435478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aeb5183110 con 0x55aeb5187cb0 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb752d9d640 1 --2- 172.21.15.7:0/1575435478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb519f860 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0058c0 tx=0x7fb74c00f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2ad65a1a1307b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb73bfff640 1 -- 172.21.15.7:0/1575435478 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c035030 con 0x55aeb5187cb0 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb73bfff640 1 -- 172.21.15.7:0/1575435478 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74c011d30 con 0x55aeb5187cb0 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.399+0000 7fb73bfff640 1 -- 172.21.15.7:0/1575435478 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0039e0 con 0x55aeb5187cb0 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.400+0000 7fb75be6c740 1 -- 172.21.15.7:0/1575435478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 msgr2=0x55aeb519f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.400+0000 7fb75be6c740 1 --2- 172.21.15.7:0/1575435478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb519f860 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0058c0 tx=0x7fb74c00f410 comp rx=0 tx=0).stop 2024-09-10T10:52:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.400+0000 7fb75be6c740 1 -- 172.21.15.7:0/1575435478 shutdown_connections 2024-09-10T10:52:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.400+0000 7fb75be6c740 1 --2- 172.21.15.7:0/1575435478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb519f860 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.400+0000 7fb75be6c740 1 --2- 172.21.15.7:0/1575435478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517d410 0x55aeb5185490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.400+0000 7fb75be6c740 1 --2- 172.21.15.7:0/1575435478 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517caf0 0x55aeb517f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.400+0000 7fb75be6c740 1 -- 172.21.15.7:0/1575435478 >> 172.21.15.7:0/1575435478 conn(0x55aeb5179060 msgr2=0x55aeb5186c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.400+0000 7fb75be6c740 1 -- 172.21.15.7:0/1575435478 shutdown_connections 2024-09-10T10:52:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.400+0000 7fb75be6c740 1 -- 172.21.15.7:0/1575435478 wait complete. 2024-09-10T10:52:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb75be6c740 1 -- start start 2024-09-10T10:52:47.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb75be6c740 1 Processor -- start 2024-09-10T10:52:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 0x55aeb518fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 0x55aeb5190050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb75be6c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb518cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb75be6c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541adf30 con 0x55aeb5187cb0 2024-09-10T10:52:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb75be6c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541af3a0 con 0x55aeb517d410 2024-09-10T10:52:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb75be6c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7541ae600 con 0x55aeb517caf0 2024-09-10T10:52:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb75be6c740 10 client.0 _ll_register_callbacks cb 0x7fb74c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb752d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb518cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb752d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb518cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43578/0 (socket says 172.21.15.7:43578) 2024-09-10T10:52:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb752d9d640 1 -- 172.21.15.7:0/3618209763 learned_addr learned my addr 172.21.15.7:0/3618209763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb751d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 0x55aeb5190050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb75259c640 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 0x55aeb518fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb752d9d640 1 -- 172.21.15.7:0/3618209763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 msgr2=0x55aeb518fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb752d9d640 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 0x55aeb518fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb752d9d640 1 -- 172.21.15.7:0/3618209763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 msgr2=0x55aeb5190050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb752d9d640 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 0x55aeb5190050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb752d9d640 1 -- 172.21.15.7:0/3618209763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aeb5190690 con 0x55aeb5187cb0 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.401+0000 7fb751d9b640 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 0x55aeb5190050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb752d9d640 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb518cae0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb74c019040 tx=0x7fb74c0041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248e133c44e9ec23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb739ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb739ffb640 1 -- 172.21.15.7:0/3618209763 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c03e020 con 0x55aeb5187cb0 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb739ffb640 1 -- 172.21.15.7:0/3618209763 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7fb74c005040 con 0x55aeb5187cb0 2024-09-10T10:52:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb75be6c740 1 -- 172.21.15.7:0/3618209763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb74c011d30 con 0x55aeb5187cb0 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb739ffb640 1 -- 172.21.15.7:0/3618209763 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb74c049050 con 0x55aeb5187cb0 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb75be6c740 10 client.15801 fetch_fsmap learned FSMap version 33 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb75be6c740 1 -- 172.21.15.7:0/3618209763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aeb5202d80 con 0x55aeb5187cb0 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb739ffb640 1 -- 172.21.15.7:0/3618209763 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74c011d30 con 0x55aeb5187cb0 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb739ffb640 1 -- 172.21.15.7:0/3618209763 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7fb74c004570 con 0x55aeb5187cb0 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb75be6c740 10 client.15801 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb75be6c740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb75be6c740 20 client.15801 populate_metadata read hostname 'smithi007' 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.402+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55aeb52033d0 0x55aeb52057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.403+0000 7fb75be6c740 4 client.15801 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.403+0000 7fb75be6c740 1 -- 172.21.15.7:0/3618209763 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aeb50e8d70 con 0x55aeb52033d0 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.403+0000 7fb75259c640 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55aeb52033d0 0x55aeb52057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.403+0000 7fb739ffb640 10 client.15801 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.403+0000 7fb75259c640 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55aeb52033d0 0x55aeb52057c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579e971a7b2b9da2 server_cookie=8e89af2e28738b0c in_seq=0 out_seq=0 2024-09-10T10:52:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb739ffb640 1 -- 172.21.15.7:0/3618209763 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aeb50e8d70 con 0x55aeb52033d0 2024-09-10T10:52:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb739ffb640 10 client.15801 handle_command_reply: tid=0 2024-09-10T10:52:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 client.15801 shutdown 2024-09-10T10:52:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 -- 172.21.15.7:0/3618209763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 msgr2=0x55aeb518cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb518cae0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb74c019040 tx=0x7fb74c0041d0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 -- 172.21.15.7:0/3618209763 shutdown_connections 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55aeb52033d0 0x55aeb52057c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55aeb5187cb0 0x55aeb518cae0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb739ffb640 0 client.15801 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aeb517d410 0x55aeb5190050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 --2- 172.21.15.7:0/3618209763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aeb517caf0 0x55aeb518fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 -- 172.21.15.7:0/3618209763 >> 172.21.15.7:0/3618209763 conn(0x55aeb5179060 msgr2=0x55aeb51779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 -- 172.21.15.7:0/3618209763 shutdown_connections 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 1 -- 172.21.15.7:0/3618209763 wait complete. 2024-09-10T10:52:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.449+0000 7fb75be6c740 20 client.15801 trim_cache size 0 max 16384 2024-09-10T10:52:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:52:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:52:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:52:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:52:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 -- 172.21.15.7:0/1147444014 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb7400a2e20 msgr2=0x7fb7400a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb7400a2e20 0x7fb7400a52e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009fd0 tx=0x7fb74c005f90 comp rx=0 tx=0).stop 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 -- 172.21.15.7:0/1147444014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754101210 msgr2=0x7fb75419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754101210 0x7fb75419aad0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00e9f0 tx=0x7fb73c00edf0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 -- 172.21.15.7:0/1147444014 shutdown_connections 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb75410e410 0x7fb75419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb7400a2e20 0x7fb7400a52e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb754101210 0x7fb75419aad0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 --2- 172.21.15.7:0/1147444014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb754100860 0x7fb75419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 -- 172.21.15.7:0/1147444014 >> 172.21.15.7:0/1147444014 conn(0x7fb754077990 msgr2=0x7fb7540ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 -- 172.21.15.7:0/1147444014 shutdown_connections 2024-09-10T10:52:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:52:47.450+0000 7fb758ff6640 1 -- 172.21.15.7:0/1147444014 wait complete. 2024-09-10T10:52:47.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 774268 11.61 MB/sec execute 227 sec latency 40.684 ms 2024-09-10T10:52:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:47 smithi103 ceph-mon[35160]: pgmap v213: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 310 op/s 2024-09-10T10:52:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:47 smithi007 ceph-mon[30873]: pgmap v213: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 310 op/s 2024-09-10T10:52:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:47 smithi133 ceph-mon[35282]: pgmap v213: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 310 op/s 2024-09-10T10:52:48.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 775910 11.58 MB/sec execute 228 sec latency 17.456 ms 2024-09-10T10:52:49.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 778024 11.59 MB/sec execute 229 sec latency 12.271 ms 2024-09-10T10:52:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:49 smithi103 ceph-mon[35160]: pgmap v214: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 2.3 MiB/s wr, 145 op/s 2024-09-10T10:52:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:49 smithi007 ceph-mon[30873]: pgmap v214: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 2.3 MiB/s wr, 145 op/s 2024-09-10T10:52:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:49 smithi133 ceph-mon[35282]: pgmap v214: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 2.3 MiB/s wr, 145 op/s 2024-09-10T10:52:50.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 780490 11.59 MB/sec execute 230 sec latency 16.612 ms 2024-09-10T10:52:51.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 781928 11.57 MB/sec execute 231 sec latency 50.295 ms 2024-09-10T10:52:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:51 smithi103 ceph-mon[35160]: pgmap v215: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 266 op/s 2024-09-10T10:52:51.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:51 smithi007 ceph-mon[30873]: pgmap v215: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 266 op/s 2024-09-10T10:52:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:51 smithi133 ceph-mon[35282]: pgmap v215: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 266 op/s 2024-09-10T10:52:52.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 784513 11.59 MB/sec execute 232 sec latency 16.122 ms 2024-09-10T10:52:53.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 786212 11.58 MB/sec execute 233 sec latency 15.553 ms 2024-09-10T10:52:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:53 smithi103 ceph-mon[35160]: pgmap v216: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-10T10:52:53.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:53 smithi007 ceph-mon[30873]: pgmap v216: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-10T10:52:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:53 smithi133 ceph-mon[35282]: pgmap v216: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-10T10:52:54.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 788538 11.59 MB/sec execute 234 sec latency 37.992 ms 2024-09-10T10:52:55.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 790617 11.57 MB/sec execute 235 sec latency 25.320 ms 2024-09-10T10:52:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:55 smithi103 ceph-mon[35160]: pgmap v217: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-10T10:52:55.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:55 smithi007 ceph-mon[30873]: pgmap v217: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-10T10:52:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:55 smithi133 ceph-mon[35282]: pgmap v217: 97 pgs: 97 active+clean; 199 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-09-10T10:52:56.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 792351 11.57 MB/sec execute 236 sec latency 17.059 ms 2024-09-10T10:52:57.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 794978 11.58 MB/sec execute 237 sec latency 18.696 ms 2024-09-10T10:52:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:57 smithi103 ceph-mon[35160]: pgmap v218: 97 pgs: 97 active+clean; 207 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-10T10:52:57.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:57 smithi007 ceph-mon[30873]: pgmap v218: 97 pgs: 97 active+clean; 207 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-10T10:52:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:57 smithi133 ceph-mon[35282]: pgmap v218: 97 pgs: 97 active+clean; 207 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-10T10:52:58.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 796809 11.58 MB/sec execute 238 sec latency 15.128 ms 2024-09-10T10:52:59.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 799087 11.58 MB/sec execute 239 sec latency 34.150 ms 2024-09-10T10:52:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:52:59 smithi103 ceph-mon[35160]: pgmap v219: 97 pgs: 97 active+clean; 207 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-09-10T10:52:59.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:52:59 smithi007 ceph-mon[30873]: pgmap v219: 97 pgs: 97 active+clean; 207 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-09-10T10:52:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:52:59 smithi133 ceph-mon[35282]: pgmap v219: 97 pgs: 97 active+clean; 207 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-09-10T10:53:00.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 800723 11.56 MB/sec execute 240 sec latency 28.862 ms 2024-09-10T10:53:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.542+0000 7f9f27eb7640 1 -- 172.21.15.7:0/1683163094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f20101390 msgr2=0x7f9f2010d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.542+0000 7f9f27eb7640 1 --2- 172.21.15.7:0/1683163094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f20101390 0x7f9f2010d8c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009cd0 tx=0x7f9f1802a8e0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.542+0000 7f9f27eb7640 1 -- 172.21.15.7:0/1683163094 shutdown_connections 2024-09-10T10:53:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.542+0000 7f9f27eb7640 1 --2- 172.21.15.7:0/1683163094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f20101390 0x7f9f2010d8c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.542+0000 7f9f27eb7640 1 --2- 172.21.15.7:0/1683163094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f201009f0 0x7f9f20100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.542+0000 7f9f27eb7640 1 --2- 172.21.15.7:0/1683163094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f20106a00 0x7f9f20106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.542+0000 7f9f27eb7640 1 -- 172.21.15.7:0/1683163094 >> 172.21.15.7:0/1683163094 conn(0x7f9f200fc6b0 msgr2=0x7f9f200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.542+0000 7f9f27eb7640 1 -- 172.21.15.7:0/1683163094 shutdown_connections 2024-09-10T10:53:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.542+0000 7f9f27eb7640 1 -- 172.21.15.7:0/1683163094 wait complete. 2024-09-10T10:53:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f27eb7640 1 Processor -- start 2024-09-10T10:53:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f27eb7640 1 -- start start 2024-09-10T10:53:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f27eb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f201009f0 0x7f9f20199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f27eb7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f20101390 0x7f9f201996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f27eb7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20106a00 0x7f9f201a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f27eb7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20110e40 con 0x7f9f20101390 2024-09-10T10:53:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f27eb7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20110cc0 con 0x7f9f20106a00 2024-09-10T10:53:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f27eb7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20110fc0 con 0x7f9f201009f0 2024-09-10T10:53:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f25452640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f20101390 0x7f9f201996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f25452640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f20101390 0x7f9f201996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45222/0 (socket says 172.21.15.7:45222) 2024-09-10T10:53:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f25c53640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f201009f0 0x7f9f20199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f25452640 1 -- 172.21.15.7:0/843114498 learned_addr learned my addr 172.21.15.7:0/843114498 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f26454640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20106a00 0x7f9f201a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f25452640 1 -- 172.21.15.7:0/843114498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f201009f0 msgr2=0x7f9f20199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f25452640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f201009f0 0x7f9f20199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f25452640 1 -- 172.21.15.7:0/843114498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20106a00 msgr2=0x7f9f201a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f25452640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20106a00 0x7f9f201a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.543+0000 7f9f25452640 1 -- 172.21.15.7:0/843114498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f201a4750 con 0x7f9f20101390 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.544+0000 7f9f25452640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f20101390 0x7f9f201996d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000e9f0 tx=0x7f9f1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.544+0000 7f9f16ffd640 1 -- 172.21.15.7:0/843114498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1000cd80 con 0x7f9f20101390 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.544+0000 7f9f16ffd640 1 -- 172.21.15.7:0/843114498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f10010910 con 0x7f9f20101390 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.544+0000 7f9f16ffd640 1 -- 172.21.15.7:0/843114498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10018990 con 0x7f9f20101390 2024-09-10T10:53:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.544+0000 7f9f27eb7640 1 -- 172.21.15.7:0/843114498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f201a4920 con 0x7f9f20101390 2024-09-10T10:53:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.544+0000 7f9f27eb7640 1 -- 172.21.15.7:0/843114498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f201a4e40 con 0x7f9f20101390 2024-09-10T10:53:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.546+0000 7f9f2ac4f740 1 Processor -- start 2024-09-10T10:53:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f2ac4f740 1 -- start start 2024-09-10T10:53:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f16ffd640 1 -- 172.21.15.7:0/843114498 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9f10010ac0 con 0x7f9f20101390 2024-09-10T10:53:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423743db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f423752b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f4237554b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4236a82f0 con 0x55f4237439d0 2024-09-10T10:53:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f423650270 con 0x55f4237530c0 2024-09-10T10:53:00.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f4236a42b0 con 0x55f4237442f0 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f26454640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423743db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f25c53640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f423752b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f25452640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f4237554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f26454640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423743db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45236/0 (socket says 172.21.15.7:45236) 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f26454640 1 -- 172.21.15.7:0/625321056 learned_addr learned my addr 172.21.15.7:0/625321056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f25c53640 1 -- 172.21.15.7:0/625321056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 msgr2=0x55f4237554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f25c53640 1 --2- 172.21.15.7:0/625321056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f4237554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f25c53640 1 -- 172.21.15.7:0/625321056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 msgr2=0x55f423743db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f25c53640 1 --2- 172.21.15.7:0/625321056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423743db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.547+0000 7f9f25c53640 1 -- 172.21.15.7:0/625321056 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f42364f340 con 0x55f4237442f0 2024-09-10T10:53:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.548+0000 7f9f16ffd640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9f0c0a2da0 0x7f9f0c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.548+0000 7f9f25c53640 1 --2- 172.21.15.7:0/625321056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f423752b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009c00 tx=0x7f9f1c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87b57a2917fc6d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.548+0000 7f9f26454640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9f0c0a2da0 0x7f9f0c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.548+0000 7f9f16ffd640 1 -- 172.21.15.7:0/843114498 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7f9f100cf770 con 0x7f9f20101390 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.548+0000 7f9f0bfff640 1 -- 172.21.15.7:0/625321056 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c004050 con 0x55f4237442f0 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.548+0000 7f9f0bfff640 1 -- 172.21.15.7:0/625321056 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1c005500 con 0x55f4237442f0 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.548+0000 7f9f0bfff640 1 -- 172.21.15.7:0/625321056 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c004e00 con 0x55f4237442f0 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/625321056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 msgr2=0x55f423752b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/625321056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f423752b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009c00 tx=0x7f9f1c011810 comp rx=0 tx=0).stop 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f26454640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9f0c0a2da0 0x7f9f0c0a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009a60 tx=0x7f9f18004a40 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/625321056 shutdown_connections 2024-09-10T10:53:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/625321056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f4237554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/625321056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f423752b80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/625321056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423743db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/625321056 >> 172.21.15.7:0/625321056 conn(0x55f423740060 msgr2=0x55f423740440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/625321056 shutdown_connections 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/625321056 wait complete. 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 -- start start 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.549+0000 7f9f2ac4f740 1 Processor -- start 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42374ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f4237504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f423750a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f423755cf0 con 0x55f4237439d0 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f423631290 con 0x55f4237530c0 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f423755e70 con 0x55f4237442f0 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f25c53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42374ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f25c53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42374ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45250/0 (socket says 172.21.15.7:45250) 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f25c53640 1 -- 172.21.15.7:0/940952598 learned_addr learned my addr 172.21.15.7:0/940952598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f2ac4f740 10 client.0 _ll_register_callbacks cb 0x55f423662140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f25452640 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f4237504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f26454640 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f423750a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f25452640 1 -- 172.21.15.7:0/940952598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 msgr2=0x55f423750a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f25452640 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f423750a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f25452640 1 -- 172.21.15.7:0/940952598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 msgr2=0x55f42374ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f25452640 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42374ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f25452640 1 -- 172.21.15.7:0/940952598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4233bb210 con 0x55f4237442f0 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.550+0000 7f9f25c53640 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42374ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f25452640 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f4237504e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f100cce90 tx=0x7f9f10009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4329944430e4f325 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f097fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f097fa640 1 -- 172.21.15.7:0/940952598 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1009b670 con 0x55f4237442f0 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f097fa640 1 -- 172.21.15.7:0/940952598 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7f9f1009b160 con 0x55f4237442f0 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/940952598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f1c005500 con 0x55f4237442f0 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f097fa640 1 -- 172.21.15.7:0/940952598 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9f1009bd10 con 0x55f4237442f0 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f2ac4f740 10 client.25501 fetch_fsmap learned FSMap version 33 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/940952598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f423487620 con 0x55f4237442f0 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f097fa640 1 -- 172.21.15.7:0/940952598 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1c005500 con 0x55f4237442f0 2024-09-10T10:53:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f097fa640 1 -- 172.21.15.7:0/940952598 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f9f100b4020 con 0x55f4237442f0 2024-09-10T10:53:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f2ac4f740 10 client.25501 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:53:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f2ac4f740 10 client.25501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f2ac4f740 20 client.25501 populate_metadata read hostname 'smithi007' 2024-09-10T10:53:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f4237630c0 0x55f423765510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f2ac4f740 4 client.25501 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.551+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/940952598 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4236b5a20 con 0x55f4237630c0 2024-09-10T10:53:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.552+0000 7f9f25c53640 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f4237630c0 0x55f423765510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.552+0000 7f9f25c53640 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f4237630c0 0x55f423765510 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4caab4c883dc30b2 server_cookie=c8915bbe5bfa1d24 in_seq=0 out_seq=0 2024-09-10T10:53:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.552+0000 7f9f097fa640 10 client.25501 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:00.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f097fa640 1 -- 172.21.15.7:0/940952598 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4236b5a20 con 0x55f4237630c0 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f097fa640 10 client.25501 handle_command_reply: tid=0 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 client.25501 shutdown 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/940952598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 msgr2=0x55f4237504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f4237504e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9f100cce90 tx=0x7f9f10009310 comp rx=0 tx=0).stop 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/940952598 shutdown_connections 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f4237630c0 0x55f423765510 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f423750a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f4237504e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f097fa640 0 client.25501 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/940952598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42374ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/940952598 >> 172.21.15.7:0/940952598 conn(0x55f423740060 msgr2=0x55f423745d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/940952598 shutdown_connections 2024-09-10T10:53:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/940952598 wait complete. 2024-09-10T10:53:00.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.553+0000 7f9f2ac4f740 20 client.25501 trim_cache size 0 max 16384 2024-09-10T10:53:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f2ac4f740 1 Processor -- start 2024-09-10T10:53:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f2ac4f740 1 -- start start 2024-09-10T10:53:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423751f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f42376ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f42376b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201adf70 con 0x55f4237439d0 2024-09-10T10:53:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201ab810 con 0x55f4237530c0 2024-09-10T10:53:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201a96a0 con 0x55f4237442f0 2024-09-10T10:53:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f25c53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423751f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f25c53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423751f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45254/0 (socket says 172.21.15.7:45254) 2024-09-10T10:53:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f25c53640 1 -- 172.21.15.7:0/1813745481 learned_addr learned my addr 172.21.15.7:0/1813745481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f26454640 1 --2- 172.21.15.7:0/1813745481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f42376b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.569+0000 7f9f25452640 1 --2- 172.21.15.7:0/1813745481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f42376ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.570+0000 7f9f25c53640 1 -- 172.21.15.7:0/1813745481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 msgr2=0x55f42376ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.570+0000 7f9f25c53640 1 --2- 172.21.15.7:0/1813745481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f42376ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.570+0000 7f9f25c53640 1 -- 172.21.15.7:0/1813745481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 msgr2=0x55f42376b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.570+0000 7f9f25c53640 1 --2- 172.21.15.7:0/1813745481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f42376b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.570+0000 7f9f25c53640 1 -- 172.21.15.7:0/1813745481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f423746500 con 0x55f4237439d0 2024-09-10T10:53:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.570+0000 7f9f25c53640 1 --2- 172.21.15.7:0/1813745481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423751f10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c014040 tx=0x7f9f1c0049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36da21c9fa58ddf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.570+0000 7f9f0b7fe640 1 -- 172.21.15.7:0/1813745481 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c0047c0 con 0x55f4237439d0 2024-09-10T10:53:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.570+0000 7f9f0b7fe640 1 -- 172.21.15.7:0/1813745481 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1c004360 con 0x55f4237439d0 2024-09-10T10:53:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.570+0000 7f9f0b7fe640 1 -- 172.21.15.7:0/1813745481 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c02e610 con 0x55f4237439d0 2024-09-10T10:53:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.571+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/1813745481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 msgr2=0x55f423751f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.571+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/1813745481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423751f10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c014040 tx=0x7f9f1c0049e0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.571+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/1813745481 shutdown_connections 2024-09-10T10:53:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.571+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/1813745481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237530c0 0x55f42376b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.571+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/1813745481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237442f0 0x55f42376ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.571+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/1813745481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f423751f10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.571+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/1813745481 >> 172.21.15.7:0/1813745481 conn(0x55f423740060 msgr2=0x55f423751800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.571+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/1813745481 shutdown_connections 2024-09-10T10:53:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.571+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/1813745481 wait complete. 2024-09-10T10:53:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.571+0000 7f9f2ac4f740 1 -- start start 2024-09-10T10:53:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f2ac4f740 1 Processor -- start 2024-09-10T10:53:00.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42375b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237442f0 0x55f423756d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f2ac4f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237530c0 0x55f423757490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201a9cf0 con 0x55f4237439d0 2024-09-10T10:53:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201ab140 con 0x55f4237442f0 2024-09-10T10:53:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f2ac4f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9f201aa3a0 con 0x55f4237530c0 2024-09-10T10:53:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f2ac4f740 10 client.0 _ll_register_callbacks cb 0x7f9f100d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f25c53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42375b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f25c53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42375b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45268/0 (socket says 172.21.15.7:45268) 2024-09-10T10:53:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f25c53640 1 -- 172.21.15.7:0/2661015657 learned_addr learned my addr 172.21.15.7:0/2661015657 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f25452640 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237442f0 0x55f423756d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f26454640 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237530c0 0x55f423757490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f25c53640 1 -- 172.21.15.7:0/2661015657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237530c0 msgr2=0x55f423757490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f25c53640 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237530c0 0x55f423757490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f25c53640 1 -- 172.21.15.7:0/2661015657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237442f0 msgr2=0x55f423756d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f25c53640 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237442f0 0x55f423756d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.572+0000 7f9f25c53640 1 -- 172.21.15.7:0/2661015657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4237ccdf0 con 0x55f4237439d0 2024-09-10T10:53:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.573+0000 7f9f25c53640 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42375b9e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c005e00 tx=0x7f9f1c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f5ebf650d5c158 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.573+0000 7f9f097fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.573+0000 7f9f097fa640 1 -- 172.21.15.7:0/2661015657 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c035070 con 0x55f4237439d0 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.573+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/2661015657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f1c004360 con 0x55f4237439d0 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.573+0000 7f9f097fa640 1 -- 172.21.15.7:0/2661015657 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8564+0+0 (secure 0 0 0) 0x7f9f1c0035f0 con 0x55f4237439d0 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.573+0000 7f9f097fa640 1 -- 172.21.15.7:0/2661015657 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9f1c003850 con 0x55f4237439d0 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.573+0000 7f9f2ac4f740 10 client.15819 fetch_fsmap learned FSMap version 33 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.573+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/2661015657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f4237cdfe0 con 0x55f4237439d0 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.573+0000 7f9f097fa640 1 -- 172.21.15.7:0/2661015657 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1c004360 con 0x55f4237439d0 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.574+0000 7f9f097fa640 1 -- 172.21.15.7:0/2661015657 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f9f1c041440 con 0x55f4237439d0 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.574+0000 7f9f2ac4f740 10 client.15819 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.574+0000 7f9f2ac4f740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.574+0000 7f9f2ac4f740 20 client.15819 populate_metadata read hostname 'smithi007' 2024-09-10T10:53:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.574+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f4237ce4f0 0x55f4237d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.574+0000 7f9f2ac4f740 4 client.15819 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:53:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.574+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/2661015657 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4236afd70 con 0x55f4237ce4f0 2024-09-10T10:53:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.574+0000 7f9f25452640 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f4237ce4f0 0x55f4237d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.574+0000 7f9f25452640 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f4237ce4f0 0x55f4237d08e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e62c31a626320a38 server_cookie=89971df29a041846 in_seq=0 out_seq=0 2024-09-10T10:53:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.574+0000 7f9f097fa640 10 client.15819 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f097fa640 1 -- 172.21.15.7:0/2661015657 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4236afd70 con 0x55f4237ce4f0 2024-09-10T10:53:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f097fa640 10 client.15819 handle_command_reply: tid=0 2024-09-10T10:53:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 client.15819 shutdown 2024-09-10T10:53:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/2661015657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 msgr2=0x55f42375b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42375b9e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c005e00 tx=0x7f9f1c009310 comp rx=0 tx=0).stop 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/2661015657 shutdown_connections 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f4237ce4f0 0x55f4237d08e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f4237530c0 0x55f423757490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4237442f0 0x55f423756d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 --2- 172.21.15.7:0/2661015657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f4237439d0 0x55f42375b9e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f097fa640 0 client.15819 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/2661015657 >> 172.21.15.7:0/2661015657 conn(0x55f423740060 msgr2=0x55f423745d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/2661015657 shutdown_connections 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.633+0000 7f9f2ac4f740 1 -- 172.21.15.7:0/2661015657 wait complete. 2024-09-10T10:53:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f2ac4f740 20 client.15819 trim_cache size 0 max 16384 2024-09-10T10:53:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:53:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:53:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:53:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:53:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f27eb7640 1 -- 172.21.15.7:0/843114498 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9f0c0a2da0 msgr2=0x7f9f0c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f27eb7640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9f0c0a2da0 0x7f9f0c0a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009a60 tx=0x7f9f18004a40 comp rx=0 tx=0).stop 2024-09-10T10:53:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f27eb7640 1 -- 172.21.15.7:0/843114498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f20101390 msgr2=0x7f9f201996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f27eb7640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f20101390 0x7f9f201996d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9f1000e9f0 tx=0x7f9f1000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f27eb7640 1 -- 172.21.15.7:0/843114498 shutdown_connections 2024-09-10T10:53:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f27eb7640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f20106a00 0x7f9f201a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f27eb7640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9f0c0a2da0 0x7f9f0c0a5260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f27eb7640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f20101390 0x7f9f201996d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f27eb7640 1 --2- 172.21.15.7:0/843114498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f201009f0 0x7f9f20199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.634+0000 7f9f27eb7640 1 -- 172.21.15.7:0/843114498 >> 172.21.15.7:0/843114498 conn(0x7f9f200fc6b0 msgr2=0x7f9f200fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.635+0000 7f9f27eb7640 1 -- 172.21.15.7:0/843114498 shutdown_connections 2024-09-10T10:53:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:00.636+0000 7f9f27eb7640 1 -- 172.21.15.7:0/843114498 wait complete. 2024-09-10T10:53:01.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 802559 11.56 MB/sec execute 241 sec latency 34.114 ms 2024-09-10T10:53:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:01 smithi103 ceph-mon[35160]: pgmap v220: 97 pgs: 97 active+clean; 214 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 354 op/s 2024-09-10T10:53:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:01 smithi007 ceph-mon[30873]: pgmap v220: 97 pgs: 97 active+clean; 214 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 354 op/s 2024-09-10T10:53:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:01 smithi133 ceph-mon[35282]: pgmap v220: 97 pgs: 97 active+clean; 214 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 354 op/s 2024-09-10T10:53:02.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 804106 11.54 MB/sec execute 242 sec latency 21.384 ms 2024-09-10T10:53:02.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:02 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:53:02.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:02 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:53:02.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:02 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:53:02.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:02 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:53:02.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:02 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:53:02.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:02 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:53:02.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:02 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:53:02.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:02 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:53:02.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:02 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:53:02.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:02 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:53:02.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:02 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:53:02.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:02 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:53:03.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 806380 11.55 MB/sec execute 243 sec latency 13.436 ms 2024-09-10T10:53:03.876 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:03 smithi133 ceph-mon[35282]: pgmap v221: 97 pgs: 97 active+clean; 214 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2024-09-10T10:53:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:03 smithi103 ceph-mon[35160]: pgmap v221: 97 pgs: 97 active+clean; 214 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2024-09-10T10:53:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:03 smithi007 ceph-mon[30873]: pgmap v221: 97 pgs: 97 active+clean; 214 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 233 op/s 2024-09-10T10:53:04.519 INFO:tasks.workunit.client.0.smithi007.stdout: 1 807339 11.53 MB/sec execute 244 sec latency 33.227 ms 2024-09-10T10:53:04.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:04 smithi103 ceph-mon[35160]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T10:53:04.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:04 smithi007 ceph-mon[30873]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T10:53:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:04 smithi133 ceph-mon[35282]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T10:53:05.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 809697 11.53 MB/sec execute 245 sec latency 24.611 ms 2024-09-10T10:53:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:05 smithi103 ceph-mon[35160]: pgmap v223: 97 pgs: 97 active+clean; 214 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 280 op/s 2024-09-10T10:53:05.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:05 smithi007 ceph-mon[30873]: pgmap v223: 97 pgs: 97 active+clean; 214 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 280 op/s 2024-09-10T10:53:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:05 smithi133 ceph-mon[35282]: pgmap v223: 97 pgs: 97 active+clean; 214 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 280 op/s 2024-09-10T10:53:06.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 811363 11.51 MB/sec execute 246 sec latency 14.812 ms 2024-09-10T10:53:07.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 813475 11.52 MB/sec execute 247 sec latency 16.257 ms 2024-09-10T10:53:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:07 smithi103 ceph-mon[35160]: pgmap v224: 97 pgs: 97 active+clean; 213 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 364 op/s 2024-09-10T10:53:07.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:07 smithi007 ceph-mon[30873]: pgmap v224: 97 pgs: 97 active+clean; 213 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 364 op/s 2024-09-10T10:53:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:07 smithi133 ceph-mon[35282]: pgmap v224: 97 pgs: 97 active+clean; 213 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 364 op/s 2024-09-10T10:53:08.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 815822 11.52 MB/sec execute 248 sec latency 14.716 ms 2024-09-10T10:53:09.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 817260 11.50 MB/sec execute 249 sec latency 26.860 ms 2024-09-10T10:53:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:09 smithi103 ceph-mon[35160]: pgmap v225: 97 pgs: 97 active+clean; 213 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 364 op/s 2024-09-10T10:53:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:09 smithi103 ceph-mon[35160]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T10:53:09.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:09 smithi007 ceph-mon[30873]: pgmap v225: 97 pgs: 97 active+clean; 213 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 364 op/s 2024-09-10T10:53:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:09 smithi007 ceph-mon[30873]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T10:53:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:09 smithi133 ceph-mon[35282]: pgmap v225: 97 pgs: 97 active+clean; 213 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 364 op/s 2024-09-10T10:53:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:09 smithi133 ceph-mon[35282]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T10:53:10.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 819759 11.51 MB/sec execute 250 sec latency 15.595 ms 2024-09-10T10:53:11.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 820802 11.49 MB/sec execute 251 sec latency 209.762 ms 2024-09-10T10:53:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:11 smithi103 ceph-mon[35160]: pgmap v227: 97 pgs: 97 active+clean; 221 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 450 op/s 2024-09-10T10:53:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:11 smithi007 ceph-mon[30873]: pgmap v227: 97 pgs: 97 active+clean; 221 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 450 op/s 2024-09-10T10:53:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:11 smithi133 ceph-mon[35282]: pgmap v227: 97 pgs: 97 active+clean; 221 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 450 op/s 2024-09-10T10:53:12.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 823343 11.50 MB/sec execute 252 sec latency 28.879 ms 2024-09-10T10:53:13.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 824848 11.48 MB/sec execute 253 sec latency 17.943 ms 2024-09-10T10:53:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.727+0000 7f4e97827640 1 -- 172.21.15.7:0/3743965872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e90101250 msgr2=0x7f4e90107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.727+0000 7f4e97827640 1 --2- 172.21.15.7:0/3743965872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e90101250 0x7f4e90107990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009860 tx=0x7f4e8002a340 comp rx=0 tx=0).stop 2024-09-10T10:53:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.727+0000 7f4e97827640 1 -- 172.21.15.7:0/3743965872 shutdown_connections 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.727+0000 7f4e97827640 1 --2- 172.21.15.7:0/3743965872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e901080f0 0x7f4e9010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.727+0000 7f4e97827640 1 --2- 172.21.15.7:0/3743965872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e90101250 0x7f4e90107990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.727+0000 7f4e97827640 1 --2- 172.21.15.7:0/3743965872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e901008a0 0x7f4e90100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.727+0000 7f4e97827640 1 -- 172.21.15.7:0/3743965872 >> 172.21.15.7:0/3743965872 conn(0x7f4e90075ef0 msgr2=0x7f4e90076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.727+0000 7f4e97827640 1 -- 172.21.15.7:0/3743965872 shutdown_connections 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.727+0000 7f4e97827640 1 -- 172.21.15.7:0/3743965872 wait complete. 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.727+0000 7f4e97827640 1 Processor -- start 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e97827640 1 -- start start 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e97827640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e901008a0 0x7f4e901983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e97827640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e901080f0 0x7f4e90198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e97827640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e9019cca0 0x7f4e9019d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e97827640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90111f20 con 0x7f4e9019cca0 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e97827640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90111da0 con 0x7f4e901080f0 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e97827640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4e901120a0 con 0x7f4e901008a0 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e955c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e901008a0 0x7f4e901983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e955c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e901008a0 0x7f4e901983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:35582/0 (socket says 172.21.15.7:35582) 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e955c3640 1 -- 172.21.15.7:0/3721487300 learned_addr learned my addr 172.21.15.7:0/3721487300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e95dc4640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e9019cca0 0x7f4e9019d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e901080f0 0x7f4e90198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e955c3640 1 -- 172.21.15.7:0/3721487300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e901080f0 msgr2=0x7f4e90198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e955c3640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e901080f0 0x7f4e90198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e955c3640 1 -- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e9019cca0 msgr2=0x7f4e9019d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e955c3640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e9019cca0 0x7f4e9019d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e955c3640 1 -- 172.21.15.7:0/3721487300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9019d790 con 0x7f4e901008a0 2024-09-10T10:53:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.728+0000 7f4e95dc4640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e9019cca0 0x7f4e9019d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.729+0000 7f4e955c3640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e901008a0 0x7f4e901983e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00d770 tx=0x7f4e8c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.729+0000 7f4e867fc640 1 -- 172.21.15.7:0/3721487300 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c017070 con 0x7f4e901008a0 2024-09-10T10:53:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.729+0000 7f4e867fc640 1 -- 172.21.15.7:0/3721487300 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e8c005500 con 0x7f4e901008a0 2024-09-10T10:53:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.729+0000 7f4e97827640 1 -- 172.21.15.7:0/3721487300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9019da80 con 0x7f4e901008a0 2024-09-10T10:53:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.729+0000 7f4e867fc640 1 -- 172.21.15.7:0/3721487300 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c004e00 con 0x7f4e901008a0 2024-09-10T10:53:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.729+0000 7f4e97827640 1 -- 172.21.15.7:0/3721487300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e901a5190 con 0x7f4e901008a0 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.731+0000 7f4e867fc640 1 -- 172.21.15.7:0/3721487300 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4e8c004fa0 con 0x7f4e901008a0 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.731+0000 7f4e9a5c5740 1 Processor -- start 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.731+0000 7f4e9a5c5740 1 -- start start 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbcaf0 0x55a53fdbced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdc7cb0 0x55a53fdca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a53fd212f0 con 0x55a53fdbcaf0 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a53fcc9270 con 0x55a53fdc7cb0 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a53fd1d2b0 con 0x55a53fdbd410 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e867fc640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4e7c0a2da0 0x7f4e7c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e95dc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbcaf0 0x55a53fdbced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e955c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e94dc2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdc7cb0 0x55a53fdca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e955c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:35592/0 (socket says 172.21.15.7:35592) 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e955c3640 1 -- 172.21.15.7:0/4018489251 learned_addr learned my addr 172.21.15.7:0/4018489251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e867fc640 1 -- 172.21.15.7:0/3721487300 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f4e8c0cee50 con 0x7f4e901008a0 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e95dc4640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4e7c0a2da0 0x7f4e7c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e94dc2640 1 -- 172.21.15.7:0/4018489251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 msgr2=0x55a53fdc7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/4018489251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e94dc2640 1 -- 172.21.15.7:0/4018489251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbcaf0 msgr2=0x55a53fdbced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/4018489251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbcaf0 0x55a53fdbced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e94dc2640 1 -- 172.21.15.7:0/4018489251 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53fcc8340 con 0x55a53fdc7cb0 2024-09-10T10:53:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e955c3640 1 --2- 172.21.15.7:0/4018489251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/4018489251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdc7cb0 0x55a53fdca0a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4e80035040 tx=0x7f4e80009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4044ea33d803233 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.732+0000 7f4e95dc4640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4e7c0a2da0 0x7f4e7c0a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009c50 tx=0x7f4e88004ab0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e7b7fe640 1 -- 172.21.15.7:0/4018489251 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80037040 con 0x55a53fdc7cb0 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e7b7fe640 1 -- 172.21.15.7:0/4018489251 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e80030e50 con 0x55a53fdc7cb0 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e7b7fe640 1 -- 172.21.15.7:0/4018489251 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e800037f0 con 0x55a53fdc7cb0 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/4018489251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdc7cb0 msgr2=0x55a53fdca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/4018489251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdc7cb0 0x55a53fdca0a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4e80035040 tx=0x7f4e80009510 comp rx=0 tx=0).stop 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/4018489251 shutdown_connections 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/4018489251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdc7cb0 0x55a53fdca0a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/4018489251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/4018489251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbcaf0 0x55a53fdbced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/4018489251 >> 172.21.15.7:0/4018489251 conn(0x55a53fdb9060 msgr2=0x55a53fdb9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/4018489251 shutdown_connections 2024-09-10T10:53:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.733+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/4018489251 wait complete. 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.734+0000 7f4e9a5c5740 1 -- start start 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.734+0000 7f4e9a5c5740 1 Processor -- start 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.734+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdc4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.734+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.734+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fdc5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.734+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a53fdca880 con 0x55a53fdc7cb0 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.734+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a53fcaa290 con 0x55a53fdbcaf0 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.734+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a53fdcaa00 con 0x55a53fdbd410 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.734+0000 7f4e9a5c5740 10 client.0 _ll_register_callbacks cb 0x55a53fcdb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.734+0000 7f4e95dc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fdc5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e94dc2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e955c3640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdc4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e95dc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fdc5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39890/0 (socket says 172.21.15.7:39890) 2024-09-10T10:53:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e95dc4640 1 -- 172.21.15.7:0/369085629 learned_addr learned my addr 172.21.15.7:0/369085629 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e94dc2640 1 -- 172.21.15.7:0/369085629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 msgr2=0x55a53fdc4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdc4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e94dc2640 1 -- 172.21.15.7:0/369085629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 msgr2=0x55a53fdc5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fdc5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e94dc2640 1 -- 172.21.15.7:0/369085629 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53fa34210 con 0x55a53fdbd410 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e95dc4640 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fdc5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc5040 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e80031000 tx=0x7f4e80005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7bd607ec5224f7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e78ff9640 1 -- 172.21.15.7:0/369085629 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8004b070 con 0x55a53fdbd410 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/369085629 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e80030e50 con 0x55a53fdbd410 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e78ff9640 1 -- 172.21.15.7:0/369085629 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f4e80004250 con 0x55a53fdbd410 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e78ff9640 1 -- 172.21.15.7:0/369085629 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4e80057050 con 0x55a53fdbd410 2024-09-10T10:53:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e9a5c5740 10 client.25519 fetch_fsmap learned FSMap version 33 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.735+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/369085629 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a53fb00620 con 0x55a53fdbd410 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e78ff9640 1 -- 172.21.15.7:0/369085629 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e80030e50 con 0x55a53fdbd410 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e78ff9640 1 -- 172.21.15.7:0/369085629 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f4e80032600 con 0x55a53fdbd410 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e9a5c5740 10 client.25519 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e9a5c5740 10 client.25519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e9a5c5740 20 client.25519 populate_metadata read hostname 'smithi007' 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a53fdd7a00 0x55a53fdd9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e9a5c5740 4 client.25519 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/369085629 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a53fd2ea20 con 0x55a53fdd7a00 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e955c3640 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a53fdd7a00 0x55a53fdd9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e955c3640 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a53fdd7a00 0x55a53fdd9df0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9fdc929014edc69 server_cookie=3c0a73d6b44d4209 in_seq=0 out_seq=0 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.736+0000 7f4e78ff9640 10 client.25519 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e78ff9640 1 -- 172.21.15.7:0/369085629 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a53fd2ea20 con 0x55a53fdd7a00 2024-09-10T10:53:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e78ff9640 10 client.25519 handle_command_reply: tid=0 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 client.25519 shutdown 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/369085629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 msgr2=0x55a53fdc5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc5040 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e80031000 tx=0x7f4e80005b20 comp rx=0 tx=0).stop 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/369085629 shutdown_connections 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a53fdd7a00 0x55a53fdd9df0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fdc5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fdc5040 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/369085629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdc4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e78ff9640 0 client.25519 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/369085629 >> 172.21.15.7:0/369085629 conn(0x55a53fdb9060 msgr2=0x55a53fdbedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/369085629 shutdown_connections 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.737+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/369085629 wait complete. 2024-09-10T10:53:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.738+0000 7f4e9a5c5740 20 client.25519 trim_cache size 0 max 16384 2024-09-10T10:53:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e9a5c5740 1 Processor -- start 2024-09-10T10:53:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e9a5c5740 1 -- start start 2024-09-10T10:53:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdc53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fddf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fddf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e901b0020 con 0x55a53fdc7cb0 2024-09-10T10:53:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e901ad8c0 con 0x55a53fdbcaf0 2024-09-10T10:53:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4e901ab730 con 0x55a53fdbd410 2024-09-10T10:53:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e95dc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fddf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e95dc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fddf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39906/0 (socket says 172.21.15.7:39906) 2024-09-10T10:53:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e95dc4640 1 -- 172.21.15.7:0/1013873178 learned_addr learned my addr 172.21.15.7:0/1013873178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e955c3640 1 --2- 172.21.15.7:0/1013873178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdc53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/1013873178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fddf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e95dc4640 1 -- 172.21.15.7:0/1013873178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 msgr2=0x55a53fddf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e95dc4640 1 --2- 172.21.15.7:0/1013873178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fddf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e95dc4640 1 -- 172.21.15.7:0/1013873178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 msgr2=0x55a53fdc53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e95dc4640 1 --2- 172.21.15.7:0/1013873178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdc53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e95dc4640 1 -- 172.21.15.7:0/1013873178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53fdb8e90 con 0x55a53fdc7cb0 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.753+0000 7f4e95dc4640 1 --2- 172.21.15.7:0/1013873178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fddf960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4e8801e6c0 tx=0x7f4e88011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77aef599a0424de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e7affd640 1 -- 172.21.15.7:0/1013873178 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88017050 con 0x55a53fdc7cb0 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e7affd640 1 -- 172.21.15.7:0/1013873178 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e88018040 con 0x55a53fdc7cb0 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e7affd640 1 -- 172.21.15.7:0/1013873178 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88004600 con 0x55a53fdc7cb0 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/1013873178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 msgr2=0x55a53fddf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/1013873178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fddf960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4e8801e6c0 tx=0x7f4e88011000 comp rx=0 tx=0).stop 2024-09-10T10:53:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/1013873178 shutdown_connections 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/1013873178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdc7cb0 0x55a53fddf960 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/1013873178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdbd410 0x55a53fddf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/1013873178 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdc53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/1013873178 >> 172.21.15.7:0/1013873178 conn(0x55a53fdb9060 msgr2=0x55a53fdc6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/1013873178 shutdown_connections 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/1013873178 wait complete. 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.754+0000 7f4e9a5c5740 1 -- start start 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e9a5c5740 1 Processor -- start 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdd04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbd410 0x55a53fdcb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e9a5c5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdc7cb0 0x55a53fdcbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e901abd80 con 0x55a53fdbd410 2024-09-10T10:53:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e901ad1f0 con 0x55a53fdbcaf0 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e9a5c5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4e901ac450 con 0x55a53fdc7cb0 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e9a5c5740 10 client.0 _ll_register_callbacks cb 0x7f4e8c097050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e94dc2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbd410 0x55a53fdcb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e94dc2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbd410 0x55a53fdcb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39920/0 (socket says 172.21.15.7:39920) 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e94dc2640 1 -- 172.21.15.7:0/3279517580 learned_addr learned my addr 172.21.15.7:0/3279517580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e955c3640 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdd04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e95dc4640 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdc7cb0 0x55a53fdcbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e94dc2640 1 -- 172.21.15.7:0/3279517580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdc7cb0 msgr2=0x55a53fdcbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdc7cb0 0x55a53fdcbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e94dc2640 1 -- 172.21.15.7:0/3279517580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 msgr2=0x55a53fdd04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdd04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.755+0000 7f4e94dc2640 1 -- 172.21.15.7:0/3279517580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53fdcc600 con 0x55a53fdbd410 2024-09-10T10:53:13.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e94dc2640 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbd410 0x55a53fdcb7e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4e80005e00 tx=0x7f4e800058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f278c9d5e636a4b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e78ff9640 1 -- 172.21.15.7:0/3279517580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8004b040 con 0x55a53fdbd410 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e78ff9640 1 -- 172.21.15.7:0/3279517580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f4e80004450 con 0x55a53fdbd410 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/3279517580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e88018040 con 0x55a53fdbd410 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e78ff9640 1 -- 172.21.15.7:0/3279517580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4e80062050 con 0x55a53fdbd410 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e9a5c5740 10 client.15837 fetch_fsmap learned FSMap version 33 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/3279517580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a53fe468f0 con 0x55a53fdbd410 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e78ff9640 1 -- 172.21.15.7:0/3279517580 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e88018040 con 0x55a53fdbd410 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e78ff9640 1 -- 172.21.15.7:0/3279517580 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f4e80003a80 con 0x55a53fdbd410 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e9a5c5740 10 client.15837 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e9a5c5740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e9a5c5740 20 client.15837 populate_metadata read hostname 'smithi007' 2024-09-10T10:53:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a53fe46fe0 0x55a53fe49400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e9a5c5740 4 client.15837 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:53:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.756+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/3279517580 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a53fd28d70 con 0x55a53fe46fe0 2024-09-10T10:53:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.757+0000 7f4e955c3640 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a53fe46fe0 0x55a53fe49400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.757+0000 7f4e78ff9640 10 client.15837 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.757+0000 7f4e955c3640 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a53fe46fe0 0x55a53fe49400 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8b637d143666e6 server_cookie=277750fe76f050d9 in_seq=0 out_seq=0 2024-09-10T10:53:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.823+0000 7f4e78ff9640 1 -- 172.21.15.7:0/3279517580 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a53fd28d70 con 0x55a53fe46fe0 2024-09-10T10:53:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.823+0000 7f4e78ff9640 10 client.15837 handle_command_reply: tid=0 2024-09-10T10:53:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.823+0000 7f4e9a5c5740 1 client.15837 shutdown 2024-09-10T10:53:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.824+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/3279517580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbd410 msgr2=0x55a53fdcb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.824+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbd410 0x55a53fdcb7e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4e80005e00 tx=0x7f4e800058c0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.824+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/3279517580 shutdown_connections 2024-09-10T10:53:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.824+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a53fe46fe0 0x55a53fe49400 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.824+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a53fdc7cb0 0x55a53fdcbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.824+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a53fdbd410 0x55a53fdcb7e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.824+0000 7f4e9a5c5740 1 --2- 172.21.15.7:0/3279517580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a53fdbcaf0 0x55a53fdd04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.824+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/3279517580 >> 172.21.15.7:0/3279517580 conn(0x55a53fdb9060 msgr2=0x55a53fdbe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.824+0000 7f4e78ff9640 0 client.15837 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.824+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/3279517580 shutdown_connections 2024-09-10T10:53:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.825+0000 7f4e9a5c5740 1 -- 172.21.15.7:0/3279517580 wait complete. 2024-09-10T10:53:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.826+0000 7f4e9a5c5740 20 client.15837 trim_cache size 0 max 16384 2024-09-10T10:53:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:53:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:53:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:53:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:53:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 -- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4e7c0a2da0 msgr2=0x7f4e7c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4e7c0a2da0 0x7f4e7c0a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009c50 tx=0x7f4e88004ab0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 -- 172.21.15.7:0/3721487300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e901008a0 msgr2=0x7f4e901983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e901008a0 0x7f4e901983e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00d770 tx=0x7f4e8c00dc30 comp rx=0 tx=0).stop 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 -- 172.21.15.7:0/3721487300 shutdown_connections 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4e7c0a2da0 0x7f4e7c0a5260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e9019cca0 0x7f4e9019d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e901080f0 0x7f4e90198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 --2- 172.21.15.7:0/3721487300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e901008a0 0x7f4e901983e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 -- 172.21.15.7:0/3721487300 >> 172.21.15.7:0/3721487300 conn(0x7f4e90075ef0 msgr2=0x7f4e90105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 -- 172.21.15.7:0/3721487300 shutdown_connections 2024-09-10T10:53:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:13.827+0000 7f4e97827640 1 -- 172.21.15.7:0/3721487300 wait complete. 2024-09-10T10:53:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:13 smithi103 ceph-mon[35160]: pgmap v228: 97 pgs: 97 active+clean; 221 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 434 op/s 2024-09-10T10:53:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:13 smithi007 ceph-mon[30873]: pgmap v228: 97 pgs: 97 active+clean; 221 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 434 op/s 2024-09-10T10:53:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:13 smithi133 ceph-mon[35282]: pgmap v228: 97 pgs: 97 active+clean; 221 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 434 op/s 2024-09-10T10:53:14.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 827240 11.50 MB/sec execute 254 sec latency 33.433 ms 2024-09-10T10:53:15.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 828785 11.48 MB/sec execute 255 sec latency 14.165 ms 2024-09-10T10:53:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:15 smithi103 ceph-mon[35160]: pgmap v229: 97 pgs: 97 active+clean; 221 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 360 op/s 2024-09-10T10:53:15.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:15 smithi007 ceph-mon[30873]: pgmap v229: 97 pgs: 97 active+clean; 221 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 360 op/s 2024-09-10T10:53:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:15 smithi133 ceph-mon[35282]: pgmap v229: 97 pgs: 97 active+clean; 221 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 360 op/s 2024-09-10T10:53:16.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 831043 11.49 MB/sec execute 256 sec latency 15.950 ms 2024-09-10T10:53:17.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 832959 11.47 MB/sec execute 257 sec latency 17.714 ms 2024-09-10T10:53:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:17 smithi103 ceph-mon[35160]: pgmap v230: 97 pgs: 97 active+clean; 213 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-09-10T10:53:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:17 smithi007 ceph-mon[30873]: pgmap v230: 97 pgs: 97 active+clean; 213 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-09-10T10:53:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:17 smithi133 ceph-mon[35282]: pgmap v230: 97 pgs: 97 active+clean; 213 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-09-10T10:53:18.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 834752 11.47 MB/sec execute 258 sec latency 17.561 ms 2024-09-10T10:53:19.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 836953 11.46 MB/sec execute 259 sec latency 15.977 ms 2024-09-10T10:53:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:19 smithi103 ceph-mon[35160]: pgmap v231: 97 pgs: 97 active+clean; 213 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-09-10T10:53:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:19 smithi007 ceph-mon[30873]: pgmap v231: 97 pgs: 97 active+clean; 213 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-09-10T10:53:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:19 smithi133 ceph-mon[35282]: pgmap v231: 97 pgs: 97 active+clean; 213 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-09-10T10:53:20.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 838372 11.46 MB/sec execute 260 sec latency 20.310 ms 2024-09-10T10:53:21.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 840835 11.46 MB/sec execute 261 sec latency 16.617 ms 2024-09-10T10:53:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:21 smithi103 ceph-mon[35160]: pgmap v232: 97 pgs: 97 active+clean; 223 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2024-09-10T10:53:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:21 smithi007 ceph-mon[30873]: pgmap v232: 97 pgs: 97 active+clean; 223 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2024-09-10T10:53:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:21 smithi133 ceph-mon[35282]: pgmap v232: 97 pgs: 97 active+clean; 223 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2024-09-10T10:53:22.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 842249 11.45 MB/sec execute 262 sec latency 12.599 ms 2024-09-10T10:53:23.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 844835 11.46 MB/sec execute 263 sec latency 14.646 ms 2024-09-10T10:53:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:23 smithi103 ceph-mon[35160]: pgmap v233: 97 pgs: 97 active+clean; 223 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-09-10T10:53:23.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:23 smithi007 ceph-mon[30873]: pgmap v233: 97 pgs: 97 active+clean; 223 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-09-10T10:53:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:23 smithi133 ceph-mon[35282]: pgmap v233: 97 pgs: 97 active+clean; 223 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-09-10T10:53:24.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 845888 11.43 MB/sec execute 264 sec latency 222.781 ms 2024-09-10T10:53:25.520 INFO:tasks.workunit.client.0.smithi007.stdout: 1 848395 11.44 MB/sec execute 265 sec latency 42.657 ms 2024-09-10T10:53:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:25 smithi103 ceph-mon[35160]: pgmap v234: 97 pgs: 97 active+clean; 223 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-09-10T10:53:25.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:25 smithi007 ceph-mon[30873]: pgmap v234: 97 pgs: 97 active+clean; 223 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-09-10T10:53:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:25 smithi133 ceph-mon[35282]: pgmap v234: 97 pgs: 97 active+clean; 223 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-09-10T10:53:26.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 850056 11.44 MB/sec execute 266 sec latency 25.499 ms 2024-09-10T10:53:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.921+0000 7f95de4b7640 1 -- 172.21.15.7:0/3982888235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d8107b70 msgr2=0x7f95d8107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.921+0000 7f95de4b7640 1 --2- 172.21.15.7:0/3982888235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d8107b70 0x7f95d8107f50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009930 tx=0x7f95c802a420 comp rx=0 tx=0).stop 2024-09-10T10:53:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.922+0000 7f95de4b7640 1 -- 172.21.15.7:0/3982888235 shutdown_connections 2024-09-10T10:53:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.922+0000 7f95de4b7640 1 --2- 172.21.15.7:0/3982888235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95d8102470 0x7f95d810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.922+0000 7f95de4b7640 1 --2- 172.21.15.7:0/3982888235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95d8101ad0 0x7f95d8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.922+0000 7f95de4b7640 1 --2- 172.21.15.7:0/3982888235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d8107b70 0x7f95d8107f50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.922+0000 7f95de4b7640 1 -- 172.21.15.7:0/3982888235 >> 172.21.15.7:0/3982888235 conn(0x7f95d80fd7d0 msgr2=0x7f95d80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.922+0000 7f95de4b7640 1 -- 172.21.15.7:0/3982888235 shutdown_connections 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.922+0000 7f95de4b7640 1 -- 172.21.15.7:0/3982888235 wait complete. 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.922+0000 7f95de4b7640 1 Processor -- start 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.922+0000 7f95de4b7640 1 -- start start 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95de4b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95d8101ad0 0x7f95d8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95de4b7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95d8102470 0x7f95d81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95de4b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d8107b70 0x7f95d819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95de4b7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8111f70 con 0x7f95d8107b70 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95de4b7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8111df0 con 0x7f95d8102470 2024-09-10T10:53:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95de4b7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81120f0 con 0x7f95d8101ad0 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95dca54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d8107b70 0x7f95d819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95dca54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d8107b70 0x7f95d819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35764/0 (socket says 172.21.15.7:35764) 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95dca54640 1 -- 172.21.15.7:0/2210060558 learned_addr learned my addr 172.21.15.7:0/2210060558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95d7fff640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95d8101ad0 0x7f95d8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95d77fe640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95d8102470 0x7f95d81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95dca54640 1 -- 172.21.15.7:0/2210060558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95d8101ad0 msgr2=0x7f95d8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95dca54640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95d8101ad0 0x7f95d8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95dca54640 1 -- 172.21.15.7:0/2210060558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95d8102470 msgr2=0x7f95d81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95dca54640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95d8102470 0x7f95d81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95dca54640 1 -- 172.21.15.7:0/2210060558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d819d370 con 0x7f95d8107b70 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95dca54640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d8107b70 0x7f95d819cc40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f95d000eb10 tx=0x7f95d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95d57fa640 1 -- 172.21.15.7:0/2210060558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0004b20 con 0x7f95d8107b70 2024-09-10T10:53:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95de4b7640 1 -- 172.21.15.7:0/2210060558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d819d660 con 0x7f95d8107b70 2024-09-10T10:53:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95d57fa640 1 -- 172.21.15.7:0/2210060558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d0004cc0 con 0x7f95d8107b70 2024-09-10T10:53:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.923+0000 7f95d57fa640 1 -- 172.21.15.7:0/2210060558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0010760 con 0x7f95d8107b70 2024-09-10T10:53:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.924+0000 7f95de4b7640 1 -- 172.21.15.7:0/2210060558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81a4ef0 con 0x7f95d8107b70 2024-09-10T10:53:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.926+0000 7f95d57fa640 1 -- 172.21.15.7:0/2210060558 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f95d0005020 con 0x7f95d8107b70 2024-09-10T10:53:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.926+0000 7f95e1253740 1 Processor -- start 2024-09-10T10:53:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.926+0000 7f95e1253740 1 -- start start 2024-09-10T10:53:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95e1253740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef202af0 0x5604ef202ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95e1253740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 0x5604ef20d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95e1253740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 0x5604ef2100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d57fa640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f95c00a2da0 0x7f95c00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95e1253740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5604ef1672f0 con 0x5604ef202af0 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95e1253740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604ef10f270 con 0x5604ef203410 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95e1253740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5604ef1632b0 con 0x5604ef20dcb0 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef202af0 0x5604ef202ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef202af0 0x5604ef202ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35780/0 (socket says 172.21.15.7:35780) 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d77fe640 1 -- 172.21.15.7:0/1378416361 learned_addr learned my addr 172.21.15.7:0/1378416361 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d7fff640 1 --2- 172.21.15.7:0/1378416361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 0x5604ef20d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1378416361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 0x5604ef2100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95dca54640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f95c00a2da0 0x7f95c00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d57fa640 1 -- 172.21.15.7:0/2210060558 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f95d00cfab0 con 0x7f95d8107b70 2024-09-10T10:53:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d77fe640 1 -- 172.21.15.7:0/1378416361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 msgr2=0x5604ef2100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1378416361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 0x5604ef2100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d77fe640 1 -- 172.21.15.7:0/1378416361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 msgr2=0x5604ef20d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1378416361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 0x5604ef20d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d77fe640 1 -- 172.21.15.7:0/1378416361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604ef10e340 con 0x5604ef202af0 2024-09-10T10:53:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d7fff640 1 --2- 172.21.15.7:0/1378416361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 0x5604ef20d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1378416361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef202af0 0x5604ef202ed0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f95cc013840 tx=0x7f95cc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200711b9601d6ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.927+0000 7f95dca54640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f95c00a2da0 0x7f95c00a5260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f95d00cd280 tx=0x7f95d0097ba0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.928+0000 7f95be7fc640 1 -- 172.21.15.7:0/1378416361 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc013170 con 0x5604ef202af0 2024-09-10T10:53:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.928+0000 7f95be7fc640 1 -- 172.21.15.7:0/1378416361 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc005500 con 0x5604ef202af0 2024-09-10T10:53:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.928+0000 7f95be7fc640 1 -- 172.21.15.7:0/1378416361 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc004ca0 con 0x5604ef202af0 2024-09-10T10:53:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.928+0000 7f95e1253740 1 -- 172.21.15.7:0/1378416361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef202af0 msgr2=0x5604ef202ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.928+0000 7f95e1253740 1 --2- 172.21.15.7:0/1378416361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef202af0 0x5604ef202ed0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f95cc013840 tx=0x7f95cc005f90 comp rx=0 tx=0).stop 2024-09-10T10:53:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.928+0000 7f95e1253740 1 -- 172.21.15.7:0/1378416361 shutdown_connections 2024-09-10T10:53:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.928+0000 7f95e1253740 1 --2- 172.21.15.7:0/1378416361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 0x5604ef2100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.928+0000 7f95e1253740 1 --2- 172.21.15.7:0/1378416361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 0x5604ef20d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.928+0000 7f95e1253740 1 --2- 172.21.15.7:0/1378416361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef202af0 0x5604ef202ed0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.928+0000 7f95e1253740 1 -- 172.21.15.7:0/1378416361 >> 172.21.15.7:0/1378416361 conn(0x5604ef1ff060 msgr2=0x5604ef1ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.929+0000 7f95e1253740 1 -- 172.21.15.7:0/1378416361 shutdown_connections 2024-09-10T10:53:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.929+0000 7f95e1253740 1 -- 172.21.15.7:0/1378416361 wait complete. 2024-09-10T10:53:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.929+0000 7f95e1253740 1 -- start start 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.929+0000 7f95e1253740 1 Processor -- start 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.929+0000 7f95e1253740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef202af0 0x5604ef20aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.929+0000 7f95e1253740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 0x5604ef20b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.929+0000 7f95e1253740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef20dcb0 0x5604ef20b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.929+0000 7f95e1253740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5604ef210880 con 0x5604ef20dcb0 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.929+0000 7f95e1253740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5604ef0f0290 con 0x5604ef203410 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.929+0000 7f95e1253740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5604ef210a00 con 0x5604ef202af0 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95e1253740 10 client.0 _ll_register_callbacks cb 0x5604ef121140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef20dcb0 0x5604ef20b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef20dcb0 0x5604ef20b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35786/0 (socket says 172.21.15.7:35786) 2024-09-10T10:53:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d77fe640 1 -- 172.21.15.7:0/1705711105 learned_addr learned my addr 172.21.15.7:0/1705711105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d7fff640 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 0x5604ef20b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef202af0 0x5604ef20aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d77fe640 1 -- 172.21.15.7:0/1705711105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef202af0 msgr2=0x5604ef20aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef202af0 0x5604ef20aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d77fe640 1 -- 172.21.15.7:0/1705711105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 msgr2=0x5604ef20b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 0x5604ef20b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d77fe640 1 -- 172.21.15.7:0/1705711105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604eee7a210 con 0x5604ef20dcb0 2024-09-10T10:53:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d7fff640 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 0x5604ef20b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef20dcb0 0x5604ef20b560 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f95cc004980 tx=0x7f95cc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fd5297be812143 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95b77fe640 1 -- 172.21.15.7:0/1705711105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc004740 con 0x5604ef20dcb0 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95b77fe640 1 -- 172.21.15.7:0/1705711105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f95cc004a20 con 0x5604ef20dcb0 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95e1253740 1 -- 172.21.15.7:0/1705711105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95cc005500 con 0x5604ef20dcb0 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95b77fe640 1 -- 172.21.15.7:0/1705711105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f95cc004c20 con 0x5604ef20dcb0 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95e1253740 10 client.15849 fetch_fsmap learned FSMap version 33 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.930+0000 7f95e1253740 1 -- 172.21.15.7:0/1705711105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5604eef46620 con 0x5604ef20dcb0 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.931+0000 7f95b77fe640 1 -- 172.21.15.7:0/1705711105 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc005500 con 0x5604ef20dcb0 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.931+0000 7f95b77fe640 1 -- 172.21.15.7:0/1705711105 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f95cc0105f0 con 0x5604ef20dcb0 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.931+0000 7f95e1253740 10 client.15849 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:53:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.931+0000 7f95e1253740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.931+0000 7f95e1253740 20 client.15849 populate_metadata read hostname 'smithi007' 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.931+0000 7f95e1253740 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5604ef21d9d0 0x5604ef21fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.931+0000 7f95e1253740 4 client.15849 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.931+0000 7f95e1253740 1 -- 172.21.15.7:0/1705711105 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604ef174a20 con 0x5604ef21d9d0 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.931+0000 7f95d7fff640 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5604ef21d9d0 0x5604ef21fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.932+0000 7f95d7fff640 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5604ef21d9d0 0x5604ef21fdc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=975507bd418c9c0d server_cookie=b305f12eca258969 in_seq=0 out_seq=0 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.932+0000 7f95b77fe640 10 client.15849 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.932+0000 7f95b77fe640 1 -- 172.21.15.7:0/1705711105 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604ef174a20 con 0x5604ef21d9d0 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.932+0000 7f95b77fe640 10 client.15849 handle_command_reply: tid=0 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.932+0000 7f95e1253740 1 client.15849 shutdown 2024-09-10T10:53:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.932+0000 7f95e1253740 1 -- 172.21.15.7:0/1705711105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef20dcb0 msgr2=0x5604ef20b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.932+0000 7f95e1253740 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef20dcb0 0x5604ef20b560 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f95cc004980 tx=0x7f95cc009260 comp rx=0 tx=0).stop 2024-09-10T10:53:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.933+0000 7f95e1253740 1 -- 172.21.15.7:0/1705711105 shutdown_connections 2024-09-10T10:53:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.933+0000 7f95e1253740 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5604ef21d9d0 0x5604ef21fdc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.933+0000 7f95e1253740 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef20dcb0 0x5604ef20b560 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.933+0000 7f95e1253740 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef203410 0x5604ef20b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.933+0000 7f95b77fe640 0 client.15849 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.933+0000 7f95e1253740 1 --2- 172.21.15.7:0/1705711105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef202af0 0x5604ef20aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.933+0000 7f95e1253740 1 -- 172.21.15.7:0/1705711105 >> 172.21.15.7:0/1705711105 conn(0x5604ef1ff060 msgr2=0x5604ef204dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.933+0000 7f95e1253740 1 -- 172.21.15.7:0/1705711105 shutdown_connections 2024-09-10T10:53:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.933+0000 7f95e1253740 1 -- 172.21.15.7:0/1705711105 wait complete. 2024-09-10T10:53:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.933+0000 7f95e1253740 20 client.15849 trim_cache size 0 max 16384 2024-09-10T10:53:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95e1253740 1 Processor -- start 2024-09-10T10:53:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95e1253740 1 -- start start 2024-09-10T10:53:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95e1253740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef202af0 0x5604ef20b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95e1253740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef225530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95e1253740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef20dcb0 0x5604ef225a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95e1253740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81b01a0 con 0x5604ef203410 2024-09-10T10:53:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95e1253740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ada40 con 0x5604ef20dcb0 2024-09-10T10:53:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95e1253740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ab8b0 con 0x5604ef202af0 2024-09-10T10:53:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef225530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef225530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35788/0 (socket says 172.21.15.7:35788) 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.949+0000 7f95d7fff640 1 -- 172.21.15.7:0/1570006660 learned_addr learned my addr 172.21.15.7:0/1570006660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1570006660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef202af0 0x5604ef20b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1570006660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef20dcb0 0x5604ef225a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95d7fff640 1 -- 172.21.15.7:0/1570006660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef202af0 msgr2=0x5604ef20b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95d7fff640 1 --2- 172.21.15.7:0/1570006660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef202af0 0x5604ef20b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95d7fff640 1 -- 172.21.15.7:0/1570006660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef20dcb0 msgr2=0x5604ef225a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95d7fff640 1 --2- 172.21.15.7:0/1570006660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef20dcb0 0x5604ef225a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95d7fff640 1 -- 172.21.15.7:0/1570006660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604ef1fee90 con 0x5604ef203410 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95d77fe640 1 --2- 172.21.15.7:0/1570006660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef20dcb0 0x5604ef225a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95d7fff640 1 --2- 172.21.15.7:0/1570006660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef225530 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f95c8031550 tx=0x7f95c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb11d196119da60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95bdffb640 1 -- 172.21.15.7:0/1570006660 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8035040 con 0x5604ef203410 2024-09-10T10:53:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95bdffb640 1 -- 172.21.15.7:0/1570006660 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c804b070 con 0x5604ef203410 2024-09-10T10:53:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.950+0000 7f95bdffb640 1 -- 172.21.15.7:0/1570006660 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8004e60 con 0x5604ef203410 2024-09-10T10:53:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.951+0000 7f95e1253740 1 -- 172.21.15.7:0/1570006660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 msgr2=0x5604ef225530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.951+0000 7f95e1253740 1 --2- 172.21.15.7:0/1570006660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef225530 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f95c8031550 tx=0x7f95c8000bc0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.951+0000 7f95e1253740 1 -- 172.21.15.7:0/1570006660 shutdown_connections 2024-09-10T10:53:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.951+0000 7f95e1253740 1 --2- 172.21.15.7:0/1570006660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef20dcb0 0x5604ef225a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.951+0000 7f95e1253740 1 --2- 172.21.15.7:0/1570006660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef225530 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.951+0000 7f95e1253740 1 --2- 172.21.15.7:0/1570006660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef202af0 0x5604ef20b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.951+0000 7f95e1253740 1 -- 172.21.15.7:0/1570006660 >> 172.21.15.7:0/1570006660 conn(0x5604ef1ff060 msgr2=0x5604ef20cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.951+0000 7f95e1253740 1 -- 172.21.15.7:0/1570006660 shutdown_connections 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.951+0000 7f95e1253740 1 -- 172.21.15.7:0/1570006660 wait complete. 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.951+0000 7f95e1253740 1 -- start start 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95e1253740 1 Processor -- start 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95e1253740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef202af0 0x5604ef209d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95e1253740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef20a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95e1253740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 0x5604ef20a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95e1253740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81abf00 con 0x5604ef203410 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95e1253740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ad370 con 0x5604ef202af0 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95e1253740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ac5d0 con 0x5604ef20dcb0 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95e1253740 10 client.0 _ll_register_callbacks cb 0x7f95d00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 0x5604ef20a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef20a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 0x5604ef20a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:39010/0 (socket says 172.21.15.7:39010) 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef20a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35796/0 (socket says 172.21.15.7:35796) 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95d77fe640 1 -- 172.21.15.7:0/810031023 learned_addr learned my addr 172.21.15.7:0/810031023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.952+0000 7f95d77fe640 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef202af0 0x5604ef209d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95d77fe640 1 -- 172.21.15.7:0/810031023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef202af0 msgr2=0x5604ef209d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95d77fe640 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef202af0 0x5604ef209d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95d77fe640 1 -- 172.21.15.7:0/810031023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 msgr2=0x5604ef20a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95d77fe640 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef20a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95d77fe640 1 -- 172.21.15.7:0/810031023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604ef214f30 con 0x5604ef20dcb0 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95d7fff640 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef20a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95d77fe640 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 0x5604ef20a7e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0058c0 tx=0x7f95cc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f16c2919224f7600 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95b77fe640 1 -- 172.21.15.7:0/810031023 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc005330 con 0x5604ef20dcb0 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95b77fe640 1 -- 172.21.15.7:0/810031023 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f95cc0054d0 con 0x5604ef20dcb0 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95e1253740 1 -- 172.21.15.7:0/810031023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95c804b070 con 0x5604ef20dcb0 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95b77fe640 1 -- 172.21.15.7:0/810031023 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f95cc010430 con 0x5604ef20dcb0 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95e1253740 10 client.25543 fetch_fsmap learned FSMap version 33 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.953+0000 7f95e1253740 1 -- 172.21.15.7:0/810031023 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5604ef216240 con 0x5604ef20dcb0 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.954+0000 7f95b77fe640 1 -- 172.21.15.7:0/810031023 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c804b070 con 0x5604ef20dcb0 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.954+0000 7f95b77fe640 1 -- 172.21.15.7:0/810031023 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f95cc030430 con 0x5604ef20dcb0 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.954+0000 7f95e1253740 10 client.25543 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.954+0000 7f95e1253740 10 client.25543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.954+0000 7f95e1253740 20 client.25543 populate_metadata read hostname 'smithi007' 2024-09-10T10:53:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.954+0000 7f95e1253740 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5604ef288500 0x5604ef28a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.954+0000 7f95e1253740 4 client.25543 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:53:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.954+0000 7f95e1253740 1 -- 172.21.15.7:0/810031023 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604ef16ed70 con 0x5604ef288500 2024-09-10T10:53:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.954+0000 7f95d7fff640 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5604ef288500 0x5604ef28a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.955+0000 7f95d7fff640 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5604ef288500 0x5604ef28a8f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff2388c16f446447 server_cookie=10d2115fdb9d78a8 in_seq=0 out_seq=0 2024-09-10T10:53:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:26.955+0000 7f95b77fe640 10 client.25543 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95b77fe640 1 -- 172.21.15.7:0/810031023 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604ef16ed70 con 0x5604ef288500 2024-09-10T10:53:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95b77fe640 10 client.25543 handle_command_reply: tid=0 2024-09-10T10:53:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95e1253740 1 client.25543 shutdown 2024-09-10T10:53:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95e1253740 1 -- 172.21.15.7:0/810031023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 msgr2=0x5604ef20a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95e1253740 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 0x5604ef20a7e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0058c0 tx=0x7f95cc009260 comp rx=0 tx=0).stop 2024-09-10T10:53:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95e1253740 1 -- 172.21.15.7:0/810031023 shutdown_connections 2024-09-10T10:53:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95e1253740 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5604ef288500 0x5604ef28a8f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95e1253740 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604ef20dcb0 0x5604ef20a7e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95e1253740 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ef203410 0x5604ef20a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95e1253740 1 --2- 172.21.15.7:0/810031023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5604ef202af0 0x5604ef209d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95e1253740 1 -- 172.21.15.7:0/810031023 >> 172.21.15.7:0/810031023 conn(0x5604ef1ff060 msgr2=0x5604ef1fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.018+0000 7f95b77fe640 0 client.25543 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95e1253740 1 -- 172.21.15.7:0/810031023 shutdown_connections 2024-09-10T10:53:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95e1253740 1 -- 172.21.15.7:0/810031023 wait complete. 2024-09-10T10:53:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95e1253740 20 client.25543 trim_cache size 0 max 16384 2024-09-10T10:53:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:53:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:53:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:53:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:53:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95de4b7640 1 -- 172.21.15.7:0/2210060558 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f95c00a2da0 msgr2=0x7f95c00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95de4b7640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f95c00a2da0 0x7f95c00a5260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f95d00cd280 tx=0x7f95d0097ba0 comp rx=0 tx=0).stop 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95de4b7640 1 -- 172.21.15.7:0/2210060558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d8107b70 msgr2=0x7f95d819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95de4b7640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d8107b70 0x7f95d819cc40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f95d000eb10 tx=0x7f95d000c2d0 comp rx=0 tx=0).stop 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95de4b7640 1 -- 172.21.15.7:0/2210060558 shutdown_connections 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95de4b7640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f95c00a2da0 0x7f95c00a5260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95de4b7640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95d8107b70 0x7f95d819cc40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95de4b7640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95d8102470 0x7f95d81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95de4b7640 1 --2- 172.21.15.7:0/2210060558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95d8101ad0 0x7f95d8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.019+0000 7f95de4b7640 1 -- 172.21.15.7:0/2210060558 >> 172.21.15.7:0/2210060558 conn(0x7f95d80fd7d0 msgr2=0x7f95d80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.020+0000 7f95de4b7640 1 -- 172.21.15.7:0/2210060558 shutdown_connections 2024-09-10T10:53:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:27.020+0000 7f95de4b7640 1 -- 172.21.15.7:0/2210060558 wait complete. 2024-09-10T10:53:27.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 852378 11.44 MB/sec execute 267 sec latency 30.090 ms 2024-09-10T10:53:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:27 smithi103 ceph-mon[35160]: pgmap v235: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-09-10T10:53:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:27 smithi007 ceph-mon[30873]: pgmap v235: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-09-10T10:53:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:27 smithi133 ceph-mon[35282]: pgmap v235: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-09-10T10:53:28.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 854501 11.43 MB/sec execute 268 sec latency 14.593 ms 2024-09-10T10:53:29.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 856256 11.43 MB/sec execute 269 sec latency 17.552 ms 2024-09-10T10:53:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:29 smithi103 ceph-mon[35160]: pgmap v236: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-09-10T10:53:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:29 smithi007 ceph-mon[30873]: pgmap v236: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-09-10T10:53:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:29 smithi133 ceph-mon[35282]: pgmap v236: 97 pgs: 97 active+clean; 239 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-09-10T10:53:30.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 858663 11.44 MB/sec execute 270 sec latency 59.052 ms 2024-09-10T10:53:31.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 860310 11.42 MB/sec execute 271 sec latency 19.848 ms 2024-09-10T10:53:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:31 smithi103 ceph-mon[35160]: pgmap v237: 97 pgs: 97 active+clean; 227 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 363 op/s 2024-09-10T10:53:31.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:31 smithi007 ceph-mon[30873]: pgmap v237: 97 pgs: 97 active+clean; 227 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 363 op/s 2024-09-10T10:53:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:31 smithi133 ceph-mon[35282]: pgmap v237: 97 pgs: 97 active+clean; 227 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 363 op/s 2024-09-10T10:53:32.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 862878 11.44 MB/sec execute 272 sec latency 66.447 ms 2024-09-10T10:53:33.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 864605 11.42 MB/sec execute 273 sec latency 12.855 ms 2024-09-10T10:53:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:33 smithi103 ceph-mon[35160]: pgmap v238: 97 pgs: 97 active+clean; 227 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-09-10T10:53:33.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:33 smithi007 ceph-mon[30873]: pgmap v238: 97 pgs: 97 active+clean; 227 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-09-10T10:53:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:33 smithi133 ceph-mon[35282]: pgmap v238: 97 pgs: 97 active+clean; 227 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-09-10T10:53:34.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 866756 11.43 MB/sec execute 274 sec latency 11.669 ms 2024-09-10T10:53:35.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 869212 11.44 MB/sec execute 275 sec latency 49.853 ms 2024-09-10T10:53:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:35 smithi103 ceph-mon[35160]: pgmap v239: 97 pgs: 97 active+clean; 227 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-09-10T10:53:35.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:35 smithi007 ceph-mon[30873]: pgmap v239: 97 pgs: 97 active+clean; 227 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-09-10T10:53:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:35 smithi133 ceph-mon[35282]: pgmap v239: 97 pgs: 97 active+clean; 227 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-09-10T10:53:36.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 870557 11.42 MB/sec execute 276 sec latency 46.359 ms 2024-09-10T10:53:37.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 873043 11.43 MB/sec execute 277 sec latency 18.639 ms 2024-09-10T10:53:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:37 smithi103 ceph-mon[35160]: pgmap v240: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2024-09-10T10:53:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:37 smithi007 ceph-mon[30873]: pgmap v240: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2024-09-10T10:53:37.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:37 smithi133 ceph-mon[35282]: pgmap v240: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 378 op/s 2024-09-10T10:53:38.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 874769 11.42 MB/sec execute 278 sec latency 16.241 ms 2024-09-10T10:53:39.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 877138 11.43 MB/sec execute 279 sec latency 16.463 ms 2024-09-10T10:53:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:39 smithi103 ceph-mon[35160]: pgmap v241: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-09-10T10:53:39.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:39 smithi007 ceph-mon[30873]: pgmap v241: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-09-10T10:53:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:39 smithi133 ceph-mon[35282]: pgmap v241: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-09-10T10:53:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.114+0000 7f7010b49640 1 -- 172.21.15.7:0/3783485115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c101ad0 msgr2=0x7f700c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.114+0000 7f7010b49640 1 --2- 172.21.15.7:0/3783485115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c101ad0 0x7f700c101f30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7000009930 tx=0x7f700002a420 comp rx=0 tx=0).stop 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.114+0000 7f7010b49640 1 -- 172.21.15.7:0/3783485115 shutdown_connections 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.114+0000 7f7010b49640 1 --2- 172.21.15.7:0/3783485115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c102470 0x7f700c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.114+0000 7f7010b49640 1 --2- 172.21.15.7:0/3783485115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c101ad0 0x7f700c101f30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.114+0000 7f7010b49640 1 --2- 172.21.15.7:0/3783485115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f700c107b70 0x7f700c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.114+0000 7f7010b49640 1 -- 172.21.15.7:0/3783485115 >> 172.21.15.7:0/3783485115 conn(0x7f700c0fd7d0 msgr2=0x7f700c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.114+0000 7f7010b49640 1 -- 172.21.15.7:0/3783485115 shutdown_connections 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.114+0000 7f7010b49640 1 -- 172.21.15.7:0/3783485115 wait complete. 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7010b49640 1 Processor -- start 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7010b49640 1 -- start start 2024-09-10T10:53:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7010b49640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f700c101ad0 0x7f700c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7010b49640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c102470 0x7f700c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7010b49640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c107b70 0x7f700c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7010b49640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f700c111f20 con 0x7f700c102470 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7010b49640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f700c111da0 con 0x7f700c101ad0 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7010b49640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f700c1120a0 con 0x7f700c107b70 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7009d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c102470 0x7f700c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f700a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f700c101ad0 0x7f700c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7009d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c102470 0x7f700c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54538/0 (socket says 172.21.15.7:54538) 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f700a59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f700c101ad0 0x7f700c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:42130/0 (socket says 172.21.15.7:42130) 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7009d9b640 1 -- 172.21.15.7:0/1391942540 learned_addr learned my addr 172.21.15.7:0/1391942540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f700ad9d640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c107b70 0x7f700c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7009d9b640 1 -- 172.21.15.7:0/1391942540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c107b70 msgr2=0x7f700c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7009d9b640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c107b70 0x7f700c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7009d9b640 1 -- 172.21.15.7:0/1391942540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f700c101ad0 msgr2=0x7f700c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7009d9b640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f700c101ad0 0x7f700c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f7009d9b640 1 -- 172.21.15.7:0/1391942540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c19d320 con 0x7f700c102470 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.115+0000 7f700ad9d640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c107b70 0x7f700c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.116+0000 7f7009d9b640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c102470 0x7f700c198870 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7000009230 tx=0x7f7000004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.116+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/1391942540 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700003f030 con 0x7f700c102470 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.116+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/1391942540 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7000003ad0 con 0x7f700c102470 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.116+0000 7f7010b49640 1 -- 172.21.15.7:0/1391942540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700c19d5b0 con 0x7f700c102470 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.116+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/1391942540 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000035070 con 0x7f700c102470 2024-09-10T10:53:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.116+0000 7f7010b49640 1 -- 172.21.15.7:0/1391942540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f700c1a4ea0 con 0x7f700c102470 2024-09-10T10:53:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.118+0000 7f70139bc740 1 Processor -- start 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.118+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/1391942540 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7000042030 con 0x7f700c102470 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.118+0000 7f70139bc740 1 -- start start 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f70139bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f70139bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b82f0 0x55eae51c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f70139bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51c70c0 0x55eae51c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f70139bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eae511c2f0 con 0x55eae51b79d0 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f70139bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eae50c4270 con 0x55eae51b82f0 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f70139bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55eae51182b0 con 0x55eae51c70c0 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f700ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f700ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54542/0 (socket says 172.21.15.7:54542) 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f700ad9d640 1 -- 172.21.15.7:0/734435228 learned_addr learned my addr 172.21.15.7:0/734435228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f7009d9b640 1 --2- 172.21.15.7:0/734435228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51c70c0 0x55eae51c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f700a59c640 1 --2- 172.21.15.7:0/734435228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b82f0 0x55eae51c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f6ffb7fe640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6ff40a2e20 0x7f6ff40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f700ad9d640 1 -- 172.21.15.7:0/734435228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51c70c0 msgr2=0x55eae51c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f700ad9d640 1 --2- 172.21.15.7:0/734435228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51c70c0 0x55eae51c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f700ad9d640 1 -- 172.21.15.7:0/734435228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b82f0 msgr2=0x55eae51c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f700ad9d640 1 --2- 172.21.15.7:0/734435228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b82f0 0x55eae51c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f700ad9d640 1 -- 172.21.15.7:0/734435228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae50c3340 con 0x55eae51b79d0 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f6ffb7fe640 1 -- 172.21.15.7:0/1391942540 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f70000ef960 con 0x7f700c102470 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.119+0000 7f700ad9d640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6ff40a2e20 0x7f6ff40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.120+0000 7f700ad9d640 1 --2- 172.21.15.7:0/734435228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51b7db0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7004011810 tx=0x7f7004011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812f53d20f9500c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.120+0000 7f6ff8ff9640 1 -- 172.21.15.7:0/734435228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004020070 con 0x55eae51b79d0 2024-09-10T10:53:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.120+0000 7f6ff8ff9640 1 -- 172.21.15.7:0/734435228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f700400fd20 con 0x55eae51b79d0 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.120+0000 7f6ff8ff9640 1 -- 172.21.15.7:0/734435228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700401b930 con 0x55eae51b79d0 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.120+0000 7f70139bc740 1 -- 172.21.15.7:0/734435228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 msgr2=0x55eae51b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.120+0000 7f70139bc740 1 --2- 172.21.15.7:0/734435228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51b7db0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7004011810 tx=0x7f7004011cd0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.120+0000 7f700ad9d640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6ff40a2e20 0x7f6ff40a52e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7004011490 tx=0x7f700401bf80 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 -- 172.21.15.7:0/734435228 shutdown_connections 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 --2- 172.21.15.7:0/734435228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51c70c0 0x55eae51c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 --2- 172.21.15.7:0/734435228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b82f0 0x55eae51c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 --2- 172.21.15.7:0/734435228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51b7db0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 -- 172.21.15.7:0/734435228 >> 172.21.15.7:0/734435228 conn(0x55eae51b4060 msgr2=0x55eae51b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 -- 172.21.15.7:0/734435228 shutdown_connections 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 -- 172.21.15.7:0/734435228 wait complete. 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 -- start start 2024-09-10T10:53:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 Processor -- start 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b79d0 0x55eae51c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b82f0 0x55eae51c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51c70c0 0x55eae51c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eae51c9cf0 con 0x55eae51b82f0 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55eae50a5290 con 0x55eae51c70c0 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55eae51c9e70 con 0x55eae51b79d0 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f70139bc740 10 client.0 _ll_register_callbacks cb 0x55eae50d6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f7009d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b82f0 0x55eae51c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f7009d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b82f0 0x55eae51c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54556/0 (socket says 172.21.15.7:54556) 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f7009d9b640 1 -- 172.21.15.7:0/3281933703 learned_addr learned my addr 172.21.15.7:0/3281933703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.121+0000 7f700a59c640 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b79d0 0x55eae51c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f700ad9d640 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51c70c0 0x55eae51c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f7009d9b640 1 -- 172.21.15.7:0/3281933703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b79d0 msgr2=0x55eae51c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f7009d9b640 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b79d0 0x55eae51c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f7009d9b640 1 -- 172.21.15.7:0/3281933703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51c70c0 msgr2=0x55eae51c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f7009d9b640 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51c70c0 0x55eae51c4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f7009d9b640 1 -- 172.21.15.7:0/3281933703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae4e2f210 con 0x55eae51b82f0 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f7009d9b640 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b82f0 0x55eae51c4520 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f70000ed0e0 tx=0x7f7000009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b268f3be4713b4c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f6ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/3281933703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000049070 con 0x55eae51b82f0 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/3281933703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f7000003c80 con 0x55eae51b82f0 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f70139bc740 1 -- 172.21.15.7:0/3281933703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f700400fd20 con 0x55eae51b82f0 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/3281933703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f70000bba40 con 0x55eae51b82f0 2024-09-10T10:53:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f70139bc740 10 client.15867 fetch_fsmap learned FSMap version 33 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f70139bc740 1 -- 172.21.15.7:0/3281933703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eae4efb620 con 0x55eae51b82f0 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/3281933703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f700400fd20 con 0x55eae51b82f0 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.122+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/3281933703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f70000b8570 con 0x55eae51b82f0 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.123+0000 7f70139bc740 10 client.15867 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.123+0000 7f70139bc740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.123+0000 7f70139bc740 20 client.15867 populate_metadata read hostname 'smithi007' 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.123+0000 7f70139bc740 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55eae51d7100 0x55eae51d9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.123+0000 7f70139bc740 4 client.15867 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.123+0000 7f70139bc740 1 -- 172.21.15.7:0/3281933703 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eae5129a20 con 0x55eae51d7100 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.123+0000 7f700a59c640 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55eae51d7100 0x55eae51d9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.123+0000 7f700a59c640 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55eae51d7100 0x55eae51d9560 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b561cbfd79be360 server_cookie=33877042b3997486 in_seq=0 out_seq=0 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.123+0000 7f6ff1ffb640 10 client.15867 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.124+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/3281933703 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eae5129a20 con 0x55eae51d7100 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.124+0000 7f6ff1ffb640 10 client.15867 handle_command_reply: tid=0 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.124+0000 7f70139bc740 1 client.15867 shutdown 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 1 -- 172.21.15.7:0/3281933703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b82f0 msgr2=0x55eae51c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b82f0 0x55eae51c4520 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f70000ed0e0 tx=0x7f7000009560 comp rx=0 tx=0).stop 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 1 -- 172.21.15.7:0/3281933703 shutdown_connections 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55eae51d7100 0x55eae51d9560 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51c70c0 0x55eae51c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b82f0 0x55eae51c4520 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f6ff1ffb640 0 client.15867 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 1 --2- 172.21.15.7:0/3281933703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b79d0 0x55eae51c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 1 -- 172.21.15.7:0/3281933703 >> 172.21.15.7:0/3281933703 conn(0x55eae51b4060 msgr2=0x55eae51b9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 1 -- 172.21.15.7:0/3281933703 shutdown_connections 2024-09-10T10:53:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 1 -- 172.21.15.7:0/3281933703 wait complete. 2024-09-10T10:53:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.125+0000 7f70139bc740 20 client.15867 trim_cache size 0 max 16384 2024-09-10T10:53:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.140+0000 7f70139bc740 1 Processor -- start 2024-09-10T10:53:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.140+0000 7f70139bc740 1 -- start start 2024-09-10T10:53:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f70139bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b79d0 0x55eae51b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f70139bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b82f0 0x55eae51dee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f70139bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51c70c0 0x55eae51df370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f70139bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f700c1ade30 con 0x55eae51c70c0 2024-09-10T10:53:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f70139bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f700c1ab6d0 con 0x55eae51b79d0 2024-09-10T10:53:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f70139bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f700c1a9560 con 0x55eae51b82f0 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51c70c0 0x55eae51df370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51c70c0 0x55eae51df370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53924/0 (socket says 172.21.15.7:53924) 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700ad9d640 1 -- 172.21.15.7:0/1597671266 learned_addr learned my addr 172.21.15.7:0/1597671266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f7009d9b640 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b82f0 0x55eae51dee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700a59c640 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b79d0 0x55eae51b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700ad9d640 1 -- 172.21.15.7:0/1597671266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b82f0 msgr2=0x55eae51dee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700ad9d640 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b82f0 0x55eae51dee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700ad9d640 1 -- 172.21.15.7:0/1597671266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b79d0 msgr2=0x55eae51b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700ad9d640 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b79d0 0x55eae51b9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700ad9d640 1 -- 172.21.15.7:0/1597671266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae51c66d0 con 0x55eae51c70c0 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700a59c640 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b79d0 0x55eae51b9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f7009d9b640 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b82f0 0x55eae51dee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f700ad9d640 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51c70c0 0x55eae51df370 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f70040117e0 tx=0x7f7004028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114476ff86af77b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f6ff3fff640 1 -- 172.21.15.7:0/1597671266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004020040 con 0x55eae51c70c0 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f6ff3fff640 1 -- 172.21.15.7:0/1597671266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7004011110 con 0x55eae51c70c0 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.141+0000 7f6ff3fff640 1 -- 172.21.15.7:0/1597671266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700400e9c0 con 0x55eae51c70c0 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 -- 172.21.15.7:0/1597671266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51c70c0 msgr2=0x55eae51df370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51c70c0 0x55eae51df370 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f70040117e0 tx=0x7f7004028000 comp rx=0 tx=0).stop 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 -- 172.21.15.7:0/1597671266 shutdown_connections 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51c70c0 0x55eae51df370 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51b82f0 0x55eae51dee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 --2- 172.21.15.7:0/1597671266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b79d0 0x55eae51b9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 -- 172.21.15.7:0/1597671266 >> 172.21.15.7:0/1597671266 conn(0x55eae51b4060 msgr2=0x55eae51b9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 -- 172.21.15.7:0/1597671266 shutdown_connections 2024-09-10T10:53:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 -- 172.21.15.7:0/1597671266 wait complete. 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 -- start start 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.142+0000 7f70139bc740 1 Processor -- start 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f70139bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f70139bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b82f0 0x55eae51cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f70139bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51c70c0 0x55eae51cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f70139bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f700c1a9bb0 con 0x55eae51b79d0 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f70139bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f700c1ab000 con 0x55eae51b82f0 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f70139bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f700c1aa260 con 0x55eae51c70c0 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f70139bc740 10 client.0 _ll_register_callbacks cb 0x7f7000009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f700a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f700a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53936/0 (socket says 172.21.15.7:53936) 2024-09-10T10:53:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f700a59c640 1 -- 172.21.15.7:0/4047400713 learned_addr learned my addr 172.21.15.7:0/4047400713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f7009d9b640 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b82f0 0x55eae51cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f700ad9d640 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51c70c0 0x55eae51cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f700a59c640 1 -- 172.21.15.7:0/4047400713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51c70c0 msgr2=0x55eae51cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f700a59c640 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51c70c0 0x55eae51cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f700a59c640 1 -- 172.21.15.7:0/4047400713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b82f0 msgr2=0x55eae51cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f700a59c640 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b82f0 0x55eae51cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f700a59c640 1 -- 172.21.15.7:0/4047400713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae51cc720 con 0x55eae51b79d0 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f700a59c640 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51cf2b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00b850 tx=0x7f6ffc00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ce8aa754a83253 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f6ff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/4047400713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc011640 con 0x55eae51b79d0 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/4047400713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f6ffc0117e0 con 0x55eae51b79d0 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.143+0000 7f70139bc740 1 -- 172.21.15.7:0/4047400713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7004011110 con 0x55eae51b79d0 2024-09-10T10:53:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/4047400713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ffc0119e0 con 0x55eae51b79d0 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f70139bc740 10 client.15873 fetch_fsmap learned FSMap version 33 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f70139bc740 1 -- 172.21.15.7:0/4047400713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eae5242770 con 0x55eae51b79d0 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/4047400713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7004011110 con 0x55eae51b79d0 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/4047400713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f6ffc026410 con 0x55eae51b79d0 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f70139bc740 10 client.15873 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f70139bc740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f70139bc740 20 client.15873 populate_metadata read hostname 'smithi007' 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f70139bc740 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55eae5242f70 0x55eae5245390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f70139bc740 4 client.15873 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f70139bc740 1 -- 172.21.15.7:0/4047400713 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eae5123d70 con 0x55eae5242f70 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.144+0000 7f7009d9b640 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55eae5242f70 0x55eae5245390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.145+0000 7f7009d9b640 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55eae5242f70 0x55eae5245390 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653194209f2322f0 server_cookie=54c75f0c87463cac in_seq=0 out_seq=0 2024-09-10T10:53:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.145+0000 7f6ff1ffb640 10 client.15873 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f6ff1ffb640 1 -- 172.21.15.7:0/4047400713 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eae5123d70 con 0x55eae5242f70 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f6ff1ffb640 10 client.15873 handle_command_reply: tid=0 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 client.15873 shutdown 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 -- 172.21.15.7:0/4047400713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 msgr2=0x55eae51cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51cf2b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00b850 tx=0x7f6ffc00bf30 comp rx=0 tx=0).stop 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 -- 172.21.15.7:0/4047400713 shutdown_connections 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55eae5242f70 0x55eae5245390 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f6ff1ffb640 0 client.15873 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eae51c70c0 0x55eae51cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55eae51b82f0 0x55eae51cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 --2- 172.21.15.7:0/4047400713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eae51b79d0 0x55eae51cf2b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 -- 172.21.15.7:0/4047400713 >> 172.21.15.7:0/4047400713 conn(0x55eae51b4060 msgr2=0x55eae51b9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 -- 172.21.15.7:0/4047400713 shutdown_connections 2024-09-10T10:53:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 1 -- 172.21.15.7:0/4047400713 wait complete. 2024-09-10T10:53:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.252+0000 7f70139bc740 20 client.15873 trim_cache size 0 max 16384 2024-09-10T10:53:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:53:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:53:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:53:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:53:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 -- 172.21.15.7:0/1391942540 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6ff40a2e20 msgr2=0x7f6ff40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6ff40a2e20 0x7f6ff40a52e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7004011490 tx=0x7f700401bf80 comp rx=0 tx=0).stop 2024-09-10T10:53:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 -- 172.21.15.7:0/1391942540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c102470 msgr2=0x7f700c198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c102470 0x7f700c198870 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7000009230 tx=0x7f7000004270 comp rx=0 tx=0).stop 2024-09-10T10:53:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 -- 172.21.15.7:0/1391942540 shutdown_connections 2024-09-10T10:53:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700c107b70 0x7f700c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6ff40a2e20 0x7f6ff40a52e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f700c102470 0x7f700c198870 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 --2- 172.21.15.7:0/1391942540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f700c101ad0 0x7f700c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 -- 172.21.15.7:0/1391942540 >> 172.21.15.7:0/1391942540 conn(0x7f700c0fd7d0 msgr2=0x7f700c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 -- 172.21.15.7:0/1391942540 shutdown_connections 2024-09-10T10:53:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:40.253+0000 7f7010b49640 1 -- 172.21.15.7:0/1391942540 wait complete. 2024-09-10T10:53:40.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 878586 11.41 MB/sec execute 280 sec latency 64.574 ms 2024-09-10T10:53:41.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 880611 11.41 MB/sec execute 281 sec latency 212.274 ms 2024-09-10T10:53:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:41 smithi103 ceph-mon[35160]: pgmap v242: 97 pgs: 97 active+clean; 243 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 384 op/s 2024-09-10T10:53:41.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:41 smithi007 ceph-mon[30873]: pgmap v242: 97 pgs: 97 active+clean; 243 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 384 op/s 2024-09-10T10:53:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:41 smithi133 ceph-mon[35282]: pgmap v242: 97 pgs: 97 active+clean; 243 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 384 op/s 2024-09-10T10:53:42.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 882180 11.40 MB/sec execute 282 sec latency 14.565 ms 2024-09-10T10:53:43.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 884467 11.41 MB/sec execute 283 sec latency 16.587 ms 2024-09-10T10:53:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:43 smithi103 ceph-mon[35160]: pgmap v243: 97 pgs: 97 active+clean; 243 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-10T10:53:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:43 smithi007 ceph-mon[30873]: pgmap v243: 97 pgs: 97 active+clean; 243 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-10T10:53:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:43 smithi133 ceph-mon[35282]: pgmap v243: 97 pgs: 97 active+clean; 243 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-10T10:53:44.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 886947 11.41 MB/sec execute 284 sec latency 15.090 ms 2024-09-10T10:53:45.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 888545 11.40 MB/sec execute 285 sec latency 26.925 ms 2024-09-10T10:53:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:45 smithi103 ceph-mon[35160]: pgmap v244: 97 pgs: 97 active+clean; 243 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-10T10:53:45.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:45 smithi007 ceph-mon[30873]: pgmap v244: 97 pgs: 97 active+clean; 243 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-10T10:53:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:45 smithi133 ceph-mon[35282]: pgmap v244: 97 pgs: 97 active+clean; 243 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-10T10:53:46.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 891188 11.42 MB/sec execute 286 sec latency 21.293 ms 2024-09-10T10:53:47.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 892814 11.40 MB/sec execute 287 sec latency 15.073 ms 2024-09-10T10:53:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:47 smithi103 ceph-mon[35160]: pgmap v245: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 390 op/s 2024-09-10T10:53:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:47 smithi007 ceph-mon[30873]: pgmap v245: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 390 op/s 2024-09-10T10:53:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:47 smithi133 ceph-mon[35282]: pgmap v245: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 390 op/s 2024-09-10T10:53:48.521 INFO:tasks.workunit.client.0.smithi007.stdout: 1 895115 11.41 MB/sec execute 288 sec latency 13.051 ms 2024-09-10T10:53:49.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 897648 11.41 MB/sec execute 289 sec latency 16.043 ms 2024-09-10T10:53:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:49 smithi103 ceph-mon[35160]: pgmap v246: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-09-10T10:53:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:49 smithi007 ceph-mon[30873]: pgmap v246: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-09-10T10:53:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:49 smithi133 ceph-mon[35282]: pgmap v246: 97 pgs: 97 active+clean; 236 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-09-10T10:53:50.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 899354 11.40 MB/sec execute 290 sec latency 53.242 ms 2024-09-10T10:53:51.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 901833 11.42 MB/sec execute 291 sec latency 29.141 ms 2024-09-10T10:53:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:51 smithi103 ceph-mon[35160]: pgmap v247: 97 pgs: 97 active+clean; 244 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 390 op/s 2024-09-10T10:53:51.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:51 smithi007 ceph-mon[30873]: pgmap v247: 97 pgs: 97 active+clean; 244 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 390 op/s 2024-09-10T10:53:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:51 smithi133 ceph-mon[35282]: pgmap v247: 97 pgs: 97 active+clean; 244 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 390 op/s 2024-09-10T10:53:52.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 903407 11.40 MB/sec execute 292 sec latency 17.643 ms 2024-09-10T10:53:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.348+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/4227346771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b841009b0 msgr2=0x7f9b84100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.348+0000 7f9b8c4d2640 1 --2- 172.21.15.7:0/4227346771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b841009b0 0x7f9b84100e10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009930 tx=0x7f9b7402a420 comp rx=0 tx=0).stop 2024-09-10T10:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.348+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/4227346771 shutdown_connections 2024-09-10T10:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.348+0000 7f9b8c4d2640 1 --2- 172.21.15.7:0/4227346771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84101350 0x7f9b8410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.348+0000 7f9b8c4d2640 1 --2- 172.21.15.7:0/4227346771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b841009b0 0x7f9b84100e10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.348+0000 7f9b8c4d2640 1 --2- 172.21.15.7:0/4227346771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b841069c0 0x7f9b84106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.348+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/4227346771 >> 172.21.15.7:0/4227346771 conn(0x7f9b84077990 msgr2=0x7f9b840fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.349+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/4227346771 shutdown_connections 2024-09-10T10:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.349+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/4227346771 wait complete. 2024-09-10T10:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.349+0000 7f9b8c4d2640 1 Processor -- start 2024-09-10T10:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.349+0000 7f9b8c4d2640 1 -- start start 2024-09-10T10:53:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8c4d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b841009b0 0x7f9b84198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8c4d2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84101350 0x7f9b84198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8c4d2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b841069c0 0x7f9b8419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8c4d2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84110dc0 con 0x7f9b841009b0 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8c4d2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84110c40 con 0x7f9b84101350 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8c4d2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84110f40 con 0x7f9b841069c0 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8a26e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b841009b0 0x7f9b84198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8a26e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b841009b0 0x7f9b84198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55562/0 (socket says 172.21.15.7:55562) 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8a26e640 1 -- 172.21.15.7:0/1427587815 learned_addr learned my addr 172.21.15.7:0/1427587815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b841069c0 0x7f9b8419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b89a6d640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84101350 0x7f9b84198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8a26e640 1 -- 172.21.15.7:0/1427587815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b841069c0 msgr2=0x7f9b8419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8a26e640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b841069c0 0x7f9b8419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8a26e640 1 -- 172.21.15.7:0/1427587815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84101350 msgr2=0x7f9b84198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8a26e640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84101350 0x7f9b84198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8a26e640 1 -- 172.21.15.7:0/1427587815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8419d2f0 con 0x7f9b841009b0 2024-09-10T10:53:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.350+0000 7f9b8a26e640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b841009b0 0x7f9b84198300 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000e9f0 tx=0x7f9b8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.351+0000 7f9b7b7fe640 1 -- 172.21.15.7:0/1427587815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8000cd80 con 0x7f9b841009b0 2024-09-10T10:53:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.351+0000 7f9b7b7fe640 1 -- 172.21.15.7:0/1427587815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b80010910 con 0x7f9b841009b0 2024-09-10T10:53:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.351+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/1427587815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8419d5e0 con 0x7f9b841009b0 2024-09-10T10:53:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.351+0000 7f9b7b7fe640 1 -- 172.21.15.7:0/1427587815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80018990 con 0x7f9b841009b0 2024-09-10T10:53:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.351+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/1427587815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b841a4e70 con 0x7f9b841009b0 2024-09-10T10:53:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.353+0000 7f9b7b7fe640 1 -- 172.21.15.7:0/1427587815 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9b80010ac0 con 0x7f9b841009b0 2024-09-10T10:53:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8f264740 1 Processor -- start 2024-09-10T10:53:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8f264740 1 -- start start 2024-09-10T10:53:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605f9dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9e2d0 0x562605facb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605faf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8f264740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562605f022f0 con 0x562605fad0c0 2024-09-10T10:53:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8f264740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562605eaa270 con 0x562605f9d9b0 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8f264740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562605efe2b0 con 0x562605f9e2d0 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b7b7fe640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9b700a2e20 0x7f9b700a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b89a6d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605faf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9b700a2e20 0x7f9b700a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8a26e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9e2d0 0x562605facb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b89a6d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605faf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55576/0 (socket says 172.21.15.7:55576) 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b8aa6f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605f9dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.354+0000 7f9b89a6d640 1 -- 172.21.15.7:0/3699681056 learned_addr learned my addr 172.21.15.7:0/3699681056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b7b7fe640 1 -- 172.21.15.7:0/1427587815 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f9b800cf6b0 con 0x7f9b841009b0 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b8aa6f640 1 -- 172.21.15.7:0/3699681056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9e2d0 msgr2=0x562605facb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/3699681056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9e2d0 0x562605facb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b8aa6f640 1 -- 172.21.15.7:0/3699681056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 msgr2=0x562605faf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/3699681056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605faf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b8aa6f640 1 -- 172.21.15.7:0/3699681056 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562605ea9340 con 0x562605f9d9b0 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/3699681056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605f9dd90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c010d60 tx=0x7f9b7c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ba1f45b3cdd8e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9b700a2e20 0x7f9b700a52e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00fc90 tx=0x7f9b7c018000 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b78bfe640 1 -- 172.21.15.7:0/3699681056 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c019040 con 0x562605f9d9b0 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b78bfe640 1 -- 172.21.15.7:0/3699681056 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7c00ee50 con 0x562605f9d9b0 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.355+0000 7f9b78bfe640 1 -- 172.21.15.7:0/3699681056 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c004020 con 0x562605f9d9b0 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 -- 172.21.15.7:0/3699681056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 msgr2=0x562605f9dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 --2- 172.21.15.7:0/3699681056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605f9dd90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c010d60 tx=0x7f9b7c00f790 comp rx=0 tx=0).stop 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 -- 172.21.15.7:0/3699681056 shutdown_connections 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 --2- 172.21.15.7:0/3699681056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605faf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 --2- 172.21.15.7:0/3699681056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9e2d0 0x562605facb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 --2- 172.21.15.7:0/3699681056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605f9dd90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 -- 172.21.15.7:0/3699681056 >> 172.21.15.7:0/3699681056 conn(0x562605f9a060 msgr2=0x562605f9a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 -- 172.21.15.7:0/3699681056 shutdown_connections 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 -- 172.21.15.7:0/3699681056 wait complete. 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 -- start start 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.356+0000 7f9b8f264740 1 Processor -- start 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605f9d9b0 0x562605faa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9e2d0 0x562605faa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605fad0c0 0x562605faab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8f264740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562605fafc30 con 0x562605f9d9b0 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8f264740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562605e8b290 con 0x562605f9e2d0 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8f264740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562605fafdb0 con 0x562605fad0c0 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8f264740 10 client.0 _ll_register_callbacks cb 0x562605ebc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8a26e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605f9d9b0 0x562605faa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8a26e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605f9d9b0 0x562605faa080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55592/0 (socket says 172.21.15.7:55592) 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8a26e640 1 -- 172.21.15.7:0/1397619554 learned_addr learned my addr 172.21.15.7:0/1397619554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605fad0c0 0x562605faab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b89a6d640 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9e2d0 0x562605faa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8a26e640 1 -- 172.21.15.7:0/1397619554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605fad0c0 msgr2=0x562605faab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8a26e640 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605fad0c0 0x562605faab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8a26e640 1 -- 172.21.15.7:0/1397619554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9e2d0 msgr2=0x562605faa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8a26e640 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9e2d0 0x562605faa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8a26e640 1 -- 172.21.15.7:0/1397619554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562605c15210 con 0x562605f9d9b0 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b8a26e640 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605f9d9b0 0x562605faa080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000e9c0 tx=0x7f9b8006abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2b049f9dd0036e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.357+0000 7f9b6dffb640 1 -- 172.21.15.7:0/1397619554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800cd200 con 0x562605f9d9b0 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b6dffb640 1 -- 172.21.15.7:0/1397619554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f9b800a4520 con 0x562605f9d9b0 2024-09-10T10:53:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b8f264740 1 -- 172.21.15.7:0/1397619554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b7c00ee50 con 0x562605f9d9b0 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b6dffb640 1 -- 172.21.15.7:0/1397619554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9b800a47b0 con 0x562605f9d9b0 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b8f264740 10 client.15885 fetch_fsmap learned FSMap version 33 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b8f264740 1 -- 172.21.15.7:0/1397619554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562605ce1620 con 0x562605f9d9b0 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b6dffb640 1 -- 172.21.15.7:0/1397619554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7c00ee50 con 0x562605f9d9b0 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b6dffb640 1 -- 172.21.15.7:0/1397619554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f9b8009b040 con 0x562605f9d9b0 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b8f264740 10 client.15885 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b8f264740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b8f264740 20 client.15885 populate_metadata read hostname 'smithi007' 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b8f264740 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562605fbd080 0x562605fbf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b8f264740 4 client.15885 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b8f264740 1 -- 172.21.15.7:0/1397619554 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562605f0fa20 con 0x562605fbd080 2024-09-10T10:53:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.358+0000 7f9b89a6d640 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562605fbd080 0x562605fbf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.359+0000 7f9b89a6d640 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562605fbd080 0x562605fbf470 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46fb37cf922e9fe3 server_cookie=b44a0015fb720e7b in_seq=0 out_seq=0 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.359+0000 7f9b6dffb640 10 client.15885 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.359+0000 7f9b6dffb640 1 -- 172.21.15.7:0/1397619554 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562605f0fa20 con 0x562605fbd080 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.359+0000 7f9b6dffb640 10 client.15885 handle_command_reply: tid=0 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.359+0000 7f9b8f264740 1 client.15885 shutdown 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 1 -- 172.21.15.7:0/1397619554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605f9d9b0 msgr2=0x562605faa080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605f9d9b0 0x562605faa080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000e9c0 tx=0x7f9b8006abe0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 1 -- 172.21.15.7:0/1397619554 shutdown_connections 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562605fbd080 0x562605fbf470 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605fad0c0 0x562605faab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9e2d0 0x562605faa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 1 --2- 172.21.15.7:0/1397619554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605f9d9b0 0x562605faa080 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b6dffb640 0 client.15885 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 1 -- 172.21.15.7:0/1397619554 >> 172.21.15.7:0/1397619554 conn(0x562605f9a060 msgr2=0x562605f9fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 1 -- 172.21.15.7:0/1397619554 shutdown_connections 2024-09-10T10:53:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 1 -- 172.21.15.7:0/1397619554 wait complete. 2024-09-10T10:53:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.360+0000 7f9b8f264740 20 client.15885 trim_cache size 0 max 16384 2024-09-10T10:53:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.375+0000 7f9b8f264740 1 Processor -- start 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.375+0000 7f9b8f264740 1 -- start start 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.375+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9d9b0 0x562605faa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.375+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9e2d0 0x562605fc4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.375+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fc5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.375+0000 7f9b8f264740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841adec0 con 0x562605fad0c0 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.375+0000 7f9b8f264740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841ab760 con 0x562605f9e2d0 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.375+0000 7f9b8f264740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841a95d0 con 0x562605f9d9b0 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b8aa6f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fc5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b8aa6f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fc5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55600/0 (socket says 172.21.15.7:55600) 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b8aa6f640 1 -- 172.21.15.7:0/251280036 learned_addr learned my addr 172.21.15.7:0/251280036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b8a26e640 1 --2- 172.21.15.7:0/251280036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9d9b0 0x562605faa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b89a6d640 1 --2- 172.21.15.7:0/251280036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9e2d0 0x562605fc4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b8aa6f640 1 -- 172.21.15.7:0/251280036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9d9b0 msgr2=0x562605faa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/251280036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9d9b0 0x562605faa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b8aa6f640 1 -- 172.21.15.7:0/251280036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9e2d0 msgr2=0x562605fc4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/251280036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9e2d0 0x562605fc4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b8aa6f640 1 -- 172.21.15.7:0/251280036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562605f9e100 con 0x562605fad0c0 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/251280036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fc5250 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c003b20 tx=0x7f9b7c003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacde7691cf351ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b6ffff640 1 -- 172.21.15.7:0/251280036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c003de0 con 0x562605fad0c0 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b6ffff640 1 -- 172.21.15.7:0/251280036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7c00ec20 con 0x562605fad0c0 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.376+0000 7f9b6ffff640 1 -- 172.21.15.7:0/251280036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0055a0 con 0x562605fad0c0 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.377+0000 7f9b8f264740 1 -- 172.21.15.7:0/251280036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 msgr2=0x562605fc5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.377+0000 7f9b8f264740 1 --2- 172.21.15.7:0/251280036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fc5250 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c003b20 tx=0x7f9b7c003ab0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.377+0000 7f9b8f264740 1 -- 172.21.15.7:0/251280036 shutdown_connections 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.377+0000 7f9b8f264740 1 --2- 172.21.15.7:0/251280036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fc5250 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.377+0000 7f9b8f264740 1 --2- 172.21.15.7:0/251280036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9e2d0 0x562605fc4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.377+0000 7f9b8f264740 1 --2- 172.21.15.7:0/251280036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9d9b0 0x562605faa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.377+0000 7f9b8f264740 1 -- 172.21.15.7:0/251280036 >> 172.21.15.7:0/251280036 conn(0x562605f9a060 msgr2=0x562605faa040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.377+0000 7f9b8f264740 1 -- 172.21.15.7:0/251280036 shutdown_connections 2024-09-10T10:53:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.377+0000 7f9b8f264740 1 -- 172.21.15.7:0/251280036 wait complete. 2024-09-10T10:53:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.377+0000 7f9b8f264740 1 -- start start 2024-09-10T10:53:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8f264740 1 Processor -- start 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605fb5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9e2d0 0x562605fb58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8f264740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fa6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8f264740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841a9c20 con 0x562605fad0c0 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8f264740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841ab090 con 0x562605f9d9b0 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8f264740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841aa2f0 con 0x562605f9e2d0 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8f264740 10 client.0 _ll_register_callbacks cb 0x7f9b80009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8aa6f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fa6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8a26e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605fb5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8a26e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605fb5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:33612/0 (socket says 172.21.15.7:33612) 2024-09-10T10:53:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8a26e640 1 -- 172.21.15.7:0/2284457014 learned_addr learned my addr 172.21.15.7:0/2284457014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b89a6d640 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9e2d0 0x562605fb58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8aa6f640 1 -- 172.21.15.7:0/2284457014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9e2d0 msgr2=0x562605fb58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9e2d0 0x562605fb58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8aa6f640 1 -- 172.21.15.7:0/2284457014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 msgr2=0x562605fb5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605fb5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8aa6f640 1 -- 172.21.15.7:0/2284457014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562605fa7180 con 0x562605fad0c0 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.378+0000 7f9b8a26e640 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605fb5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b8aa6f640 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fa6980 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00f7d0 tx=0x7f9b7c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa70a9908b59c84b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b6dffb640 1 -- 172.21.15.7:0/2284457014 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c005320 con 0x562605fad0c0 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b6dffb640 1 -- 172.21.15.7:0/2284457014 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8564+0+0 (secure 0 0 0) 0x7f9b7c0054c0 con 0x562605fad0c0 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b8f264740 1 -- 172.21.15.7:0/2284457014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b7c00ec20 con 0x562605fad0c0 2024-09-10T10:53:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b6dffb640 1 -- 172.21.15.7:0/2284457014 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9b7c0056c0 con 0x562605fad0c0 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b8f264740 10 client.15891 fetch_fsmap learned FSMap version 33 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b8f264740 1 -- 172.21.15.7:0/2284457014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5626060278b0 con 0x562605fad0c0 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b6dffb640 1 -- 172.21.15.7:0/2284457014 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7c00ec20 con 0x562605fad0c0 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b6dffb640 1 -- 172.21.15.7:0/2284457014 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 33) ==== 8350+0+0 (secure 0 0 0) 0x7f9b7c030070 con 0x562605fad0c0 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b8f264740 10 client.15891 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b8f264740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b8f264740 20 client.15891 populate_metadata read hostname 'smithi007' 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b8f264740 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562606027dc0 0x56260602a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b8f264740 4 client.15891 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.379+0000 7f9b8f264740 1 -- 172.21.15.7:0/2284457014 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562605f09d70 con 0x562606027dc0 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.380+0000 7f9b8a26e640 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562606027dc0 0x56260602a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.380+0000 7f9b8a26e640 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562606027dc0 0x56260602a1b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be2ba3a7cd5f16d2 server_cookie=259bc35e94007eb1 in_seq=0 out_seq=0 2024-09-10T10:53:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.380+0000 7f9b6dffb640 10 client.15891 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b6dffb640 1 -- 172.21.15.7:0/2284457014 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562605f09d70 con 0x562606027dc0 2024-09-10T10:53:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b6dffb640 10 client.15891 handle_command_reply: tid=0 2024-09-10T10:53:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 client.15891 shutdown 2024-09-10T10:53:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 -- 172.21.15.7:0/2284457014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 msgr2=0x562605fa6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fa6980 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00f7d0 tx=0x7f9b7c003970 comp rx=0 tx=0).stop 2024-09-10T10:53:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 -- 172.21.15.7:0/2284457014 shutdown_connections 2024-09-10T10:53:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562606027dc0 0x56260602a1b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b6dffb640 0 client.15891 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:53:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562605fad0c0 0x562605fa6980 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562605f9e2d0 0x562605fb58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 --2- 172.21.15.7:0/2284457014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562605f9d9b0 0x562605fb5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 -- 172.21.15.7:0/2284457014 >> 172.21.15.7:0/2284457014 conn(0x562605f9a060 msgr2=0x562605fa84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 -- 172.21.15.7:0/2284457014 shutdown_connections 2024-09-10T10:53:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 1 -- 172.21.15.7:0/2284457014 wait complete. 2024-09-10T10:53:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.447+0000 7f9b8f264740 20 client.15891 trim_cache size 0 max 16384 2024-09-10T10:53:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:53:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:53:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:53:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:53:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/1427587815 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9b700a2e20 msgr2=0x7f9b700a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9b700a2e20 0x7f9b700a52e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00fc90 tx=0x7f9b7c018000 comp rx=0 tx=0).stop 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/1427587815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b841009b0 msgr2=0x7f9b84198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b841009b0 0x7f9b84198300 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000e9f0 tx=0x7f9b8000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/1427587815 shutdown_connections 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b841069c0 0x7f9b8419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b84101350 0x7f9b84198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9b700a2e20 0x7f9b700a52e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 --2- 172.21.15.7:0/1427587815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b841009b0 0x7f9b84198300 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/1427587815 >> 172.21.15.7:0/1427587815 conn(0x7f9b84077990 msgr2=0x7f9b840699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/1427587815 shutdown_connections 2024-09-10T10:53:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:53:53.448+0000 7f9b8c4d2640 1 -- 172.21.15.7:0/1427587815 wait complete. 2024-09-10T10:53:53.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 905602 11.41 MB/sec execute 293 sec latency 33.019 ms 2024-09-10T10:53:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:53 smithi103 ceph-mon[35160]: pgmap v248: 97 pgs: 97 active+clean; 244 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2024-09-10T10:53:53.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:53 smithi007 ceph-mon[30873]: pgmap v248: 97 pgs: 97 active+clean; 244 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2024-09-10T10:53:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:53 smithi133 ceph-mon[35282]: pgmap v248: 97 pgs: 97 active+clean; 244 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2024-09-10T10:53:54.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 907668 11.39 MB/sec execute 294 sec latency 15.039 ms 2024-09-10T10:53:55.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 909431 11.40 MB/sec execute 295 sec latency 13.344 ms 2024-09-10T10:53:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:55 smithi103 ceph-mon[35160]: pgmap v249: 97 pgs: 97 active+clean; 244 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2024-09-10T10:53:55.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:55 smithi007 ceph-mon[30873]: pgmap v249: 97 pgs: 97 active+clean; 244 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2024-09-10T10:53:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:55 smithi133 ceph-mon[35282]: pgmap v249: 97 pgs: 97 active+clean; 244 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2024-09-10T10:53:56.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 911753 11.40 MB/sec execute 296 sec latency 33.503 ms 2024-09-10T10:53:57.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 913294 11.39 MB/sec execute 297 sec latency 11.892 ms 2024-09-10T10:53:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:57 smithi103 ceph-mon[35160]: pgmap v250: 97 pgs: 97 active+clean; 240 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 434 op/s 2024-09-10T10:53:57.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:57 smithi007 ceph-mon[30873]: pgmap v250: 97 pgs: 97 active+clean; 240 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 434 op/s 2024-09-10T10:53:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:57 smithi133 ceph-mon[35282]: pgmap v250: 97 pgs: 97 active+clean; 240 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 434 op/s 2024-09-10T10:53:58.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 915771 11.39 MB/sec execute 298 sec latency 17.049 ms 2024-09-10T10:53:59.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 916947 11.40 MB/sec execute 299 sec latency 44.344 ms 2024-09-10T10:53:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:53:59 smithi103 ceph-mon[35160]: pgmap v251: 97 pgs: 97 active+clean; 240 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 279 op/s 2024-09-10T10:53:59.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:53:59 smithi007 ceph-mon[30873]: pgmap v251: 97 pgs: 97 active+clean; 240 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 279 op/s 2024-09-10T10:53:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:53:59 smithi133 ceph-mon[35282]: pgmap v251: 97 pgs: 97 active+clean; 240 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 279 op/s 2024-09-10T10:54:00.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 918946 11.42 MB/sec execute 300 sec latency 16.058 ms 2024-09-10T10:54:01.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920361 11.43 MB/sec execute 301 sec latency 72.808 ms 2024-09-10T10:54:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:01 smithi103 ceph-mon[35160]: pgmap v252: 97 pgs: 97 active+clean; 248 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 395 op/s 2024-09-10T10:54:01.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:01 smithi007 ceph-mon[30873]: pgmap v252: 97 pgs: 97 active+clean; 248 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 395 op/s 2024-09-10T10:54:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:01 smithi133 ceph-mon[35282]: pgmap v252: 97 pgs: 97 active+clean; 248 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 395 op/s 2024-09-10T10:54:02.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.39 MB/sec execute 302 sec latency 423.250 ms 2024-09-10T10:54:02.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:02 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:02.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:02 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:54:02.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:02 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:02.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:02 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:54:02.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:02 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:02.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:02 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:54:03.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.35 MB/sec execute 303 sec latency 1423.282 ms 2024-09-10T10:54:03.876 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:03 smithi133 ceph-mon[35282]: pgmap v253: 97 pgs: 97 active+clean; 248 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-09-10T10:54:03.876 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:03 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:54:03.877 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:03 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:54:03.877 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:03 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:54:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:03 smithi103 ceph-mon[35160]: pgmap v253: 97 pgs: 97 active+clean; 248 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-09-10T10:54:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:03 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:54:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:03 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:54:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:03 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:54:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:03 smithi007 ceph-mon[30873]: pgmap v253: 97 pgs: 97 active+clean; 248 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-09-10T10:54:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:03 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:54:03.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:03 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:54:03.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:03 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:54:04.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.32 MB/sec execute 304 sec latency 2423.314 ms 2024-09-10T10:54:04.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:04 smithi103 ceph-mon[35160]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T10:54:04.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:04 smithi007 ceph-mon[30873]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T10:54:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:04 smithi133 ceph-mon[35282]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T10:54:05.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.28 MB/sec execute 305 sec latency 3423.378 ms 2024-09-10T10:54:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:05 smithi103 ceph-mon[35160]: pgmap v255: 97 pgs: 97 active+clean; 248 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 328 op/s 2024-09-10T10:54:05.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:05 smithi007 ceph-mon[30873]: pgmap v255: 97 pgs: 97 active+clean; 248 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 328 op/s 2024-09-10T10:54:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:05 smithi133 ceph-mon[35282]: pgmap v255: 97 pgs: 97 active+clean; 248 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 328 op/s 2024-09-10T10:54:06.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.24 MB/sec execute 306 sec latency 4423.434 ms 2024-09-10T10:54:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.556+0000 7f08bdf4b640 1 -- 172.21.15.7:0/2288823847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b81009f0 msgr2=0x7f08b8100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.556+0000 7f08bdf4b640 1 --2- 172.21.15.7:0/2288823847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b81009f0 0x7f08b8100e50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009930 tx=0x7f08ac02a420 comp rx=0 tx=0).stop 2024-09-10T10:54:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.556+0000 7f08bdf4b640 1 -- 172.21.15.7:0/2288823847 shutdown_connections 2024-09-10T10:54:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.556+0000 7f08bdf4b640 1 --2- 172.21.15.7:0/2288823847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b8101390 0x7f08b810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.556+0000 7f08bdf4b640 1 --2- 172.21.15.7:0/2288823847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b81009f0 0x7f08b8100e50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.556+0000 7f08bdf4b640 1 --2- 172.21.15.7:0/2288823847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08b8106a00 0x7f08b8106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.556+0000 7f08bdf4b640 1 -- 172.21.15.7:0/2288823847 >> 172.21.15.7:0/2288823847 conn(0x7f08b80fc6b0 msgr2=0x7f08b80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.556+0000 7f08bdf4b640 1 -- 172.21.15.7:0/2288823847 shutdown_connections 2024-09-10T10:54:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.556+0000 7f08bdf4b640 1 -- 172.21.15.7:0/2288823847 wait complete. 2024-09-10T10:54:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08bdf4b640 1 Processor -- start 2024-09-10T10:54:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08bdf4b640 1 -- start start 2024-09-10T10:54:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08bdf4b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81009f0 0x7f08b8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08bdf4b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08b8101390 0x7f08b8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08bdf4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8106a00 0x7f08b819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08bdf4b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8110e40 con 0x7f08b8106a00 2024-09-10T10:54:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08bdf4b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8110cc0 con 0x7f08b8101390 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08bdf4b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8110fc0 con 0x7f08b81009f0 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08b7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8106a00 0x7f08b819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08b7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8106a00 0x7f08b819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57162/0 (socket says 172.21.15.7:57162) 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.557+0000 7f08b7fff640 1 -- 172.21.15.7:0/1487519106 learned_addr learned my addr 172.21.15.7:0/1487519106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08b8101390 0x7f08b8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b77fe640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81009f0 0x7f08b8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b7fff640 1 -- 172.21.15.7:0/1487519106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81009f0 msgr2=0x7f08b8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b7fff640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81009f0 0x7f08b8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b7fff640 1 -- 172.21.15.7:0/1487519106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08b8101390 msgr2=0x7f08b8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b7fff640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08b8101390 0x7f08b8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b7fff640 1 -- 172.21.15.7:0/1487519106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b819d400 con 0x7f08b8106a00 2024-09-10T10:54:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b7fff640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8106a00 0x7f08b819ccd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f08b000cce0 tx=0x7f08b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b4ff9640 1 -- 172.21.15.7:0/1487519106 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b000f450 con 0x7f08b8106a00 2024-09-10T10:54:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b4ff9640 1 -- 172.21.15.7:0/1487519106 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b000fad0 con 0x7f08b8106a00 2024-09-10T10:54:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08b4ff9640 1 -- 172.21.15.7:0/1487519106 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0018c10 con 0x7f08b8106a00 2024-09-10T10:54:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08bdf4b640 1 -- 172.21.15.7:0/1487519106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b819d6f0 con 0x7f08b8106a00 2024-09-10T10:54:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.558+0000 7f08bdf4b640 1 -- 172.21.15.7:0/1487519106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b81a4f80 con 0x7f08b8106a00 2024-09-10T10:54:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.561+0000 7f08b4ff9640 1 -- 172.21.15.7:0/1487519106 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f08b000f5f0 con 0x7f08b8106a00 2024-09-10T10:54:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.561+0000 7f08c0dc4740 1 Processor -- start 2024-09-10T10:54:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.561+0000 7f08c0dc4740 1 -- start start 2024-09-10T10:54:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.561+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c1af0 0x563fa72c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.561+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72c2410 0x563fa72cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.561+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72cccb0 0x563fa72cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.561+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563fa72262f0 con 0x563fa72cccb0 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.561+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563fa71ce270 con 0x563fa72c2410 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.561+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563fa72222b0 con 0x563fa72c1af0 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72cccb0 0x563fa72cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72cccb0 0x563fa72cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57170/0 (socket says 172.21.15.7:57170) 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b6ffd640 1 -- 172.21.15.7:0/4194989662 learned_addr learned my addr 172.21.15.7:0/4194989662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b4ff9640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08a00a2e20 0x7f08a00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b77fe640 1 --2- 172.21.15.7:0/4194989662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72c2410 0x563fa72cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/4194989662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c1af0 0x563fa72c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b6ffd640 1 -- 172.21.15.7:0/4194989662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c1af0 msgr2=0x563fa72c1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/4194989662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c1af0 0x563fa72c1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b7fff640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08a00a2e20 0x7f08a00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b6ffd640 1 -- 172.21.15.7:0/4194989662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72c2410 msgr2=0x563fa72cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/4194989662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72c2410 0x563fa72cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b6ffd640 1 -- 172.21.15.7:0/4194989662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa71cd340 con 0x563fa72cccb0 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b77fe640 1 --2- 172.21.15.7:0/4194989662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72c2410 0x563fa72cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b4ff9640 1 -- 172.21.15.7:0/1487519106 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8608+0+0 (secure 0 0 0) 0x7f08b00cfb10 con 0x7f08b8106a00 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/4194989662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72cccb0 0x563fa72cf0a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f08ac02a940 tx=0x7f08ac005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2cf50ed066d919 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f089dffb640 1 -- 172.21.15.7:0/4194989662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac035070 con 0x563fa72cccb0 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f089dffb640 1 -- 172.21.15.7:0/4194989662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ac031730 con 0x563fa72cccb0 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.562+0000 7f089dffb640 1 -- 172.21.15.7:0/4194989662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0047a0 con 0x563fa72cccb0 2024-09-10T10:54:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08c0dc4740 1 -- 172.21.15.7:0/4194989662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72cccb0 msgr2=0x563fa72cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/4194989662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72cccb0 0x563fa72cf0a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f08ac02a940 tx=0x7f08ac005c00 comp rx=0 tx=0).stop 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08b7fff640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08a00a2e20 0x7f08a00a52e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08b009bf70 tx=0x7f08b00c6b70 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08c0dc4740 1 -- 172.21.15.7:0/4194989662 shutdown_connections 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/4194989662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72cccb0 0x563fa72cf0a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/4194989662 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72c2410 0x563fa72cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/4194989662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c1af0 0x563fa72c1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08c0dc4740 1 -- 172.21.15.7:0/4194989662 >> 172.21.15.7:0/4194989662 conn(0x563fa72be060 msgr2=0x563fa72be440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08c0dc4740 1 -- 172.21.15.7:0/4194989662 shutdown_connections 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08c0dc4740 1 -- 172.21.15.7:0/4194989662 wait complete. 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.563+0000 7f08c0dc4740 1 -- start start 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08c0dc4740 1 Processor -- start 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 0x563fa72ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563fa72cf880 con 0x563fa72c1af0 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563fa71af290 con 0x563fa72cccb0 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563fa72cfa00 con 0x563fa72c2410 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08c0dc4740 10 client.0 _ll_register_callbacks cb 0x563fa71e0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08b6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08b6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57180/0 (socket says 172.21.15.7:57180) 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08b6ffd640 1 -- 172.21.15.7:0/439892596 learned_addr learned my addr 172.21.15.7:0/439892596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08b77fe640 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 0x563fa72ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08b6ffd640 1 -- 172.21.15.7:0/439892596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 msgr2=0x563fa72ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 0x563fa72ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08b6ffd640 1 -- 172.21.15.7:0/439892596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 msgr2=0x563fa72ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.564+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f08b6ffd640 1 -- 172.21.15.7:0/439892596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa6f39210 con 0x563fa72c1af0 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f08b77fe640 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 0x563fa72ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72c9bb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f08ac046040 tx=0x7f08ac005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1be6e08539b7db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f0896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f0896ffd640 1 -- 172.21.15.7:0/439892596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac050030 con 0x563fa72c1af0 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f0896ffd640 1 -- 172.21.15.7:0/439892596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8608+0+0 (secure 0 0 0) 0x7f08ac03d450 con 0x563fa72c1af0 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f08c0dc4740 1 -- 172.21.15.7:0/439892596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ac031730 con 0x563fa72c1af0 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f0896ffd640 1 -- 172.21.15.7:0/439892596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f08ac035040 con 0x563fa72c1af0 2024-09-10T10:54:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f08c0dc4740 10 client.15903 fetch_fsmap learned FSMap version 34 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f08c0dc4740 1 -- 172.21.15.7:0/439892596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563fa7005620 con 0x563fa72c1af0 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f0896ffd640 1 -- 172.21.15.7:0/439892596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ac031730 con 0x563fa72c1af0 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.565+0000 7f0896ffd640 1 -- 172.21.15.7:0/439892596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f08ac004ae0 con 0x563fa72c1af0 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.566+0000 7f08c0dc4740 10 client.15903 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.566+0000 7f08c0dc4740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.566+0000 7f08c0dc4740 20 client.15903 populate_metadata read hostname 'smithi007' 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.566+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563fa72dcae0 0x563fa72deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.566+0000 7f08c0dc4740 4 client.15903 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.566+0000 7f08c0dc4740 1 -- 172.21.15.7:0/439892596 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fa7233a20 con 0x563fa72dcae0 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.566+0000 7f08b77fe640 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563fa72dcae0 0x563fa72deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.566+0000 7f08b77fe640 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563fa72dcae0 0x563fa72deed0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c169786ae11ea52a server_cookie=1dfc0ff4a7075c3b in_seq=0 out_seq=0 2024-09-10T10:54:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.566+0000 7f0896ffd640 10 client.15903 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f0896ffd640 1 -- 172.21.15.7:0/439892596 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fa7233a20 con 0x563fa72dcae0 2024-09-10T10:54:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f0896ffd640 10 client.15903 handle_command_reply: tid=0 2024-09-10T10:54:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 client.15903 shutdown 2024-09-10T10:54:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 -- 172.21.15.7:0/439892596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 msgr2=0x563fa72c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72c9bb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f08ac046040 tx=0x7f08ac005400 comp rx=0 tx=0).stop 2024-09-10T10:54:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 -- 172.21.15.7:0/439892596 shutdown_connections 2024-09-10T10:54:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563fa72dcae0 0x563fa72deed0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f0896ffd640 0 client.15903 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 0x563fa72ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/439892596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72c9bb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 -- 172.21.15.7:0/439892596 >> 172.21.15.7:0/439892596 conn(0x563fa72be060 msgr2=0x563fa72c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 -- 172.21.15.7:0/439892596 shutdown_connections 2024-09-10T10:54:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 1 -- 172.21.15.7:0/439892596 wait complete. 2024-09-10T10:54:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.567+0000 7f08c0dc4740 20 client.15903 trim_cache size 0 max 16384 2024-09-10T10:54:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08c0dc4740 1 Processor -- start 2024-09-10T10:54:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08c0dc4740 1 -- start start 2024-09-10T10:54:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 0x563fa72ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81b0170 con 0x563fa72c1af0 2024-09-10T10:54:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81ad9e0 con 0x563fa72cccb0 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81ab830 con 0x563fa72c2410 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08b6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08b6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57196/0 (socket says 172.21.15.7:57196) 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08b6ffd640 1 -- 172.21.15.7:0/4243823402 learned_addr learned my addr 172.21.15.7:0/4243823402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08b77fe640 1 --2- 172.21.15.7:0/4243823402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 0x563fa72ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08b6ffd640 1 -- 172.21.15.7:0/4243823402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 msgr2=0x563fa72ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/4243823402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 0x563fa72ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08b6ffd640 1 -- 172.21.15.7:0/4243823402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 msgr2=0x563fa72e44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/4243823402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.582+0000 7f08b6ffd640 1 -- 172.21.15.7:0/4243823402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa72c8100 con 0x563fa72c1af0 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/4243823402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72cb410 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f08ac02a940 tx=0x7f08ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7af70e8bf815c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f089d7fa640 1 -- 172.21.15.7:0/4243823402 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac061020 con 0x563fa72c1af0 2024-09-10T10:54:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f089d7fa640 1 -- 172.21.15.7:0/4243823402 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ac005120 con 0x563fa72c1af0 2024-09-10T10:54:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f089d7fa640 1 -- 172.21.15.7:0/4243823402 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac050040 con 0x563fa72c1af0 2024-09-10T10:54:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f08c0dc4740 1 -- 172.21.15.7:0/4243823402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 msgr2=0x563fa72cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/4243823402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72cb410 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f08ac02a940 tx=0x7f08ac000bc0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f08c0dc4740 1 -- 172.21.15.7:0/4243823402 shutdown_connections 2024-09-10T10:54:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/4243823402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72e44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/4243823402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c2410 0x563fa72ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/4243823402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c1af0 0x563fa72cb410 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f08c0dc4740 1 -- 172.21.15.7:0/4243823402 >> 172.21.15.7:0/4243823402 conn(0x563fa72be060 msgr2=0x563fa72d8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f08c0dc4740 1 -- 172.21.15.7:0/4243823402 shutdown_connections 2024-09-10T10:54:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.583+0000 7f08c0dc4740 1 -- 172.21.15.7:0/4243823402 wait complete. 2024-09-10T10:54:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08c0dc4740 1 -- start start 2024-09-10T10:54:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08c0dc4740 1 Processor -- start 2024-09-10T10:54:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c1af0 0x563fa72c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c2410 0x563fa72c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08c0dc4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81abea0 con 0x563fa72c2410 2024-09-10T10:54:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81ad310 con 0x563fa72cccb0 2024-09-10T10:54:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08c0dc4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08b81ac570 con 0x563fa72c1af0 2024-09-10T10:54:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08c0dc4740 10 client.0 _ll_register_callbacks cb 0x7f08ac032ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08b77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c2410 0x563fa72c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08b77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c2410 0x563fa72c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57208/0 (socket says 172.21.15.7:57208) 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08b77fe640 1 -- 172.21.15.7:0/3592637466 learned_addr learned my addr 172.21.15.7:0/3592637466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.584+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c1af0 0x563fa72c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08b77fe640 1 -- 172.21.15.7:0/3592637466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c1af0 msgr2=0x563fa72c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08b77fe640 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c1af0 0x563fa72c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08b77fe640 1 -- 172.21.15.7:0/3592637466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 msgr2=0x563fa72c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08b77fe640 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08b77fe640 1 -- 172.21.15.7:0/3592637466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa734a880 con 0x563fa72c2410 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08b77fe640 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c2410 0x563fa72c70c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f08a8000c00 tx=0x7f08a8013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfebdcd0cc302d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f0896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:54:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f0896ffd640 1 -- 172.21.15.7:0/3592637466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8004440 con 0x563fa72c2410 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f0896ffd640 1 -- 172.21.15.7:0/3592637466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8608+0+0 (secure 0 0 0) 0x7f08a8005020 con 0x563fa72c2410 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08c0dc4740 1 -- 172.21.15.7:0/3592637466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ac005120 con 0x563fa72c2410 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f0896ffd640 1 -- 172.21.15.7:0/3592637466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f08a802a050 con 0x563fa72c2410 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08c0dc4740 10 client.15909 fetch_fsmap learned FSMap version 34 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f08c0dc4740 1 -- 172.21.15.7:0/3592637466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563fa734bb90 con 0x563fa72c2410 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.585+0000 7f0896ffd640 1 -- 172.21.15.7:0/3592637466 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ac005120 con 0x563fa72c2410 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.586+0000 7f0896ffd640 1 -- 172.21.15.7:0/3592637466 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f08a80035f0 con 0x563fa72c2410 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.586+0000 7f08c0dc4740 10 client.15909 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.586+0000 7f08c0dc4740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.586+0000 7f08c0dc4740 20 client.15909 populate_metadata read hostname 'smithi007' 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.586+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563fa734c370 0x563fa734e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.586+0000 7f08c0dc4740 4 client.15909 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:54:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.586+0000 7f08c0dc4740 1 -- 172.21.15.7:0/3592637466 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fa722dd70 con 0x563fa734c370 2024-09-10T10:54:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.586+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563fa734c370 0x563fa734e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.586+0000 7f0896ffd640 10 client.15909 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.586+0000 7f08b6ffd640 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563fa734c370 0x563fa734e790 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8789aeda22486f96 server_cookie=14890bed90b851de in_seq=0 out_seq=0 2024-09-10T10:54:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.664+0000 7f0896ffd640 1 -- 172.21.15.7:0/3592637466 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fa722dd70 con 0x563fa734c370 2024-09-10T10:54:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.664+0000 7f0896ffd640 10 client.15909 handle_command_reply: tid=0 2024-09-10T10:54:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 client.15909 shutdown 2024-09-10T10:54:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 -- 172.21.15.7:0/3592637466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c2410 msgr2=0x563fa72c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c2410 0x563fa72c70c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f08a8000c00 tx=0x7f08a8013830 comp rx=0 tx=0).stop 2024-09-10T10:54:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 -- 172.21.15.7:0/3592637466 shutdown_connections 2024-09-10T10:54:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563fa734c370 0x563fa734e790 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f0896ffd640 0 client.15909 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563fa72cccb0 0x563fa72c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fa72c2410 0x563fa72c70c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 --2- 172.21.15.7:0/3592637466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563fa72c1af0 0x563fa72c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 -- 172.21.15.7:0/3592637466 >> 172.21.15.7:0/3592637466 conn(0x563fa72be060 msgr2=0x563fa72bc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 -- 172.21.15.7:0/3592637466 shutdown_connections 2024-09-10T10:54:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 1 -- 172.21.15.7:0/3592637466 wait complete. 2024-09-10T10:54:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.665+0000 7f08c0dc4740 20 client.15909 trim_cache size 0 max 16384 2024-09-10T10:54:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:54:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:54:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:54:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:54:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 -- 172.21.15.7:0/1487519106 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08a00a2e20 msgr2=0x7f08a00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08a00a2e20 0x7f08a00a52e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08b009bf70 tx=0x7f08b00c6b70 comp rx=0 tx=0).stop 2024-09-10T10:54:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 -- 172.21.15.7:0/1487519106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8106a00 msgr2=0x7f08b819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8106a00 0x7f08b819ccd0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f08b000cce0 tx=0x7f08b000a620 comp rx=0 tx=0).stop 2024-09-10T10:54:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 -- 172.21.15.7:0/1487519106 shutdown_connections 2024-09-10T10:54:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f08a00a2e20 0x7f08a00a52e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b8106a00 0x7f08b819ccd0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f08b8101390 0x7f08b8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 --2- 172.21.15.7:0/1487519106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08b81009f0 0x7f08b8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 -- 172.21.15.7:0/1487519106 >> 172.21.15.7:0/1487519106 conn(0x7f08b80fc6b0 msgr2=0x7f08b806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 -- 172.21.15.7:0/1487519106 shutdown_connections 2024-09-10T10:54:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:06.666+0000 7f08bdf4b640 1 -- 172.21.15.7:0/1487519106 wait complete. 2024-09-10T10:54:07.225 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:54:06 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: ignoring --setuser ceph since I am not root 2024-09-10T10:54:07.225 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:54:06 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: ignoring --setgroup ceph since I am not root 2024-09-10T10:54:07.225 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:54:06 smithi133 ceph-mds[65473]: -- 172.21.15.133:0/3845731808 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633f483f860 con 0x5633f481c800 2024-09-10T10:54:07.225 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 10:54:06 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: starting mds.f at 2024-09-10T10:54:07.522 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.21 MB/sec execute 307 sec latency 5423.531 ms 2024-09-10T10:54:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:07 smithi103 ceph-mon[35160]: pgmap v256: 97 pgs: 97 active+clean; 251 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-10T10:54:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:07 smithi103 ceph-mon[35160]: daemon mds.f restarted 2024-09-10T10:54:07.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:07 smithi007 ceph-mon[30873]: pgmap v256: 97 pgs: 97 active+clean; 251 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-10T10:54:07.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:07 smithi007 ceph-mon[30873]: daemon mds.f restarted 2024-09-10T10:54:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:07 smithi133 ceph-mon[35282]: pgmap v256: 97 pgs: 97 active+clean; 251 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-10T10:54:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:07 smithi133 ceph-mon[35282]: daemon mds.f restarted 2024-09-10T10:54:08.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.17 MB/sec execute 308 sec latency 6423.620 ms 2024-09-10T10:54:08.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:08 smithi103 ceph-mon[35160]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T10:54:08.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:08 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6834/3897006549,v1:172.21.15.133:6835/3897006549] up:boot 2024-09-10T10:54:08.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:08 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T10:54:08.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:08 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:54:08.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:08 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:08.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:08 smithi103 ceph-mon[35160]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T10:54:08.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:08 smithi007 ceph-mon[30873]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T10:54:08.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:08 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6834/3897006549,v1:172.21.15.133:6835/3897006549] up:boot 2024-09-10T10:54:08.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:08 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T10:54:08.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:08 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:54:08.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:08 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:08.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:08 smithi007 ceph-mon[30873]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T10:54:08.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:08 smithi133 ceph-mon[35282]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T10:54:08.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:08 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6834/3897006549,v1:172.21.15.133:6835/3897006549] up:boot 2024-09-10T10:54:08.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:08 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T10:54:08.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:08 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:54:08.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:08 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:08.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:08 smithi133 ceph-mon[35282]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T10:54:09.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.13 MB/sec execute 309 sec latency 7423.749 ms 2024-09-10T10:54:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:09 smithi103 ceph-mon[35160]: pgmap v258: 97 pgs: 97 active+clean; 251 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-09-10T10:54:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:09 smithi007 ceph-mon[30873]: pgmap v258: 97 pgs: 97 active+clean; 251 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-09-10T10:54:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:09 smithi133 ceph-mon[35282]: pgmap v258: 97 pgs: 97 active+clean; 251 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-09-10T10:54:10.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.10 MB/sec execute 310 sec latency 8423.888 ms 2024-09-10T10:54:11.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.06 MB/sec execute 311 sec latency 9423.964 ms 2024-09-10T10:54:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:11 smithi103 ceph-mon[35160]: pgmap v260: 97 pgs: 97 active+clean; 238 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 3.9 MiB/s wr, 334 op/s 2024-09-10T10:54:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:11 smithi007 ceph-mon[30873]: pgmap v260: 97 pgs: 97 active+clean; 238 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 3.9 MiB/s wr, 334 op/s 2024-09-10T10:54:11.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:11 smithi133 ceph-mon[35282]: pgmap v260: 97 pgs: 97 active+clean; 238 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 3.9 MiB/s wr, 334 op/s 2024-09-10T10:54:12.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 11.03 MB/sec execute 312 sec latency 10424.099 ms 2024-09-10T10:54:13.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 10.99 MB/sec execute 313 sec latency 11424.194 ms 2024-09-10T10:54:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:13 smithi103 ceph-mon[35160]: pgmap v261: 97 pgs: 97 active+clean; 238 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-09-10T10:54:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:13 smithi007 ceph-mon[30873]: pgmap v261: 97 pgs: 97 active+clean; 238 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-09-10T10:54:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:13 smithi133 ceph-mon[35282]: pgmap v261: 97 pgs: 97 active+clean; 238 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-09-10T10:54:14.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 10.96 MB/sec execute 314 sec latency 12424.310 ms 2024-09-10T10:54:15.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1 920853 10.92 MB/sec execute 315 sec latency 13424.400 ms 2024-09-10T10:54:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:15 smithi103 ceph-mon[35160]: pgmap v262: 97 pgs: 97 active+clean; 238 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T10:54:15.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:15 smithi007 ceph-mon[30873]: pgmap v262: 97 pgs: 97 active+clean; 238 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T10:54:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:15 smithi133 ceph-mon[35282]: pgmap v262: 97 pgs: 97 active+clean; 238 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T10:54:16.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1 921431 10.89 MB/sec execute 316 sec latency 14163.096 ms 2024-09-10T10:54:17.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1 924000 10.91 MB/sec execute 317 sec latency 20.385 ms 2024-09-10T10:54:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:17 smithi103 ceph-mon[35160]: pgmap v263: 97 pgs: 97 active+clean; 237 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-09-10T10:54:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:17 smithi007 ceph-mon[30873]: pgmap v263: 97 pgs: 97 active+clean; 237 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-09-10T10:54:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:17 smithi133 ceph-mon[35282]: pgmap v263: 97 pgs: 97 active+clean; 237 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-09-10T10:54:18.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 925770 10.90 MB/sec execute 318 sec latency 15.162 ms 2024-09-10T10:54:19.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 927761 10.90 MB/sec execute 319 sec latency 10.643 ms 2024-09-10T10:54:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.759+0000 7f74f23d8640 1 -- 172.21.15.7:0/334674354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ec101ad0 msgr2=0x7f74ec101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.759+0000 7f74f23d8640 1 --2- 172.21.15.7:0/334674354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ec101ad0 0x7f74ec101f30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009930 tx=0x7f74e002a420 comp rx=0 tx=0).stop 2024-09-10T10:54:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.759+0000 7f74f23d8640 1 -- 172.21.15.7:0/334674354 shutdown_connections 2024-09-10T10:54:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.759+0000 7f74f23d8640 1 --2- 172.21.15.7:0/334674354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74ec102470 0x7f74ec10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.759+0000 7f74f23d8640 1 --2- 172.21.15.7:0/334674354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ec101ad0 0x7f74ec101f30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.759+0000 7f74f23d8640 1 --2- 172.21.15.7:0/334674354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec107b70 0x7f74ec107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.759+0000 7f74f23d8640 1 -- 172.21.15.7:0/334674354 >> 172.21.15.7:0/334674354 conn(0x7f74ec0fd7d0 msgr2=0x7f74ec0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.759+0000 7f74f23d8640 1 -- 172.21.15.7:0/334674354 shutdown_connections 2024-09-10T10:54:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.759+0000 7f74f23d8640 1 -- 172.21.15.7:0/334674354 wait complete. 2024-09-10T10:54:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74f23d8640 1 Processor -- start 2024-09-10T10:54:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74f23d8640 1 -- start start 2024-09-10T10:54:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74f23d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ec101ad0 0x7f74ec198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74f23d8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec102470 0x7f74ec198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74f23d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74ec107b70 0x7f74ec19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74f23d8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec111f20 con 0x7f74ec101ad0 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74f23d8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec111da0 con 0x7f74ec102470 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74f23d8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1120a0 con 0x7f74ec107b70 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ec101ad0 0x7f74ec198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ec101ad0 0x7f74ec198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44062/0 (socket says 172.21.15.7:44062) 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74ebfff640 1 -- 172.21.15.7:0/3901288333 learned_addr learned my addr 172.21.15.7:0/3901288333 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74eb7fe640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec102470 0x7f74ec198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.760+0000 7f74f0975640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74ec107b70 0x7f74ec19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74ebfff640 1 -- 172.21.15.7:0/3901288333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74ec107b70 msgr2=0x7f74ec19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74ebfff640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74ec107b70 0x7f74ec19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74ebfff640 1 -- 172.21.15.7:0/3901288333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec102470 msgr2=0x7f74ec198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74ebfff640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec102470 0x7f74ec198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74ebfff640 1 -- 172.21.15.7:0/3901288333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ec19d320 con 0x7f74ec101ad0 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74eb7fe640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec102470 0x7f74ec198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74ebfff640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ec101ad0 0x7f74ec198330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00e9f0 tx=0x7f74dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74e97fa640 1 -- 172.21.15.7:0/3901288333 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc00cd80 con 0x7f74ec101ad0 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74f23d8640 1 -- 172.21.15.7:0/3901288333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ec19d610 con 0x7f74ec101ad0 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74e97fa640 1 -- 172.21.15.7:0/3901288333 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74dc010910 con 0x7f74ec101ad0 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74e97fa640 1 -- 172.21.15.7:0/3901288333 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0188f0 con 0x7f74ec101ad0 2024-09-10T10:54:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.761+0000 7f74f23d8640 1 -- 172.21.15.7:0/3901288333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ec1a4ea0 con 0x7f74ec101ad0 2024-09-10T10:54:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74e97fa640 1 -- 172.21.15.7:0/3901288333 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f74dc010ac0 con 0x7f74ec101ad0 2024-09-10T10:54:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74f5249740 1 Processor -- start 2024-09-10T10:54:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74f5249740 1 -- start start 2024-09-10T10:54:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74f5249740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bbeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74f5249740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74f5249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bc9cb0 0x561f00bcc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74f5249740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f00b232f0 con 0x561f00bbf410 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74f5249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f00acb270 con 0x561f00bc9cb0 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74f5249740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561f00b1f2b0 con 0x561f00bbeaf0 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bc9cb0 0x561f00bcc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74f0975640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bbeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44070/0 (socket says 172.21.15.7:44070) 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74ebfff640 1 -- 172.21.15.7:0/3560355438 learned_addr learned my addr 172.21.15.7:0/3560355438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74ebfff640 1 -- 172.21.15.7:0/3560355438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 msgr2=0x561f00bbeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74ebfff640 1 --2- 172.21.15.7:0/3560355438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bbeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74ebfff640 1 -- 172.21.15.7:0/3560355438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bc9cb0 msgr2=0x561f00bcc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74ebfff640 1 --2- 172.21.15.7:0/3560355438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bc9cb0 0x561f00bcc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74ebfff640 1 -- 172.21.15.7:0/3560355438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f00aca340 con 0x561f00bbf410 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74e97fa640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f74d40a2da0 0x7f74d40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.764+0000 7f74eb7fe640 1 --2- 172.21.15.7:0/3560355438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bc9cb0 0x561f00bcc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.765+0000 7f74f0975640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f74d40a2da0 0x7f74d40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.765+0000 7f74e97fa640 1 -- 172.21.15.7:0/3901288333 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7f74dc0cf7c0 con 0x7f74ec101ad0 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.765+0000 7f74ebfff640 1 --2- 172.21.15.7:0/3560355438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc9770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0cce50 tx=0x7f74dc0d7690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b4e0b4c348a7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.765+0000 7f74d27fc640 1 -- 172.21.15.7:0/3560355438 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc09bb30 con 0x561f00bbf410 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.765+0000 7f74d27fc640 1 -- 172.21.15.7:0/3560355438 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74dc0a4920 con 0x561f00bbf410 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.765+0000 7f74d27fc640 1 -- 172.21.15.7:0/3560355438 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74dc09b6c0 con 0x561f00bbf410 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 -- 172.21.15.7:0/3560355438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 msgr2=0x561f00bc9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f0975640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f74d40a2da0 0x7f74d40a5260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74e4005fd0 tx=0x7f74e4009500 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 --2- 172.21.15.7:0/3560355438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc9770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0cce50 tx=0x7f74dc0d7690 comp rx=0 tx=0).stop 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 -- 172.21.15.7:0/3560355438 shutdown_connections 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 --2- 172.21.15.7:0/3560355438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bc9cb0 0x561f00bcc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 --2- 172.21.15.7:0/3560355438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc9770 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 --2- 172.21.15.7:0/3560355438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bbeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 -- 172.21.15.7:0/3560355438 >> 172.21.15.7:0/3560355438 conn(0x561f00bbb060 msgr2=0x561f00bbb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 -- 172.21.15.7:0/3560355438 shutdown_connections 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 -- 172.21.15.7:0/3560355438 wait complete. 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 -- start start 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.766+0000 7f74f5249740 1 Processor -- start 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f5249740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bc6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f5249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 0x561f00bc7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f5249740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bc7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f5249740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f00bcc880 con 0x561f00bc9cb0 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f5249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561f00aac290 con 0x561f00bbf410 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f5249740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561f00bcca00 con 0x561f00bbeaf0 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f5249740 10 client.0 _ll_register_callbacks cb 0x561f00add140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f0975640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bc7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f0975640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bc7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44080/0 (socket says 172.21.15.7:44080) 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f0975640 1 -- 172.21.15.7:0/3322522499 learned_addr learned my addr 172.21.15.7:0/3322522499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74ebfff640 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bc6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74eb7fe640 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 0x561f00bc7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f0975640 1 -- 172.21.15.7:0/3322522499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 msgr2=0x561f00bc6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f0975640 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bc6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f0975640 1 -- 172.21.15.7:0/3322522499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 msgr2=0x561f00bc7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f0975640 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 0x561f00bc7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f0975640 1 -- 172.21.15.7:0/3322522499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f00836210 con 0x561f00bc9cb0 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f0975640 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bc7640 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f74e4003fb0 tx=0x7f74e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6653d4dbfa29cccd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74cb7fe640 1 -- 172.21.15.7:0/3322522499 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e400f8f0 con 0x561f00bc9cb0 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74f5249740 1 -- 172.21.15.7:0/3322522499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74dc0a4920 con 0x561f00bc9cb0 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.767+0000 7f74cb7fe640 1 -- 172.21.15.7:0/3322522499 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7f74e400fa90 con 0x561f00bc9cb0 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74cb7fe640 1 -- 172.21.15.7:0/3322522499 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f74e400fc90 con 0x561f00bc9cb0 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74f5249740 10 client.15927 fetch_fsmap learned FSMap version 37 2024-09-10T10:54:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74f5249740 1 -- 172.21.15.7:0/3322522499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561f00902620 con 0x561f00bc9cb0 2024-09-10T10:54:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74cb7fe640 1 -- 172.21.15.7:0/3322522499 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74dc0a4920 con 0x561f00bc9cb0 2024-09-10T10:54:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74cb7fe640 1 -- 172.21.15.7:0/3322522499 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f74e4026070 con 0x561f00bc9cb0 2024-09-10T10:54:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74f5249740 10 client.15927 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T10:54:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74f5249740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:54:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74f5249740 20 client.15927 populate_metadata read hostname 'smithi007' 2024-09-10T10:54:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74f5249740 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561f00bd9c80 0x561f00bdc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74f5249740 4 client.15927 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:54:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74f5249740 1 -- 172.21.15.7:0/3322522499 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f00b30a20 con 0x561f00bd9c80 2024-09-10T10:54:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.768+0000 7f74ebfff640 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561f00bd9c80 0x561f00bdc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.769+0000 7f74ebfff640 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561f00bd9c80 0x561f00bdc0b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d386aee3108089a server_cookie=f09df91264223fcc in_seq=0 out_seq=0 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.769+0000 7f74cb7fe640 10 client.15927 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.769+0000 7f74cb7fe640 1 -- 172.21.15.7:0/3322522499 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f00b30a20 con 0x561f00bd9c80 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.769+0000 7f74cb7fe640 10 client.15927 handle_command_reply: tid=0 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.769+0000 7f74f5249740 1 client.15927 shutdown 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 1 -- 172.21.15.7:0/3322522499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 msgr2=0x561f00bc7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bc7640 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f74e4003fb0 tx=0x7f74e4004ab0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 1 -- 172.21.15.7:0/3322522499 shutdown_connections 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561f00bd9c80 0x561f00bdc0b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bc7640 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74cb7fe640 0 client.15927 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 0x561f00bc7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 1 --2- 172.21.15.7:0/3322522499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bc6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 1 -- 172.21.15.7:0/3322522499 >> 172.21.15.7:0/3322522499 conn(0x561f00bbb060 msgr2=0x561f00bc0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 1 -- 172.21.15.7:0/3322522499 shutdown_connections 2024-09-10T10:54:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 1 -- 172.21.15.7:0/3322522499 wait complete. 2024-09-10T10:54:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.770+0000 7f74f5249740 20 client.15927 trim_cache size 0 max 16384 2024-09-10T10:54:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.785+0000 7f74f5249740 1 Processor -- start 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.785+0000 7f74f5249740 1 -- start start 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74f5249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbeaf0 0x561f00bc5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74f5249740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74f5249740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bc9cb0 0x561f00bc22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74f5249740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1b0180 con 0x561f00bbf410 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74f5249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1ada20 con 0x561f00bbeaf0 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74f5249740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1ab890 con 0x561f00bc9cb0 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74eb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbeaf0 0x561f00bc5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74eb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44088/0 (socket says 172.21.15.7:44088) 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74eb7fe640 1 -- 172.21.15.7:0/718807176 learned_addr learned my addr 172.21.15.7:0/718807176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74ebfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbeaf0 0x561f00bc5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:33456/0 (socket says 172.21.15.7:33456) 2024-09-10T10:54:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74f0975640 1 --2- 172.21.15.7:0/718807176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bc9cb0 0x561f00bc22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74eb7fe640 1 -- 172.21.15.7:0/718807176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bc9cb0 msgr2=0x561f00bc22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74eb7fe640 1 --2- 172.21.15.7:0/718807176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bc9cb0 0x561f00bc22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74eb7fe640 1 -- 172.21.15.7:0/718807176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbeaf0 msgr2=0x561f00bc5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74eb7fe640 1 --2- 172.21.15.7:0/718807176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbeaf0 0x561f00bc5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74eb7fe640 1 -- 172.21.15.7:0/718807176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f00bd8870 con 0x561f00bbf410 2024-09-10T10:54:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.786+0000 7f74ebfff640 1 --2- 172.21.15.7:0/718807176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbeaf0 0x561f00bc5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.787+0000 7f74eb7fe640 1 --2- 172.21.15.7:0/718807176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc1560 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f74e002a940 tx=0x7f74e00315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b0b7c90922aaa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.787+0000 7f74d1ffb640 1 -- 172.21.15.7:0/718807176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0035040 con 0x561f00bbf410 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.787+0000 7f74d1ffb640 1 -- 172.21.15.7:0/718807176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e0004030 con 0x561f00bbf410 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.787+0000 7f74d1ffb640 1 -- 172.21.15.7:0/718807176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e00047a0 con 0x561f00bbf410 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.788+0000 7f74f5249740 1 -- 172.21.15.7:0/718807176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 msgr2=0x561f00bc1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.788+0000 7f74f5249740 1 --2- 172.21.15.7:0/718807176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc1560 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f74e002a940 tx=0x7f74e00315b0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.788+0000 7f74f5249740 1 -- 172.21.15.7:0/718807176 shutdown_connections 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.788+0000 7f74f5249740 1 --2- 172.21.15.7:0/718807176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bc9cb0 0x561f00bc22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.788+0000 7f74f5249740 1 --2- 172.21.15.7:0/718807176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bbf410 0x561f00bc1560 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.788+0000 7f74f5249740 1 --2- 172.21.15.7:0/718807176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbeaf0 0x561f00bc5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.788+0000 7f74f5249740 1 -- 172.21.15.7:0/718807176 >> 172.21.15.7:0/718807176 conn(0x561f00bbb060 msgr2=0x561f00bc8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.788+0000 7f74f5249740 1 -- 172.21.15.7:0/718807176 shutdown_connections 2024-09-10T10:54:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.788+0000 7f74f5249740 1 -- 172.21.15.7:0/718807176 wait complete. 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.788+0000 7f74f5249740 1 -- start start 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74f5249740 1 Processor -- start 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74f5249740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bd2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74f5249740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 0x561f00bd2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74f5249740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bcd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74f5249740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1abee0 con 0x561f00bc9cb0 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74f5249740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1ad350 con 0x561f00bbf410 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74f5249740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f74ec1ac5b0 con 0x561f00bbeaf0 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74f5249740 10 client.0 _ll_register_callbacks cb 0x7f74e4009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 0x561f00bd2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74ebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bd2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74f0975640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bcd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74ebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bd2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:37048/0 (socket says 172.21.15.7:37048) 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74ebfff640 1 -- 172.21.15.7:0/2280577460 learned_addr learned my addr 172.21.15.7:0/2280577460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.789+0000 7f74eb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 0x561f00bd2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:33458/0 (socket says 172.21.15.7:33458) 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74f0975640 1 -- 172.21.15.7:0/2280577460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 msgr2=0x561f00bd2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74f0975640 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bd2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74f0975640 1 -- 172.21.15.7:0/2280577460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 msgr2=0x561f00bd2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74f0975640 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 0x561f00bd2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74f0975640 1 -- 172.21.15.7:0/2280577460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f00bcdd80 con 0x561f00bc9cb0 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74eb7fe640 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 0x561f00bd2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74f0975640 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bcd550 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f74e40285d0 tx=0x7f74e400fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3c1263a6c84496 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74cb7fe640 1 -- 172.21.15.7:0/2280577460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4028a50 con 0x561f00bc9cb0 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74cb7fe640 1 -- 172.21.15.7:0/2280577460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7f74e4004b20 con 0x561f00bc9cb0 2024-09-10T10:54:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74f5249740 1 -- 172.21.15.7:0/2280577460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74e0004030 con 0x561f00bc9cb0 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74cb7fe640 1 -- 172.21.15.7:0/2280577460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f74e403f050 con 0x561f00bc9cb0 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74f5249740 10 client.15933 fetch_fsmap learned FSMap version 37 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74f5249740 1 -- 172.21.15.7:0/2280577460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561f00c44650 con 0x561f00bc9cb0 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.790+0000 7f74cb7fe640 1 -- 172.21.15.7:0/2280577460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e0004030 con 0x561f00bc9cb0 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.791+0000 7f74cb7fe640 1 -- 172.21.15.7:0/2280577460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f74e4037020 con 0x561f00bc9cb0 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.791+0000 7f74f5249740 10 client.15933 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.791+0000 7f74f5249740 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.791+0000 7f74f5249740 20 client.15933 populate_metadata read hostname 'smithi007' 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.791+0000 7f74f5249740 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561f00c44dc0 0x561f00c471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.791+0000 7f74f5249740 4 client.15933 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.791+0000 7f74f5249740 1 -- 172.21.15.7:0/2280577460 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f00b2ad70 con 0x561f00c44dc0 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.791+0000 7f74ebfff640 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561f00c44dc0 0x561f00c471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.791+0000 7f74ebfff640 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561f00c44dc0 0x561f00c471e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67aaaa44cc77f7e1 server_cookie=df3c06eabee7b667 in_seq=0 out_seq=0 2024-09-10T10:54:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.791+0000 7f74cb7fe640 10 client.15933 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.883+0000 7f74cb7fe640 1 -- 172.21.15.7:0/2280577460 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f00b2ad70 con 0x561f00c44dc0 2024-09-10T10:54:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.883+0000 7f74cb7fe640 10 client.15933 handle_command_reply: tid=0 2024-09-10T10:54:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.883+0000 7f74f5249740 1 client.15933 shutdown 2024-09-10T10:54:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 1 -- 172.21.15.7:0/2280577460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 msgr2=0x561f00bcd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bcd550 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f74e40285d0 tx=0x7f74e400fb20 comp rx=0 tx=0).stop 2024-09-10T10:54:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 1 -- 172.21.15.7:0/2280577460 shutdown_connections 2024-09-10T10:54:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561f00c44dc0 0x561f00c471e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f00bc9cb0 0x561f00bcd550 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561f00bbf410 0x561f00bd2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74cb7fe640 0 client.15933 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 1 --2- 172.21.15.7:0/2280577460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f00bbeaf0 0x561f00bd2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 1 -- 172.21.15.7:0/2280577460 >> 172.21.15.7:0/2280577460 conn(0x561f00bbb060 msgr2=0x561f00bb99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 1 -- 172.21.15.7:0/2280577460 shutdown_connections 2024-09-10T10:54:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 1 -- 172.21.15.7:0/2280577460 wait complete. 2024-09-10T10:54:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f5249740 20 client.15933 trim_cache size 0 max 16384 2024-09-10T10:54:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:54:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:54:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:54:19.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:54:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f23d8640 1 -- 172.21.15.7:0/3901288333 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f74d40a2da0 msgr2=0x7f74d40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f23d8640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f74d40a2da0 0x7f74d40a5260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74e4005fd0 tx=0x7f74e4009500 comp rx=0 tx=0).stop 2024-09-10T10:54:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f23d8640 1 -- 172.21.15.7:0/3901288333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ec101ad0 msgr2=0x7f74ec198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.884+0000 7f74f23d8640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ec101ad0 0x7f74ec198330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00e9f0 tx=0x7f74dc00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.885+0000 7f74f23d8640 1 -- 172.21.15.7:0/3901288333 shutdown_connections 2024-09-10T10:54:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.885+0000 7f74f23d8640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74ec107b70 0x7f74ec19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.885+0000 7f74f23d8640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74ec102470 0x7f74ec198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.885+0000 7f74f23d8640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f74d40a2da0 0x7f74d40a5260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.885+0000 7f74f23d8640 1 --2- 172.21.15.7:0/3901288333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74ec101ad0 0x7f74ec198330 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.885+0000 7f74f23d8640 1 -- 172.21.15.7:0/3901288333 >> 172.21.15.7:0/3901288333 conn(0x7f74ec0fd7d0 msgr2=0x7f74ec0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.885+0000 7f74f23d8640 1 -- 172.21.15.7:0/3901288333 shutdown_connections 2024-09-10T10:54:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:19.885+0000 7f74f23d8640 1 -- 172.21.15.7:0/3901288333 wait complete. 2024-09-10T10:54:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:19 smithi103 ceph-mon[35160]: pgmap v264: 97 pgs: 97 active+clean; 237 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-09-10T10:54:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:19 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:19 smithi007 ceph-mon[30873]: pgmap v264: 97 pgs: 97 active+clean; 237 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-09-10T10:54:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:19 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:19 smithi133 ceph-mon[35282]: pgmap v264: 97 pgs: 97 active+clean; 237 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-09-10T10:54:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:19 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:20.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 930103 10.90 MB/sec execute 320 sec latency 34.772 ms 2024-09-10T10:54:21.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 931440 10.90 MB/sec execute 321 sec latency 53.765 ms 2024-09-10T10:54:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:21 smithi103 ceph-mon[35160]: pgmap v265: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.9 MiB/s wr, 211 op/s 2024-09-10T10:54:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:21 smithi007 ceph-mon[30873]: pgmap v265: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.9 MiB/s wr, 211 op/s 2024-09-10T10:54:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:21 smithi133 ceph-mon[35282]: pgmap v265: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.9 MiB/s wr, 211 op/s 2024-09-10T10:54:22.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 933885 10.90 MB/sec execute 322 sec latency 15.063 ms 2024-09-10T10:54:23.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 935344 10.89 MB/sec execute 323 sec latency 12.706 ms 2024-09-10T10:54:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:23 smithi103 ceph-mon[35160]: pgmap v266: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-09-10T10:54:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:23 smithi007 ceph-mon[30873]: pgmap v266: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-09-10T10:54:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:23 smithi133 ceph-mon[35282]: pgmap v266: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-09-10T10:54:24.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 938154 10.91 MB/sec execute 324 sec latency 15.497 ms 2024-09-10T10:54:25.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 939886 10.90 MB/sec execute 325 sec latency 12.654 ms 2024-09-10T10:54:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:25 smithi103 ceph-mon[35160]: pgmap v267: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-09-10T10:54:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:25 smithi007 ceph-mon[30873]: pgmap v267: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-09-10T10:54:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:25 smithi133 ceph-mon[35282]: pgmap v267: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-09-10T10:54:26.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 941923 10.91 MB/sec execute 326 sec latency 46.398 ms 2024-09-10T10:54:27.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 944455 10.91 MB/sec execute 327 sec latency 13.894 ms 2024-09-10T10:54:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:27 smithi103 ceph-mon[35160]: pgmap v268: 97 pgs: 97 active+clean; 246 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 258 op/s 2024-09-10T10:54:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:27 smithi007 ceph-mon[30873]: pgmap v268: 97 pgs: 97 active+clean; 246 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 258 op/s 2024-09-10T10:54:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:27 smithi133 ceph-mon[35282]: pgmap v268: 97 pgs: 97 active+clean; 246 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 258 op/s 2024-09-10T10:54:28.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 945910 10.90 MB/sec execute 328 sec latency 22.896 ms 2024-09-10T10:54:29.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 948103 10.90 MB/sec execute 329 sec latency 221.128 ms 2024-09-10T10:54:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:29 smithi103 ceph-mon[35160]: pgmap v269: 97 pgs: 97 active+clean; 246 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 228 op/s 2024-09-10T10:54:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:29 smithi007 ceph-mon[30873]: pgmap v269: 97 pgs: 97 active+clean; 246 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 228 op/s 2024-09-10T10:54:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:29 smithi133 ceph-mon[35282]: pgmap v269: 97 pgs: 97 active+clean; 246 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 228 op/s 2024-09-10T10:54:30.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 949611 10.89 MB/sec execute 330 sec latency 307.353 ms 2024-09-10T10:54:31.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 952161 10.91 MB/sec execute 331 sec latency 74.458 ms 2024-09-10T10:54:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:31 smithi103 ceph-mon[35160]: pgmap v270: 97 pgs: 97 active+clean; 253 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 339 op/s 2024-09-10T10:54:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:31 smithi007 ceph-mon[30873]: pgmap v270: 97 pgs: 97 active+clean; 253 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 339 op/s 2024-09-10T10:54:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:31 smithi133 ceph-mon[35282]: pgmap v270: 97 pgs: 97 active+clean; 253 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 339 op/s 2024-09-10T10:54:32.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 953726 10.90 MB/sec execute 332 sec latency 17.021 ms 2024-09-10T10:54:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.978+0000 7f6c6d328640 1 -- 172.21.15.7:0/301278553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6810c0d0 msgr2=0x7f6c6810e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.978+0000 7f6c6d328640 1 --2- 172.21.15.7:0/301278553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6810c0d0 0x7f6c6810e550 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009ca0 tx=0x7f6c6002a950 comp rx=0 tx=0).stop 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.978+0000 7f6c6d328640 1 -- 172.21.15.7:0/301278553 shutdown_connections 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.978+0000 7f6c6d328640 1 --2- 172.21.15.7:0/301278553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c6810c0d0 0x7f6c6810e550 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.978+0000 7f6c6d328640 1 --2- 172.21.15.7:0/301278553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c681035a0 0x7f6c6810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.978+0000 7f6c6d328640 1 --2- 172.21.15.7:0/301278553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c68102c80 0x7f6c68103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.978+0000 7f6c6d328640 1 -- 172.21.15.7:0/301278553 >> 172.21.15.7:0/301278553 conn(0x7f6c68077990 msgr2=0x7f6c680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.978+0000 7f6c6d328640 1 -- 172.21.15.7:0/301278553 shutdown_connections 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.978+0000 7f6c6d328640 1 -- 172.21.15.7:0/301278553 wait complete. 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c6d328640 1 Processor -- start 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c6d328640 1 -- start start 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c6d328640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c68102c80 0x7f6c681982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c6d328640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c681035a0 0x7f6c68198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c6d328640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c6810c0d0 0x7f6c6819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c6d328640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68111eb0 con 0x7f6c681035a0 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c6d328640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68111d30 con 0x7f6c6810c0d0 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c6d328640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68112030 con 0x7f6c68102c80 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c667fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c681035a0 0x7f6c68198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c667fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c681035a0 0x7f6c68198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33498/0 (socket says 172.21.15.7:33498) 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c667fc640 1 -- 172.21.15.7:0/3385264151 learned_addr learned my addr 172.21.15.7:0/3385264151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c68102c80 0x7f6c681982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.979+0000 7f6c677fe640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c6810c0d0 0x7f6c6819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c667fc640 1 -- 172.21.15.7:0/3385264151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c68102c80 msgr2=0x7f6c681982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c667fc640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c68102c80 0x7f6c681982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c667fc640 1 -- 172.21.15.7:0/3385264151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c6810c0d0 msgr2=0x7f6c6819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c667fc640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c6810c0d0 0x7f6c6819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c667fc640 1 -- 172.21.15.7:0/3385264151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6819d2d0 con 0x7f6c681035a0 2024-09-10T10:54:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c667fc640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c681035a0 0x7f6c68198820 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00e9f0 tx=0x7f6c5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c4ffff640 1 -- 172.21.15.7:0/3385264151 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00ccc0 con 0x7f6c681035a0 2024-09-10T10:54:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c6d328640 1 -- 172.21.15.7:0/3385264151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6819d5c0 con 0x7f6c681035a0 2024-09-10T10:54:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c4ffff640 1 -- 172.21.15.7:0/3385264151 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c5c00ce60 con 0x7f6c681035a0 2024-09-10T10:54:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c6d328640 1 -- 172.21.15.7:0/3385264151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c681a4e50 con 0x7f6c681035a0 2024-09-10T10:54:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.980+0000 7f6c4ffff640 1 -- 172.21.15.7:0/3385264151 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c018630 con 0x7f6c681035a0 2024-09-10T10:54:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c4ffff640 1 -- 172.21.15.7:0/3385264151 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6c5c0046c0 con 0x7f6c681035a0 2024-09-10T10:54:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c700b6740 1 Processor -- start 2024-09-10T10:54:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c700b6740 1 -- start start 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 0x558e3f7efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f7feb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ff0c0 0x558e3f8014b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c700b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558e3f7542f0 con 0x558e3f7ef9b0 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c700b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e3f6fc270 con 0x558e3f7ff0c0 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c700b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558e3f7502b0 con 0x558e3f7f02d0 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c4ffff640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c500a2da0 0x7f6c500a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f7feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f7feb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:52142/0 (socket says 172.21.15.7:52142) 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c66ffd640 1 -- 172.21.15.7:0/489978866 learned_addr learned my addr 172.21.15.7:0/489978866 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c667fc640 1 --2- 172.21.15.7:0/489978866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ff0c0 0x558e3f8014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.983+0000 7f6c677fe640 1 --2- 172.21.15.7:0/489978866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 0x558e3f7efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c677fe640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c500a2da0 0x7f6c500a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c66ffd640 1 -- 172.21.15.7:0/489978866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ff0c0 msgr2=0x558e3f8014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/489978866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ff0c0 0x558e3f8014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c66ffd640 1 -- 172.21.15.7:0/489978866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 msgr2=0x558e3f7efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/489978866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 0x558e3f7efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c4ffff640 1 -- 172.21.15.7:0/3385264151 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7f6c5c0cf870 con 0x7f6c681035a0 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c66ffd640 1 -- 172.21.15.7:0/489978866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3f6fb340 con 0x558e3f7f02d0 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/489978866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f7feb80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800bd10 tx=0x7f6c580107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40f4ebfa9b8eaf23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c677fe640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c500a2da0 0x7f6c500a5260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009430 tx=0x7f6c60005ef0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c4d7fa640 1 -- 172.21.15.7:0/489978866 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58004030 con 0x558e3f7f02d0 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c4d7fa640 1 -- 172.21.15.7:0/489978866 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c580041d0 con 0x558e3f7f02d0 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.984+0000 7f6c4d7fa640 1 -- 172.21.15.7:0/489978866 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58011840 con 0x558e3f7f02d0 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.985+0000 7f6c700b6740 1 -- 172.21.15.7:0/489978866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 msgr2=0x558e3f7feb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.985+0000 7f6c700b6740 1 --2- 172.21.15.7:0/489978866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f7feb80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800bd10 tx=0x7f6c580107a0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.985+0000 7f6c700b6740 1 -- 172.21.15.7:0/489978866 shutdown_connections 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.985+0000 7f6c700b6740 1 --2- 172.21.15.7:0/489978866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ff0c0 0x558e3f8014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.985+0000 7f6c700b6740 1 --2- 172.21.15.7:0/489978866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f7feb80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.985+0000 7f6c700b6740 1 --2- 172.21.15.7:0/489978866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 0x558e3f7efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.985+0000 7f6c700b6740 1 -- 172.21.15.7:0/489978866 >> 172.21.15.7:0/489978866 conn(0x558e3f7ec060 msgr2=0x558e3f7ec440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.985+0000 7f6c700b6740 1 -- 172.21.15.7:0/489978866 shutdown_connections 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.985+0000 7f6c700b6740 1 -- 172.21.15.7:0/489978866 wait complete. 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.985+0000 7f6c700b6740 1 -- start start 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c700b6740 1 Processor -- start 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 0x558e3f7fc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f7fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ff0c0 0x558e3f7fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c700b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558e3f801c90 con 0x558e3f7ff0c0 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c700b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558e3f6dd290 con 0x558e3f7ef9b0 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c700b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558e3f801e10 con 0x558e3f7f02d0 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c700b6740 10 client.0 _ll_register_callbacks cb 0x558e3f70e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c677fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ff0c0 0x558e3f7fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c677fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ff0c0 0x558e3f7fcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33508/0 (socket says 172.21.15.7:33508) 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c667fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f7fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c677fe640 1 -- 172.21.15.7:0/814014595 learned_addr learned my addr 172.21.15.7:0/814014595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 0x558e3f7fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c677fe640 1 -- 172.21.15.7:0/814014595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 msgr2=0x558e3f7fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c677fe640 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f7fc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c677fe640 1 -- 172.21.15.7:0/814014595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 msgr2=0x558e3f7fc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c677fe640 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 0x558e3f7fc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c677fe640 1 -- 172.21.15.7:0/814014595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e3f467210 con 0x558e3f7ff0c0 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c677fe640 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ff0c0 0x558e3f7fcae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009a60 tx=0x7f6c60009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c94286fd385c1d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.986+0000 7f6c4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c4aff5640 1 -- 172.21.15.7:0/814014595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6004e030 con 0x558e3f7ff0c0 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c4aff5640 1 -- 172.21.15.7:0/814014595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7f6c60004410 con 0x558e3f7ff0c0 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c700b6740 1 -- 172.21.15.7:0/814014595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c580041d0 con 0x558e3f7ff0c0 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c4aff5640 1 -- 172.21.15.7:0/814014595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6c60034040 con 0x558e3f7ff0c0 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c700b6740 10 client.15945 fetch_fsmap learned FSMap version 37 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c700b6740 1 -- 172.21.15.7:0/814014595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558e3f533620 con 0x558e3f7ff0c0 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c4aff5640 1 -- 172.21.15.7:0/814014595 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c580041d0 con 0x558e3f7ff0c0 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c4aff5640 1 -- 172.21.15.7:0/814014595 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f6c6004aae0 con 0x558e3f7ff0c0 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c700b6740 10 client.15945 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T10:54:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c700b6740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:54:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c700b6740 20 client.15945 populate_metadata read hostname 'smithi007' 2024-09-10T10:54:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c700b6740 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e3f80f070 0x558e3f8114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c700b6740 4 client.15945 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:54:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c700b6740 1 -- 172.21.15.7:0/814014595 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e3f761a20 con 0x558e3f80f070 2024-09-10T10:54:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.987+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e3f80f070 0x558e3f8114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.988+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e3f80f070 0x558e3f8114a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1acde6171c74230 server_cookie=ecc87be3095cbda2 in_seq=0 out_seq=0 2024-09-10T10:54:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.988+0000 7f6c4aff5640 10 client.15945 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.988+0000 7f6c4aff5640 1 -- 172.21.15.7:0/814014595 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e3f761a20 con 0x558e3f80f070 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.988+0000 7f6c4aff5640 10 client.15945 handle_command_reply: tid=0 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 client.15945 shutdown 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 -- 172.21.15.7:0/814014595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ff0c0 msgr2=0x558e3f7fcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ff0c0 0x558e3f7fcae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009a60 tx=0x7f6c60009360 comp rx=0 tx=0).stop 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 -- 172.21.15.7:0/814014595 shutdown_connections 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e3f80f070 0x558e3f8114a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ff0c0 0x558e3f7fcae0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f7fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c4aff5640 0 client.15945 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 --2- 172.21.15.7:0/814014595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 0x558e3f7fc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 -- 172.21.15.7:0/814014595 >> 172.21.15.7:0/814014595 conn(0x558e3f7ec060 msgr2=0x558e3f7f1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 -- 172.21.15.7:0/814014595 shutdown_connections 2024-09-10T10:54:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 1 -- 172.21.15.7:0/814014595 wait complete. 2024-09-10T10:54:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:32.989+0000 7f6c700b6740 20 client.15945 trim_cache size 0 max 16384 2024-09-10T10:54:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.003+0000 7f6c700b6740 1 Processor -- start 2024-09-10T10:54:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.003+0000 7f6c700b6740 1 -- start start 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 0x558e3f7fc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f816e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ff0c0 0x558e3f817340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c700b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681adf10 con 0x558e3f7ef9b0 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c700b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681ab7b0 con 0x558e3f7ff0c0 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c700b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681a9620 con 0x558e3f7f02d0 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 0x558e3f7fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 0x558e3f7fc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33522/0 (socket says 172.21.15.7:33522) 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c66ffd640 1 -- 172.21.15.7:0/3228881895 learned_addr learned my addr 172.21.15.7:0/3228881895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c677fe640 1 --2- 172.21.15.7:0/3228881895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ff0c0 0x558e3f817340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c667fc640 1 --2- 172.21.15.7:0/3228881895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f816e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c66ffd640 1 -- 172.21.15.7:0/3228881895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 msgr2=0x558e3f816e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/3228881895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f816e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c66ffd640 1 -- 172.21.15.7:0/3228881895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ff0c0 msgr2=0x558e3f817340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/3228881895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ff0c0 0x558e3f817340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c66ffd640 1 -- 172.21.15.7:0/3228881895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3f7e62f0 con 0x558e3f7ef9b0 2024-09-10T10:54:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/3228881895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 0x558e3f7fc730 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6c58000c00 tx=0x7f6c58003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760ef30246ae4791 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.004+0000 7f6c4cff9640 1 -- 172.21.15.7:0/3228881895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58005240 con 0x558e3f7ef9b0 2024-09-10T10:54:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c4cff9640 1 -- 172.21.15.7:0/3228881895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c580053e0 con 0x558e3f7ef9b0 2024-09-10T10:54:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c4cff9640 1 -- 172.21.15.7:0/3228881895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c580116a0 con 0x558e3f7ef9b0 2024-09-10T10:54:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c700b6740 1 -- 172.21.15.7:0/3228881895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 msgr2=0x558e3f7fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c700b6740 1 --2- 172.21.15.7:0/3228881895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 0x558e3f7fc730 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6c58000c00 tx=0x7f6c58003f10 comp rx=0 tx=0).stop 2024-09-10T10:54:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c700b6740 1 -- 172.21.15.7:0/3228881895 shutdown_connections 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c700b6740 1 --2- 172.21.15.7:0/3228881895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ff0c0 0x558e3f817340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c700b6740 1 --2- 172.21.15.7:0/3228881895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7f02d0 0x558e3f816e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c700b6740 1 --2- 172.21.15.7:0/3228881895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7ef9b0 0x558e3f7fc730 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c700b6740 1 -- 172.21.15.7:0/3228881895 >> 172.21.15.7:0/3228881895 conn(0x558e3f7ec060 msgr2=0x558e3f7fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c700b6740 1 -- 172.21.15.7:0/3228881895 shutdown_connections 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.005+0000 7f6c700b6740 1 -- 172.21.15.7:0/3228881895 wait complete. 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c700b6740 1 -- start start 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c700b6740 1 Processor -- start 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 0x558e3f8079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7f02d0 0x558e3f7f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c700b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7ff0c0 0x558e3f7f9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c700b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681a9c70 con 0x558e3f7f02d0 2024-09-10T10:54:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c700b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681ab0e0 con 0x558e3f7ef9b0 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c700b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681aa340 con 0x558e3f7ff0c0 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c700b6740 10 client.0 _ll_register_callbacks cb 0x7f6c6002ff90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c667fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7f02d0 0x558e3f7f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c677fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7ff0c0 0x558e3f7f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 0x558e3f8079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c667fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7f02d0 0x558e3f7f8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33532/0 (socket says 172.21.15.7:33532) 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c667fc640 1 -- 172.21.15.7:0/1146585723 learned_addr learned my addr 172.21.15.7:0/1146585723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c667fc640 1 -- 172.21.15.7:0/1146585723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7ff0c0 msgr2=0x558e3f7f9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c667fc640 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7ff0c0 0x558e3f7f9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.006+0000 7f6c667fc640 1 -- 172.21.15.7:0/1146585723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 msgr2=0x558e3f8079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c667fc640 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 0x558e3f8079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c667fc640 1 -- 172.21.15.7:0/1146585723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e3f878dd0 con 0x558e3f7f02d0 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 0x558e3f8079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c667fc640 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7f02d0 0x558e3f7f8ed0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00c6a0 tx=0x7f6c5c0c3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70c6c8a4525087d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c4aff5640 1 -- 172.21.15.7:0/1146585723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c07f060 con 0x558e3f7f02d0 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c4aff5640 1 -- 172.21.15.7:0/1146585723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7f6c5c07ad80 con 0x558e3f7f02d0 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c700b6740 1 -- 172.21.15.7:0/1146585723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c580053e0 con 0x558e3f7f02d0 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c4aff5640 1 -- 172.21.15.7:0/1146585723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6c5c0a0110 con 0x558e3f7f02d0 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c700b6740 10 client.15951 fetch_fsmap learned FSMap version 37 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c700b6740 1 -- 172.21.15.7:0/1146585723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558e3f879f80 con 0x558e3f7f02d0 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c4aff5640 1 -- 172.21.15.7:0/1146585723 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c580053e0 con 0x558e3f7f02d0 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.007+0000 7f6c4aff5640 1 -- 172.21.15.7:0/1146585723 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 8350+0+0 (secure 0 0 0) 0x7f6c5c09ac20 con 0x558e3f7f02d0 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.008+0000 7f6c700b6740 10 client.15951 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.008+0000 7f6c700b6740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:54:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.008+0000 7f6c700b6740 20 client.15951 populate_metadata read hostname 'smithi007' 2024-09-10T10:54:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.008+0000 7f6c700b6740 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e3f87a700 0x558e3f87cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.008+0000 7f6c700b6740 4 client.15951 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:54:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.008+0000 7f6c700b6740 1 -- 172.21.15.7:0/1146585723 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e3f75bd70 con 0x558e3f87a700 2024-09-10T10:54:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.008+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e3f87a700 0x558e3f87cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.008+0000 7f6c66ffd640 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e3f87a700 0x558e3f87cb20 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe973374f75c51 server_cookie=5f4ee1616a23730b in_seq=0 out_seq=0 2024-09-10T10:54:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.008+0000 7f6c4aff5640 10 client.15951 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c4aff5640 1 -- 172.21.15.7:0/1146585723 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e3f75bd70 con 0x558e3f87a700 2024-09-10T10:54:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c4aff5640 10 client.15951 handle_command_reply: tid=0 2024-09-10T10:54:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 client.15951 shutdown 2024-09-10T10:54:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 -- 172.21.15.7:0/1146585723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7f02d0 msgr2=0x558e3f7f8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7f02d0 0x558e3f7f8ed0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00c6a0 tx=0x7f6c5c0c3040 comp rx=0 tx=0).stop 2024-09-10T10:54:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 -- 172.21.15.7:0/1146585723 shutdown_connections 2024-09-10T10:54:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558e3f87a700 0x558e3f87cb20 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e3f7ff0c0 0x558e3f7f9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c4aff5640 0 client.15951 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558e3f7f02d0 0x558e3f7f8ed0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 --2- 172.21.15.7:0/1146585723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558e3f7ef9b0 0x558e3f8079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 -- 172.21.15.7:0/1146585723 >> 172.21.15.7:0/1146585723 conn(0x558e3f7ec060 msgr2=0x558e3f7fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 -- 172.21.15.7:0/1146585723 shutdown_connections 2024-09-10T10:54:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 1 -- 172.21.15.7:0/1146585723 wait complete. 2024-09-10T10:54:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.113+0000 7f6c700b6740 20 client.15951 trim_cache size 0 max 16384 2024-09-10T10:54:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:54:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:54:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:54:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:54:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 -- 172.21.15.7:0/3385264151 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c500a2da0 msgr2=0x7f6c500a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c500a2da0 0x7f6c500a5260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009430 tx=0x7f6c60005ef0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 -- 172.21.15.7:0/3385264151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c681035a0 msgr2=0x7f6c68198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c681035a0 0x7f6c68198820 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00e9f0 tx=0x7f6c5c00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 -- 172.21.15.7:0/3385264151 shutdown_connections 2024-09-10T10:54:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c6810c0d0 0x7f6c6819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c500a2da0 0x7f6c500a5260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c681035a0 0x7f6c68198820 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 --2- 172.21.15.7:0/3385264151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c68102c80 0x7f6c681982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 -- 172.21.15.7:0/3385264151 >> 172.21.15.7:0/3385264151 conn(0x7f6c68077990 msgr2=0x7f6c680fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 -- 172.21.15.7:0/3385264151 shutdown_connections 2024-09-10T10:54:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:33.114+0000 7f6c6d328640 1 -- 172.21.15.7:0/3385264151 wait complete. 2024-09-10T10:54:33.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 955967 10.90 MB/sec execute 333 sec latency 66.857 ms 2024-09-10T10:54:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:33 smithi103 ceph-mon[35160]: pgmap v271: 97 pgs: 97 active+clean; 253 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 238 op/s 2024-09-10T10:54:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:33 smithi007 ceph-mon[30873]: pgmap v271: 97 pgs: 97 active+clean; 253 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 238 op/s 2024-09-10T10:54:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:33 smithi133 ceph-mon[35282]: pgmap v271: 97 pgs: 97 active+clean; 253 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 238 op/s 2024-09-10T10:54:34.524 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.87 MB/sec execute 334 sec latency 633.028 ms 2024-09-10T10:54:34.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:34 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:34.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:34 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:34.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:34 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:35.525 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.84 MB/sec execute 335 sec latency 1633.142 ms 2024-09-10T10:54:35.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:35 smithi007 ceph-mon[30873]: pgmap v272: 97 pgs: 97 active+clean; 253 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 238 op/s 2024-09-10T10:54:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:35 smithi133 ceph-mon[35282]: pgmap v272: 97 pgs: 97 active+clean; 253 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 238 op/s 2024-09-10T10:54:36.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:35 smithi103 ceph-mon[35160]: pgmap v272: 97 pgs: 97 active+clean; 253 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 238 op/s 2024-09-10T10:54:36.525 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.81 MB/sec execute 336 sec latency 2633.289 ms 2024-09-10T10:54:37.525 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.78 MB/sec execute 337 sec latency 3633.405 ms 2024-09-10T10:54:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:37 smithi007 ceph-mon[30873]: pgmap v273: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 397 op/s 2024-09-10T10:54:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:37 smithi133 ceph-mon[35282]: pgmap v273: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 397 op/s 2024-09-10T10:54:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:37 smithi103 ceph-mon[35160]: pgmap v273: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 397 op/s 2024-09-10T10:54:38.525 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.74 MB/sec execute 338 sec latency 4633.556 ms 2024-09-10T10:54:39.525 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.71 MB/sec execute 339 sec latency 5633.638 ms 2024-09-10T10:54:39.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:39 smithi007 ceph-mon[30873]: pgmap v274: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 270 op/s 2024-09-10T10:54:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:39 smithi133 ceph-mon[35282]: pgmap v274: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 270 op/s 2024-09-10T10:54:40.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:39 smithi103 ceph-mon[35160]: pgmap v274: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 270 op/s 2024-09-10T10:54:40.525 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.68 MB/sec execute 340 sec latency 6633.764 ms 2024-09-10T10:54:41.525 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.65 MB/sec execute 341 sec latency 7633.882 ms 2024-09-10T10:54:41.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:41 smithi007 ceph-mon[30873]: pgmap v275: 97 pgs: 97 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 279 op/s 2024-09-10T10:54:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:41 smithi133 ceph-mon[35282]: pgmap v275: 97 pgs: 97 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 279 op/s 2024-09-10T10:54:42.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:41 smithi103 ceph-mon[35160]: pgmap v275: 97 pgs: 97 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 279 op/s 2024-09-10T10:54:42.525 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.62 MB/sec execute 342 sec latency 8633.990 ms 2024-09-10T10:54:43.525 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.59 MB/sec execute 343 sec latency 9634.089 ms 2024-09-10T10:54:43.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:43 smithi007 ceph-mon[30873]: pgmap v276: 97 pgs: 97 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 3.0 MiB/s wr, 168 op/s 2024-09-10T10:54:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:43 smithi133 ceph-mon[35282]: pgmap v276: 97 pgs: 97 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 3.0 MiB/s wr, 168 op/s 2024-09-10T10:54:44.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:43 smithi103 ceph-mon[35160]: pgmap v276: 97 pgs: 97 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 3.0 MiB/s wr, 168 op/s 2024-09-10T10:54:44.526 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.56 MB/sec execute 344 sec latency 10634.184 ms 2024-09-10T10:54:45.526 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.53 MB/sec execute 345 sec latency 11634.242 ms 2024-09-10T10:54:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:45 smithi133 ceph-mon[35282]: pgmap v277: 97 pgs: 97 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 3.0 MiB/s wr, 168 op/s 2024-09-10T10:54:46.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:45 smithi103 ceph-mon[35160]: pgmap v277: 97 pgs: 97 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 3.0 MiB/s wr, 168 op/s 2024-09-10T10:54:46.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:45 smithi007 ceph-mon[30873]: pgmap v277: 97 pgs: 97 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 3.0 MiB/s wr, 168 op/s 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.216+0000 7fd55cc65640 1 -- 172.21.15.7:0/1499828085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd55810c0d0 msgr2=0x7fd55810e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.216+0000 7fd55cc65640 1 --2- 172.21.15.7:0/1499828085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd55810c0d0 0x7fd55810e570 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd550009900 tx=0x7fd55002a480 comp rx=0 tx=0).stop 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.216+0000 7fd55cc65640 1 -- 172.21.15.7:0/1499828085 shutdown_connections 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.216+0000 7fd55cc65640 1 --2- 172.21.15.7:0/1499828085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd55810c0d0 0x7fd55810e570 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.216+0000 7fd55cc65640 1 --2- 172.21.15.7:0/1499828085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd558103660 0x7fd55810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.216+0000 7fd55cc65640 1 --2- 172.21.15.7:0/1499828085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd558102d40 0x7fd558103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.216+0000 7fd55cc65640 1 -- 172.21.15.7:0/1499828085 >> 172.21.15.7:0/1499828085 conn(0x7fd5580fc6f0 msgr2=0x7fd5580feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.216+0000 7fd55cc65640 1 -- 172.21.15.7:0/1499828085 shutdown_connections 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.216+0000 7fd55cc65640 1 -- 172.21.15.7:0/1499828085 wait complete. 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55cc65640 1 Processor -- start 2024-09-10T10:54:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55cc65640 1 -- start start 2024-09-10T10:54:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55cc65640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd558102d40 0x7fd558198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55cc65640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd558103660 0x7fd558198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55cc65640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd55810c0d0 0x7fd55819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55cc65640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd558111ee0 con 0x7fd558102d40 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55cc65640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd558111d60 con 0x7fd558103660 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55cc65640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd558112060 con 0x7fd55810c0d0 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd558102d40 0x7fd558198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd558102d40 0x7fd558198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50412/0 (socket says 172.21.15.7:50412) 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55659c640 1 -- 172.21.15.7:0/3442710038 learned_addr learned my addr 172.21.15.7:0/3442710038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd556d9d640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd55810c0d0 0x7fd55819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd555d9b640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd558103660 0x7fd558198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55659c640 1 -- 172.21.15.7:0/3442710038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd55810c0d0 msgr2=0x7fd55819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55659c640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd55810c0d0 0x7fd55819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55659c640 1 -- 172.21.15.7:0/3442710038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd558103660 msgr2=0x7fd558198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.217+0000 7fd55659c640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd558103660 0x7fd558198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.218+0000 7fd55659c640 1 -- 172.21.15.7:0/3442710038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55819d300 con 0x7fd558102d40 2024-09-10T10:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.218+0000 7fd555d9b640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd558103660 0x7fd558198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.218+0000 7fd55659c640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd558102d40 0x7fd558198310 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd54800e9f0 tx=0x7fd54800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.218+0000 7fd5477fe640 1 -- 172.21.15.7:0/3442710038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54800cd80 con 0x7fd558102d40 2024-09-10T10:54:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.218+0000 7fd5477fe640 1 -- 172.21.15.7:0/3442710038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd548010910 con 0x7fd558102d40 2024-09-10T10:54:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.218+0000 7fd55cc65640 1 -- 172.21.15.7:0/3442710038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55819d5f0 con 0x7fd558102d40 2024-09-10T10:54:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.218+0000 7fd5477fe640 1 -- 172.21.15.7:0/3442710038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548018990 con 0x7fd558102d40 2024-09-10T10:54:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.218+0000 7fd55cc65640 1 -- 172.21.15.7:0/3442710038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5581a4e80 con 0x7fd558102d40 2024-09-10T10:54:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd5477fe640 1 -- 172.21.15.7:0/3442710038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd548010ac0 con 0x7fd558102d40 2024-09-10T10:54:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd55fa06740 1 Processor -- start 2024-09-10T10:54:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd55fa06740 1 -- start start 2024-09-10T10:54:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc4fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc50410 0x56239cc5a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 0x56239cc5d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd55fa06740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56239cbb42f0 con 0x56239cc4faf0 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd55fa06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56239cb5c270 con 0x56239cc50410 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd55fa06740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56239cbb02b0 con 0x56239cc5acb0 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd556d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc4fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.221+0000 7fd556d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc4fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50422/0 (socket says 172.21.15.7:50422) 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd556d9d640 1 -- 172.21.15.7:0/639113286 learned_addr learned my addr 172.21.15.7:0/639113286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd555d9b640 1 --2- 172.21.15.7:0/639113286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 0x56239cc5d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd55659c640 1 --2- 172.21.15.7:0/639113286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc50410 0x56239cc5a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd5477fe640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5400a2da0 0x7fd5400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd556d9d640 1 -- 172.21.15.7:0/639113286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 msgr2=0x56239cc5d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd556d9d640 1 --2- 172.21.15.7:0/639113286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 0x56239cc5d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd556d9d640 1 -- 172.21.15.7:0/639113286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc50410 msgr2=0x56239cc5a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd556d9d640 1 --2- 172.21.15.7:0/639113286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc50410 0x56239cc5a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd556d9d640 1 -- 172.21.15.7:0/639113286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239cb5b340 con 0x56239cc4faf0 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd5477fe640 1 -- 172.21.15.7:0/3442710038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7fd5480cf820 con 0x7fd558102d40 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd556d9d640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5400a2da0 0x7fd5400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd556d9d640 1 --2- 172.21.15.7:0/639113286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc4fed0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd55002aa70 tx=0x7fd550005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413bb1cae73d4d8b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd544ff9640 1 -- 172.21.15.7:0/639113286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550040070 con 0x56239cc4faf0 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd544ff9640 1 -- 172.21.15.7:0/639113286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550003c90 con 0x56239cc4faf0 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.222+0000 7fd544ff9640 1 -- 172.21.15.7:0/639113286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550035040 con 0x56239cc4faf0 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.223+0000 7fd55fa06740 1 -- 172.21.15.7:0/639113286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 msgr2=0x56239cc4fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.223+0000 7fd556d9d640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5400a2da0 0x7fd5400a5260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd550005ec0 tx=0x7fd550046850 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.223+0000 7fd55fa06740 1 --2- 172.21.15.7:0/639113286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc4fed0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd55002aa70 tx=0x7fd550005c00 comp rx=0 tx=0).stop 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.223+0000 7fd55fa06740 1 -- 172.21.15.7:0/639113286 shutdown_connections 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.223+0000 7fd55fa06740 1 --2- 172.21.15.7:0/639113286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 0x56239cc5d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.223+0000 7fd55fa06740 1 --2- 172.21.15.7:0/639113286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc50410 0x56239cc5a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.223+0000 7fd55fa06740 1 --2- 172.21.15.7:0/639113286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc4fed0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.223+0000 7fd55fa06740 1 -- 172.21.15.7:0/639113286 >> 172.21.15.7:0/639113286 conn(0x56239cc4c060 msgr2=0x56239cc4c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.223+0000 7fd55fa06740 1 -- 172.21.15.7:0/639113286 shutdown_connections 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.223+0000 7fd55fa06740 1 -- 172.21.15.7:0/639113286 wait complete. 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55fa06740 1 -- start start 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55fa06740 1 Processor -- start 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc57af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc50410 0x56239cc58030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 0x56239cc58570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55fa06740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56239cc5d880 con 0x56239cc4faf0 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55fa06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56239cb3d290 con 0x56239cc5acb0 2024-09-10T10:54:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55fa06740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56239cc5da00 con 0x56239cc50410 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55fa06740 10 client.0 _ll_register_callbacks cb 0x56239cb6e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc57af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc57af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50424/0 (socket says 172.21.15.7:50424) 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd55659c640 1 -- 172.21.15.7:0/3359281262 learned_addr learned my addr 172.21.15.7:0/3359281262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.224+0000 7fd556d9d640 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 0x56239cc58570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd555d9b640 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc50410 0x56239cc58030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd55659c640 1 -- 172.21.15.7:0/3359281262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc50410 msgr2=0x56239cc58030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd55659c640 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc50410 0x56239cc58030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd55659c640 1 -- 172.21.15.7:0/3359281262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 msgr2=0x56239cc58570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd55659c640 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 0x56239cc58570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd55659c640 1 -- 172.21.15.7:0/3359281262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239c8c7210 con 0x56239cc4faf0 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd556d9d640 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 0x56239cc58570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd55659c640 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc57af0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd54800c6a0 tx=0x7fd54809b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bd19ad8acde1bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd53dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd53dffb640 1 -- 172.21.15.7:0/3359281262 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548018d30 con 0x56239cc4faf0 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd53dffb640 1 -- 172.21.15.7:0/3359281262 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7fd54809b160 con 0x56239cc4faf0 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd55fa06740 1 -- 172.21.15.7:0/3359281262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd550003c90 con 0x56239cc4faf0 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd53dffb640 1 -- 172.21.15.7:0/3359281262 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd5480a00c0 con 0x56239cc4faf0 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd55fa06740 10 client.15963 fetch_fsmap learned FSMap version 38 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd55fa06740 1 -- 172.21.15.7:0/3359281262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56239c993620 con 0x56239cc4faf0 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd53dffb640 1 -- 172.21.15.7:0/3359281262 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550003c90 con 0x56239cc4faf0 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.225+0000 7fd53dffb640 1 -- 172.21.15.7:0/3359281262 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7fd548097b80 con 0x56239cc4faf0 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.226+0000 7fd55fa06740 10 client.15963 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.226+0000 7fd55fa06740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.226+0000 7fd55fa06740 20 client.15963 populate_metadata read hostname 'smithi007' 2024-09-10T10:54:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.226+0000 7fd55fa06740 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56239cc6abb0 0x56239cc6d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.226+0000 7fd55fa06740 4 client.15963 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:54:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.226+0000 7fd55fa06740 1 -- 172.21.15.7:0/3359281262 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239cbc1a20 con 0x56239cc6abb0 2024-09-10T10:54:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.226+0000 7fd555d9b640 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56239cc6abb0 0x56239cc6d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.226+0000 7fd555d9b640 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56239cc6abb0 0x56239cc6d020 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b62f653cea8f07f server_cookie=3f7053dad16212ad in_seq=0 out_seq=0 2024-09-10T10:54:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.226+0000 7fd53dffb640 10 client.15963 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd53dffb640 1 -- 172.21.15.7:0/3359281262 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56239cbc1a20 con 0x56239cc6abb0 2024-09-10T10:54:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd53dffb640 10 client.15963 handle_command_reply: tid=0 2024-09-10T10:54:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd55fa06740 1 client.15963 shutdown 2024-09-10T10:54:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd55fa06740 1 -- 172.21.15.7:0/3359281262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 msgr2=0x56239cc57af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd55fa06740 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc57af0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd54800c6a0 tx=0x7fd54809b730 comp rx=0 tx=0).stop 2024-09-10T10:54:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd55fa06740 1 -- 172.21.15.7:0/3359281262 shutdown_connections 2024-09-10T10:54:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd55fa06740 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56239cc6abb0 0x56239cc6d020 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd55fa06740 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 0x56239cc58570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd53dffb640 0 client.15963 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd55fa06740 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc50410 0x56239cc58030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd55fa06740 1 --2- 172.21.15.7:0/3359281262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc57af0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd55fa06740 1 -- 172.21.15.7:0/3359281262 >> 172.21.15.7:0/3359281262 conn(0x56239cc4c060 msgr2=0x56239cc51dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.227+0000 7fd55fa06740 1 -- 172.21.15.7:0/3359281262 shutdown_connections 2024-09-10T10:54:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.228+0000 7fd55fa06740 1 -- 172.21.15.7:0/3359281262 wait complete. 2024-09-10T10:54:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.228+0000 7fd55fa06740 20 client.15963 trim_cache size 0 max 16384 2024-09-10T10:54:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55fa06740 1 Processor -- start 2024-09-10T10:54:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55fa06740 1 -- start start 2024-09-10T10:54:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc50410 0x56239cc72930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 0x56239cc72e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55fa06740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581b01c0 con 0x56239cc4faf0 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55fa06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581ada30 con 0x56239cc50410 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55fa06740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581ab8a0 con 0x56239cc5acb0 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50430/0 (socket says 172.21.15.7:50430) 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd55659c640 1 -- 172.21.15.7:0/2636170783 learned_addr learned my addr 172.21.15.7:0/2636170783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd556d9d640 1 --2- 172.21.15.7:0/2636170783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 0x56239cc72e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.243+0000 7fd555d9b640 1 --2- 172.21.15.7:0/2636170783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc50410 0x56239cc72930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.244+0000 7fd55659c640 1 -- 172.21.15.7:0/2636170783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 msgr2=0x56239cc72e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.244+0000 7fd55659c640 1 --2- 172.21.15.7:0/2636170783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 0x56239cc72e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.244+0000 7fd55659c640 1 -- 172.21.15.7:0/2636170783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc50410 msgr2=0x56239cc72930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.244+0000 7fd55659c640 1 --2- 172.21.15.7:0/2636170783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc50410 0x56239cc72930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.244+0000 7fd55659c640 1 -- 172.21.15.7:0/2636170783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239cc4be90 con 0x56239cc4faf0 2024-09-10T10:54:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.244+0000 7fd556d9d640 1 --2- 172.21.15.7:0/2636170783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 0x56239cc72e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.244+0000 7fd55659c640 1 --2- 172.21.15.7:0/2636170783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc583a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd54800cfd0 tx=0x7fd54809b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8547105e63ed296c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.244+0000 7fd53ffff640 1 -- 172.21.15.7:0/2636170783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54809bae0 con 0x56239cc4faf0 2024-09-10T10:54:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.244+0000 7fd53ffff640 1 -- 172.21.15.7:0/2636170783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54809af40 con 0x56239cc4faf0 2024-09-10T10:54:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.244+0000 7fd53ffff640 1 -- 172.21.15.7:0/2636170783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54809b160 con 0x56239cc4faf0 2024-09-10T10:54:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 -- 172.21.15.7:0/2636170783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 msgr2=0x56239cc583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 --2- 172.21.15.7:0/2636170783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc583a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd54800cfd0 tx=0x7fd54809b7a0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 -- 172.21.15.7:0/2636170783 shutdown_connections 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 --2- 172.21.15.7:0/2636170783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc5acb0 0x56239cc72e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 --2- 172.21.15.7:0/2636170783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc50410 0x56239cc72930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 --2- 172.21.15.7:0/2636170783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc583a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 -- 172.21.15.7:0/2636170783 >> 172.21.15.7:0/2636170783 conn(0x56239cc4c060 msgr2=0x56239cc59b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 -- 172.21.15.7:0/2636170783 shutdown_connections 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 -- 172.21.15.7:0/2636170783 wait complete. 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 -- start start 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 Processor -- start 2024-09-10T10:54:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc5ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc50410 0x56239cc5efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 0x56239cc5f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581abef0 con 0x56239cc4faf0 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581ad360 con 0x56239cc5acb0 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581ac5c0 con 0x56239cc50410 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55fa06740 10 client.0 _ll_register_callbacks cb 0x7fd54800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.245+0000 7fd55659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc5ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc5ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50446/0 (socket says 172.21.15.7:50446) 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55659c640 1 -- 172.21.15.7:0/2730253833 learned_addr learned my addr 172.21.15.7:0/2730253833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd555d9b640 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc50410 0x56239cc5efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd556d9d640 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 0x56239cc5f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55659c640 1 -- 172.21.15.7:0/2730253833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc50410 msgr2=0x56239cc5efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55659c640 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc50410 0x56239cc5efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55659c640 1 -- 172.21.15.7:0/2730253833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 msgr2=0x56239cc5f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55659c640 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 0x56239cc5f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55659c640 1 -- 172.21.15.7:0/2730253833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239cc472d0 con 0x56239cc4faf0 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55659c640 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc5ea60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd54809b9c0 tx=0x7fd54809b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef81e2ccb3038a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd53dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd53dffb640 1 -- 172.21.15.7:0/2730253833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54809bae0 con 0x56239cc4faf0 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd53dffb640 1 -- 172.21.15.7:0/2730253833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7fd548097a60 con 0x56239cc4faf0 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55fa06740 1 -- 172.21.15.7:0/2730253833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd54809af40 con 0x56239cc4faf0 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd53dffb640 1 -- 172.21.15.7:0/2730253833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd5480a4980 con 0x56239cc4faf0 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55fa06740 10 client.15969 fetch_fsmap learned FSMap version 38 2024-09-10T10:54:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd55fa06740 1 -- 172.21.15.7:0/2730253833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56239cc4ccf0 con 0x56239cc4faf0 2024-09-10T10:54:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.246+0000 7fd53dffb640 1 -- 172.21.15.7:0/2730253833 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd54809af40 con 0x56239cc4faf0 2024-09-10T10:54:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.247+0000 7fd53dffb640 1 -- 172.21.15.7:0/2730253833 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7fd5480ba4b0 con 0x56239cc4faf0 2024-09-10T10:54:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.247+0000 7fd55fa06740 10 client.15969 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T10:54:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.247+0000 7fd55fa06740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:54:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.247+0000 7fd55fa06740 20 client.15969 populate_metadata read hostname 'smithi007' 2024-09-10T10:54:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.247+0000 7fd55fa06740 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56239cc6abb0 0x56239ccd7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.247+0000 7fd55fa06740 4 client.15969 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:54:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.247+0000 7fd55fa06740 1 -- 172.21.15.7:0/2730253833 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56239cbbbd70 con 0x56239cc6abb0 2024-09-10T10:54:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.247+0000 7fd555d9b640 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56239cc6abb0 0x56239ccd7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.247+0000 7fd555d9b640 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56239cc6abb0 0x56239ccd7d50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc19188c56d45ff5 server_cookie=60c22ca0d4f8b363 in_seq=0 out_seq=0 2024-09-10T10:54:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:46.247+0000 7fd53dffb640 10 client.15969 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:46.526 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.50 MB/sec execute 346 sec latency 12634.311 ms 2024-09-10T10:54:47.526 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956280 10.47 MB/sec execute 347 sec latency 13634.413 ms 2024-09-10T10:54:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.845+0000 7fd53dffb640 1 -- 172.21.15.7:0/2730253833 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56239cbbbd70 con 0x56239cc6abb0 2024-09-10T10:54:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.845+0000 7fd53dffb640 10 client.15969 handle_command_reply: tid=0 2024-09-10T10:54:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.845+0000 7fd55fa06740 1 client.15969 shutdown 2024-09-10T10:54:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.845+0000 7fd55fa06740 1 -- 172.21.15.7:0/2730253833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 msgr2=0x56239cc5ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.845+0000 7fd55fa06740 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc5ea60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd54809b9c0 tx=0x7fd54809b410 comp rx=0 tx=0).stop 2024-09-10T10:54:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55fa06740 1 -- 172.21.15.7:0/2730253833 shutdown_connections 2024-09-10T10:54:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55fa06740 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56239cc6abb0 0x56239ccd7d50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55fa06740 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56239cc5acb0 0x56239cc5f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55fa06740 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56239cc50410 0x56239cc5efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd53dffb640 0 client.15969 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:54:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55fa06740 1 --2- 172.21.15.7:0/2730253833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56239cc4faf0 0x56239cc5ea60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55fa06740 1 -- 172.21.15.7:0/2730253833 >> 172.21.15.7:0/2730253833 conn(0x56239cc4c060 msgr2=0x56239cc4a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55fa06740 1 -- 172.21.15.7:0/2730253833 shutdown_connections 2024-09-10T10:54:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55fa06740 1 -- 172.21.15.7:0/2730253833 wait complete. 2024-09-10T10:54:47.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55fa06740 20 client.15969 trim_cache size 0 max 16384 2024-09-10T10:54:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:54:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:54:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:54:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:54:47.849 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55cc65640 1 -- 172.21.15.7:0/3442710038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5400a2da0 msgr2=0x7fd5400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55cc65640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5400a2da0 0x7fd5400a5260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd550005ec0 tx=0x7fd550046850 comp rx=0 tx=0).stop 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55cc65640 1 -- 172.21.15.7:0/3442710038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd558102d40 msgr2=0x7fd558198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.846+0000 7fd55cc65640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd558102d40 0x7fd558198310 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd54800e9f0 tx=0x7fd54800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.847+0000 7fd55cc65640 1 -- 172.21.15.7:0/3442710038 shutdown_connections 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.847+0000 7fd55cc65640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd55810c0d0 0x7fd55819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.847+0000 7fd55cc65640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd558103660 0x7fd558198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.847+0000 7fd55cc65640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5400a2da0 0x7fd5400a5260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.847+0000 7fd55cc65640 1 --2- 172.21.15.7:0/3442710038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd558102d40 0x7fd558198310 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.847+0000 7fd55cc65640 1 -- 172.21.15.7:0/3442710038 >> 172.21.15.7:0/3442710038 conn(0x7fd5580fc6f0 msgr2=0x7fd5580fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.847+0000 7fd55cc65640 1 -- 172.21.15.7:0/3442710038 shutdown_connections 2024-09-10T10:54:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:54:47.847+0000 7fd55cc65640 1 -- 172.21.15.7:0/3442710038 wait complete. 2024-09-10T10:54:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:47 smithi133 ceph-mon[35282]: pgmap v278: 97 pgs: 97 active+clean; 268 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-09-10T10:54:48.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:47 smithi103 ceph-mon[35160]: pgmap v278: 97 pgs: 97 active+clean; 268 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-09-10T10:54:48.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:47 smithi007 ceph-mon[30873]: pgmap v278: 97 pgs: 97 active+clean; 268 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.2 MiB/s wr, 178 op/s 2024-09-10T10:54:48.526 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956465 10.44 MB/sec execute 348 sec latency 13946.691 ms 2024-09-10T10:54:49.526 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956465 10.41 MB/sec execute 349 sec latency 1546.934 ms 2024-09-10T10:54:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:49 smithi133 ceph-mon[35282]: pgmap v279: 97 pgs: 97 active+clean; 268 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 218 KiB/s wr, 18 op/s 2024-09-10T10:54:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:49 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:50.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:49 smithi103 ceph-mon[35160]: pgmap v279: 97 pgs: 97 active+clean; 268 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 218 KiB/s wr, 18 op/s 2024-09-10T10:54:50.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:49 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:49 smithi007 ceph-mon[30873]: pgmap v279: 97 pgs: 97 active+clean; 268 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 218 KiB/s wr, 18 op/s 2024-09-10T10:54:50.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:49 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:54:50.526 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956465 10.38 MB/sec execute 350 sec latency 2547.043 ms 2024-09-10T10:54:51.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956533 10.35 MB/sec execute 351 sec latency 3277.630 ms 2024-09-10T10:54:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:51 smithi133 ceph-mon[35282]: pgmap v280: 97 pgs: 97 active+clean; 264 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 295 KiB/s wr, 36 op/s 2024-09-10T10:54:52.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:51 smithi103 ceph-mon[35160]: pgmap v280: 97 pgs: 97 active+clean; 264 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 295 KiB/s wr, 36 op/s 2024-09-10T10:54:52.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:51 smithi007 ceph-mon[30873]: pgmap v280: 97 pgs: 97 active+clean; 264 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 295 KiB/s wr, 36 op/s 2024-09-10T10:54:52.526 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956533 10.32 MB/sec execute 352 sec latency 1224.557 ms 2024-09-10T10:54:53.526 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956533 10.29 MB/sec execute 353 sec latency 2224.679 ms 2024-09-10T10:54:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:53 smithi133 ceph-mon[35282]: pgmap v281: 97 pgs: 97 active+clean; 264 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 236 KiB/s wr, 26 op/s 2024-09-10T10:54:54.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:53 smithi103 ceph-mon[35160]: pgmap v281: 97 pgs: 97 active+clean; 264 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 236 KiB/s wr, 26 op/s 2024-09-10T10:54:54.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:53 smithi007 ceph-mon[30873]: pgmap v281: 97 pgs: 97 active+clean; 264 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 236 KiB/s wr, 26 op/s 2024-09-10T10:54:54.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956533 10.26 MB/sec execute 354 sec latency 3224.826 ms 2024-09-10T10:54:55.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956533 10.23 MB/sec execute 355 sec latency 4224.909 ms 2024-09-10T10:54:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:55 smithi133 ceph-mon[35282]: pgmap v282: 97 pgs: 97 active+clean; 264 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 236 KiB/s wr, 26 op/s 2024-09-10T10:54:56.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:55 smithi103 ceph-mon[35160]: pgmap v282: 97 pgs: 97 active+clean; 264 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 236 KiB/s wr, 26 op/s 2024-09-10T10:54:56.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:55 smithi007 ceph-mon[30873]: pgmap v282: 97 pgs: 97 active+clean; 264 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 236 KiB/s wr, 26 op/s 2024-09-10T10:54:56.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956656 10.20 MB/sec execute 356 sec latency 4958.439 ms 2024-09-10T10:54:57.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956656 10.17 MB/sec execute 357 sec latency 1196.259 ms 2024-09-10T10:54:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:57 smithi133 ceph-mon[35282]: pgmap v283: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 996 KiB/s wr, 57 op/s 2024-09-10T10:54:58.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:57 smithi103 ceph-mon[35160]: pgmap v283: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 996 KiB/s wr, 57 op/s 2024-09-10T10:54:58.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:57 smithi007 ceph-mon[30873]: pgmap v283: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 996 KiB/s wr, 57 op/s 2024-09-10T10:54:58.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956656 10.15 MB/sec execute 358 sec latency 2196.381 ms 2024-09-10T10:54:59.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956656 10.12 MB/sec execute 359 sec latency 3196.452 ms 2024-09-10T10:54:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:54:59 smithi133 ceph-mon[35282]: pgmap v284: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 838 KiB/s wr, 48 op/s 2024-09-10T10:55:00.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:54:59 smithi103 ceph-mon[35160]: pgmap v284: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 838 KiB/s wr, 48 op/s 2024-09-10T10:55:00.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:54:59 smithi007 ceph-mon[30873]: pgmap v284: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 838 KiB/s wr, 48 op/s 2024-09-10T10:55:00.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 956723 10.09 MB/sec execute 360 sec latency 3817.935 ms 2024-09-10T10:55:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.949+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/3924677562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e68101ae0 msgr2=0x7f3e68101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.949+0000 7f3e6fa5e640 1 --2- 172.21.15.7:0/3924677562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e68101ae0 0x7f3e68101f40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009930 tx=0x7f3e5c02a420 comp rx=0 tx=0).stop 2024-09-10T10:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.949+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/3924677562 shutdown_connections 2024-09-10T10:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.949+0000 7f3e6fa5e640 1 --2- 172.21.15.7:0/3924677562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e68102480 0x7f3e6810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.949+0000 7f3e6fa5e640 1 --2- 172.21.15.7:0/3924677562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e68101ae0 0x7f3e68101f40 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.949+0000 7f3e6fa5e640 1 --2- 172.21.15.7:0/3924677562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e68107b80 0x7f3e68107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.949+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/3924677562 >> 172.21.15.7:0/3924677562 conn(0x7f3e680fd7e0 msgr2=0x7f3e680ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.949+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/3924677562 shutdown_connections 2024-09-10T10:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.949+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/3924677562 wait complete. 2024-09-10T10:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6fa5e640 1 Processor -- start 2024-09-10T10:55:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6fa5e640 1 -- start start 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6fa5e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e68101ae0 0x7f3e68198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6fa5e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e68102480 0x7f3e68198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6fa5e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e68107b80 0x7f3e6819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6fa5e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68111f80 con 0x7f3e68107b80 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6fa5e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68111e00 con 0x7f3e68101ae0 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6fa5e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68112100 con 0x7f3e68102480 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6dffb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e68107b80 0x7f3e6819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6dffb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e68107b80 0x7f3e6819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43284/0 (socket says 172.21.15.7:43284) 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6dffb640 1 -- 172.21.15.7:0/1507071644 learned_addr learned my addr 172.21.15.7:0/1507071644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e68102480 0x7f3e68198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.950+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e68101ae0 0x7f3e68198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e6dffb640 1 -- 172.21.15.7:0/1507071644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e68102480 msgr2=0x7f3e68198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e6dffb640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e68102480 0x7f3e68198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e6dffb640 1 -- 172.21.15.7:0/1507071644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e68101ae0 msgr2=0x7f3e68198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e6dffb640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e68101ae0 0x7f3e68198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e6dffb640 1 -- 172.21.15.7:0/1507071644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6819d400 con 0x7f3e68107b80 2024-09-10T10:55:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e68102480 0x7f3e68198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e6dffb640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e68107b80 0x7f3e6819ccd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000eb10 tx=0x7f3e6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e5a7fc640 1 -- 172.21.15.7:0/1507071644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60004b20 con 0x7f3e68107b80 2024-09-10T10:55:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e5a7fc640 1 -- 172.21.15.7:0/1507071644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e60004cc0 con 0x7f3e68107b80 2024-09-10T10:55:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e5a7fc640 1 -- 172.21.15.7:0/1507071644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60010760 con 0x7f3e68107b80 2024-09-10T10:55:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/1507071644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6819d6f0 con 0x7f3e68107b80 2024-09-10T10:55:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.951+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/1507071644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e681a4f80 con 0x7f3e68107b80 2024-09-10T10:55:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.954+0000 7f3e5a7fc640 1 -- 172.21.15.7:0/1507071644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3e60005020 con 0x7f3e68107b80 2024-09-10T10:55:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e727fe740 1 Processor -- start 2024-09-10T10:55:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e727fe740 1 -- start start 2024-09-10T10:55:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 0x56075762eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757639770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560757639cb0 0x56075763c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e727fe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607575932f0 con 0x56075762f410 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e727fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56075753b270 con 0x56075762eaf0 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e727fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56075758f2b0 con 0x560757639cb0 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e6d7fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757639770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e6d7fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757639770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43294/0 (socket says 172.21.15.7:43294) 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/2825233303 learned_addr learned my addr 172.21.15.7:0/2825233303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/2825233303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560757639cb0 0x56075763c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/2825233303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 0x56075762eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/2825233303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560757639cb0 msgr2=0x56075763c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/2825233303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560757639cb0 0x56075763c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/2825233303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 msgr2=0x56075762eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/2825233303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 0x56075762eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/2825233303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075753a340 con 0x56075762f410 2024-09-10T10:55:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.955+0000 7f3e5a7fc640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3e540a2da0 0x7f3e540a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.956+0000 7f3e6dffb640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3e540a2da0 0x7f3e540a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.956+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/2825233303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757639770 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400aba0 tx=0x7f3e6400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81ddb67dfeb644b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.956+0000 7f3e5a7fc640 1 -- 172.21.15.7:0/1507071644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7f3e600cfb20 con 0x7f3e68107b80 2024-09-10T10:55:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.956+0000 7f3e537fe640 1 -- 172.21.15.7:0/2825233303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6401d070 con 0x56075762f410 2024-09-10T10:55:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.956+0000 7f3e537fe640 1 -- 172.21.15.7:0/2825233303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e64018b20 con 0x56075762f410 2024-09-10T10:55:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.956+0000 7f3e537fe640 1 -- 172.21.15.7:0/2825233303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64010b30 con 0x56075762f410 2024-09-10T10:55:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 -- 172.21.15.7:0/2825233303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 msgr2=0x560757639770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e6dffb640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3e540a2da0 0x7f3e540a5260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e6009bd10 tx=0x7f3e6008a040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 --2- 172.21.15.7:0/2825233303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757639770 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400aba0 tx=0x7f3e6400f790 comp rx=0 tx=0).stop 2024-09-10T10:55:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 -- 172.21.15.7:0/2825233303 shutdown_connections 2024-09-10T10:55:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 --2- 172.21.15.7:0/2825233303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560757639cb0 0x56075763c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 --2- 172.21.15.7:0/2825233303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757639770 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 --2- 172.21.15.7:0/2825233303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 0x56075762eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 -- 172.21.15.7:0/2825233303 >> 172.21.15.7:0/2825233303 conn(0x56075762b060 msgr2=0x56075762b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 -- 172.21.15.7:0/2825233303 shutdown_connections 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 -- 172.21.15.7:0/2825233303 wait complete. 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 -- start start 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.957+0000 7f3e727fe740 1 Processor -- start 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 0x560757636b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757637080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560757639cb0 0x5607576375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e727fe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56075763c880 con 0x56075762f410 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e727fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56075751c290 con 0x56075762eaf0 2024-09-10T10:55:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e727fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56075763ca00 con 0x560757639cb0 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e727fe740 10 client.0 _ll_register_callbacks cb 0x56075754d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6d7fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757637080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6d7fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757637080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43310/0 (socket says 172.21.15.7:43310) 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/4128831451 learned_addr learned my addr 172.21.15.7:0/4128831451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 0x560757636b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560757639cb0 0x5607576375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/4128831451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560757639cb0 msgr2=0x5607576375c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560757639cb0 0x5607576375c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/4128831451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 msgr2=0x560757636b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 0x560757636b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/4128831451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607572a6210 con 0x56075762f410 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 0x560757636b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757637080 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e64018440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1592a2af45a32d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4128831451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6401d050 con 0x56075762f410 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.958+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4128831451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7f3e64018900 con 0x56075762f410 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e727fe740 1 -- 172.21.15.7:0/4128831451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e64018b20 con 0x56075762f410 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4128831451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3e64031050 con 0x56075762f410 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e727fe740 10 client.15981 fetch_fsmap learned FSMap version 39 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e727fe740 1 -- 172.21.15.7:0/4128831451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560757372620 con 0x56075762f410 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4128831451 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e64018b20 con 0x56075762f410 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4128831451 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f3e64005160 con 0x56075762f410 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e727fe740 10 client.15981 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e727fe740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:55:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e727fe740 20 client.15981 populate_metadata read hostname 'smithi007' 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560757649b40 0x56075764bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e727fe740 4 client.15981 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e727fe740 1 -- 172.21.15.7:0/4128831451 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607575a0a20 con 0x560757649b40 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.959+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560757649b40 0x56075764bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.960+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560757649b40 0x56075764bf30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9adc89379f944c server_cookie=8c36d3e6164a4dda in_seq=0 out_seq=0 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.960+0000 7f3e50ff9640 10 client.15981 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.960+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4128831451 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607575a0a20 con 0x560757649b40 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.960+0000 7f3e50ff9640 10 client.15981 handle_command_reply: tid=0 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 client.15981 shutdown 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 -- 172.21.15.7:0/4128831451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 msgr2=0x560757637080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757637080 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e64018440 comp rx=0 tx=0).stop 2024-09-10T10:55:00.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 -- 172.21.15.7:0/4128831451 shutdown_connections 2024-09-10T10:55:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560757649b40 0x56075764bf30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560757639cb0 0x5607576375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x560757637080 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e50ff9640 0 client.15981 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4128831451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56075762eaf0 0x560757636b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 -- 172.21.15.7:0/4128831451 >> 172.21.15.7:0/4128831451 conn(0x56075762b060 msgr2=0x560757630dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 -- 172.21.15.7:0/4128831451 shutdown_connections 2024-09-10T10:55:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 1 -- 172.21.15.7:0/4128831451 wait complete. 2024-09-10T10:55:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.961+0000 7f3e727fe740 20 client.15981 trim_cache size 0 max 16384 2024-09-10T10:55:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e727fe740 1 Processor -- start 2024-09-10T10:55:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e727fe740 1 -- start start 2024-09-10T10:55:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56075762eaf0 0x560757631560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x5607576373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560757639cb0 0x560757651560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e727fe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b0230 con 0x56075762f410 2024-09-10T10:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e727fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681ada70 con 0x560757639cb0 2024-09-10T10:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e727fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681ab8e0 con 0x56075762eaf0 2024-09-10T10:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e6d7fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x5607576373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e6d7fa640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x5607576373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43324/0 (socket says 172.21.15.7:43324) 2024-09-10T10:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/3339392010 learned_addr learned my addr 172.21.15.7:0/3339392010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/3339392010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56075762eaf0 0x560757631560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/3339392010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56075762eaf0 msgr2=0x560757631560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/3339392010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56075762eaf0 0x560757631560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/3339392010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560757639cb0 msgr2=0x560757651560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/3339392010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560757639cb0 0x560757651560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.976+0000 7f3e6d7fa640 1 -- 172.21.15.7:0/3339392010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075762ae90 con 0x56075762f410 2024-09-10T10:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/3339392010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x5607576373f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3e64005680 tx=0x7f3e6400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454ab9f244c1e3a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e52ffd640 1 -- 172.21.15.7:0/3339392010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6401d070 con 0x56075762f410 2024-09-10T10:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e52ffd640 1 -- 172.21.15.7:0/3339392010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e6403a070 con 0x56075762f410 2024-09-10T10:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e52ffd640 1 -- 172.21.15.7:0/3339392010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64010720 con 0x56075762f410 2024-09-10T10:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e727fe740 1 -- 172.21.15.7:0/3339392010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 msgr2=0x5607576373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e727fe740 1 --2- 172.21.15.7:0/3339392010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x5607576373f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3e64005680 tx=0x7f3e6400f790 comp rx=0 tx=0).stop 2024-09-10T10:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e727fe740 1 -- 172.21.15.7:0/3339392010 shutdown_connections 2024-09-10T10:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e727fe740 1 --2- 172.21.15.7:0/3339392010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560757639cb0 0x560757651560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e727fe740 1 --2- 172.21.15.7:0/3339392010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762f410 0x5607576373f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e727fe740 1 --2- 172.21.15.7:0/3339392010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56075762eaf0 0x560757631560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e727fe740 1 -- 172.21.15.7:0/3339392010 >> 172.21.15.7:0/3339392010 conn(0x56075762b060 msgr2=0x560757638bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e727fe740 1 -- 172.21.15.7:0/3339392010 shutdown_connections 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.977+0000 7f3e727fe740 1 -- 172.21.15.7:0/3339392010 wait complete. 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e727fe740 1 -- start start 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e727fe740 1 Processor -- start 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762eaf0 0x560757642170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56075762f410 0x560757633630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e727fe740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560757639cb0 0x560757633b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e727fe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681abf30 con 0x56075762eaf0 2024-09-10T10:55:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e727fe740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681ad3a0 con 0x560757639cb0 2024-09-10T10:55:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e727fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681ac600 con 0x56075762f410 2024-09-10T10:55:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e727fe740 10 client.0 _ll_register_callbacks cb 0x7f3e6401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e6cff9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762eaf0 0x560757642170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e6cff9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762eaf0 0x560757642170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43336/0 (socket says 172.21.15.7:43336) 2024-09-10T10:55:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.978+0000 7f3e6cff9640 1 -- 172.21.15.7:0/4173701821 learned_addr learned my addr 172.21.15.7:0/4173701821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e6cff9640 1 -- 172.21.15.7:0/4173701821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56075762f410 msgr2=0x560757633630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56075762f410 0x560757633630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e6cff9640 1 -- 172.21.15.7:0/4173701821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560757639cb0 msgr2=0x560757633b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560757639cb0 0x560757633b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e6cff9640 1 -- 172.21.15.7:0/4173701821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560757634130 con 0x56075762eaf0 2024-09-10T10:55:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e6cff9640 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762eaf0 0x560757642170 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c02aa10 tx=0x7f3e5c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e47e484eaba7aef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:55:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4173701821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c04d020 con 0x56075762eaf0 2024-09-10T10:55:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4173701821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8564+0+0 (secure 0 0 0) 0x7f3e5c005550 con 0x56075762eaf0 2024-09-10T10:55:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e727fe740 1 -- 172.21.15.7:0/4173701821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e6403a070 con 0x56075762eaf0 2024-09-10T10:55:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4173701821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3e5c035040 con 0x56075762eaf0 2024-09-10T10:55:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e727fe740 10 client.15987 fetch_fsmap learned FSMap version 39 2024-09-10T10:55:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e727fe740 1 -- 172.21.15.7:0/4173701821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5607576b4360 con 0x56075762eaf0 2024-09-10T10:55:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.979+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4173701821 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e6403a070 con 0x56075762eaf0 2024-09-10T10:55:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.980+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4173701821 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f3e64019ce0 con 0x56075762eaf0 2024-09-10T10:55:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.980+0000 7f3e727fe740 10 client.15987 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:55:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.980+0000 7f3e727fe740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:55:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.980+0000 7f3e727fe740 20 client.15987 populate_metadata read hostname 'smithi007' 2024-09-10T10:55:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.980+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5607576b4b70 0x5607576b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.980+0000 7f3e727fe740 4 client.15987 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:55:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.980+0000 7f3e727fe740 1 -- 172.21.15.7:0/4173701821 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56075759ad70 con 0x5607576b4b70 2024-09-10T10:55:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.980+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5607576b4b70 0x5607576b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.980+0000 7f3e50ff9640 10 client.15987 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:00.980+0000 7f3e6d7fa640 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5607576b4b70 0x5607576b6f90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=930f09ae344f7a55 server_cookie=45b89da7200b5563 in_seq=0 out_seq=0 2024-09-10T10:55:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e50ff9640 1 -- 172.21.15.7:0/4173701821 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56075759ad70 con 0x5607576b4b70 2024-09-10T10:55:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e50ff9640 10 client.15987 handle_command_reply: tid=0 2024-09-10T10:55:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e727fe740 1 client.15987 shutdown 2024-09-10T10:55:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e727fe740 1 -- 172.21.15.7:0/4173701821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762eaf0 msgr2=0x560757642170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762eaf0 0x560757642170 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c02aa10 tx=0x7f3e5c005e70 comp rx=0 tx=0).stop 2024-09-10T10:55:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e727fe740 1 -- 172.21.15.7:0/4173701821 shutdown_connections 2024-09-10T10:55:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5607576b4b70 0x5607576b6f90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560757639cb0 0x560757633b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56075762f410 0x560757633630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e50ff9640 0 client.15987 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e727fe740 1 --2- 172.21.15.7:0/4173701821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56075762eaf0 0x560757642170 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e727fe740 1 -- 172.21.15.7:0/4173701821 >> 172.21.15.7:0/4173701821 conn(0x56075762b060 msgr2=0x5607576299a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.012+0000 7f3e727fe740 1 -- 172.21.15.7:0/4173701821 shutdown_connections 2024-09-10T10:55:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e727fe740 1 -- 172.21.15.7:0/4173701821 wait complete. 2024-09-10T10:55:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e727fe740 20 client.15987 trim_cache size 0 max 16384 2024-09-10T10:55:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:55:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:55:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:55:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:55:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/1507071644 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3e540a2da0 msgr2=0x7f3e540a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3e540a2da0 0x7f3e540a5260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e6009bd10 tx=0x7f3e6008a040 comp rx=0 tx=0).stop 2024-09-10T10:55:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/1507071644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e68107b80 msgr2=0x7f3e6819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e68107b80 0x7f3e6819ccd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000eb10 tx=0x7f3e6000c2d0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/1507071644 shutdown_connections 2024-09-10T10:55:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f3e540a2da0 0x7f3e540a5260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e68107b80 0x7f3e6819ccd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e68102480 0x7f3e68198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 --2- 172.21.15.7:0/1507071644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3e68101ae0 0x7f3e68198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/1507071644 >> 172.21.15.7:0/1507071644 conn(0x7f3e680fd7e0 msgr2=0x7f3e680fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/1507071644 shutdown_connections 2024-09-10T10:55:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:01.013+0000 7f3e6fa5e640 1 -- 172.21.15.7:0/1507071644 wait complete. 2024-09-10T10:55:01.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 957303 10.07 MB/sec execute 361 sec latency 1016.560 ms 2024-09-10T10:55:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:01 smithi133 ceph-mon[35282]: pgmap v285: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-10T10:55:02.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:01 smithi103 ceph-mon[35160]: pgmap v285: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-10T10:55:02.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:01 smithi007 ceph-mon[30873]: pgmap v285: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-09-10T10:55:02.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 959530 10.08 MB/sec execute 362 sec latency 62.840 ms 2024-09-10T10:55:03.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 959952 10.06 MB/sec execute 363 sec latency 493.813 ms 2024-09-10T10:55:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:03 smithi133 ceph-mon[35282]: pgmap v286: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.5 MiB/s wr, 63 op/s 2024-09-10T10:55:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:03 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:55:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:03 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:55:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:03 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:55:03.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:03 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:55:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:03 smithi103 ceph-mon[35160]: pgmap v286: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.5 MiB/s wr, 63 op/s 2024-09-10T10:55:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:03 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:55:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:03 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:55:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:03 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:55:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:03 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:55:04.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:03 smithi007 ceph-mon[30873]: pgmap v286: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.5 MiB/s wr, 63 op/s 2024-09-10T10:55:04.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:03 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:55:04.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:03 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:55:04.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:03 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:55:04.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:03 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:55:04.527 INFO:tasks.workunit.client.0.smithi007.stdout: 1 959952 10.03 MB/sec execute 364 sec latency 1493.900 ms 2024-09-10T10:55:05.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:04 smithi103 ceph-mon[35160]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T10:55:05.085 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:04 smithi007 ceph-mon[30873]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T10:55:05.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:04 smithi133 ceph-mon[35282]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T10:55:05.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 959952 10.00 MB/sec execute 365 sec latency 2493.947 ms 2024-09-10T10:55:06.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:05 smithi103 ceph-mon[35160]: pgmap v288: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.8 MiB/s wr, 76 op/s 2024-09-10T10:55:06.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:05 smithi007 ceph-mon[30873]: pgmap v288: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.8 MiB/s wr, 76 op/s 2024-09-10T10:55:06.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:05 smithi133 ceph-mon[35282]: pgmap v288: 97 pgs: 97 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.8 MiB/s wr, 76 op/s 2024-09-10T10:55:06.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 960076 9.98 MB/sec execute 366 sec latency 3221.915 ms 2024-09-10T10:55:07.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 960076 9.95 MB/sec execute 367 sec latency 1183.087 ms 2024-09-10T10:55:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:07 smithi103 ceph-mon[35160]: pgmap v289: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 2.3 MiB/s wr, 146 op/s 2024-09-10T10:55:08.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:07 smithi007 ceph-mon[30873]: pgmap v289: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 2.3 MiB/s wr, 146 op/s 2024-09-10T10:55:08.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:07 smithi133 ceph-mon[35282]: pgmap v289: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 2.3 MiB/s wr, 146 op/s 2024-09-10T10:55:08.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 960076 9.92 MB/sec execute 368 sec latency 2183.203 ms 2024-09-10T10:55:09.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 960076 9.89 MB/sec execute 369 sec latency 3183.273 ms 2024-09-10T10:55:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:09 smithi103 ceph-mon[35160]: pgmap v290: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 2.3 MiB/s wr, 146 op/s 2024-09-10T10:55:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:09 smithi103 ceph-mon[35160]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T10:55:09.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:09 smithi007 ceph-mon[30873]: pgmap v290: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 2.3 MiB/s wr, 146 op/s 2024-09-10T10:55:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:09 smithi007 ceph-mon[30873]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T10:55:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:09 smithi133 ceph-mon[35282]: pgmap v290: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 2.3 MiB/s wr, 146 op/s 2024-09-10T10:55:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:09 smithi133 ceph-mon[35282]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T10:55:10.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 960076 9.87 MB/sec execute 370 sec latency 4183.367 ms 2024-09-10T10:55:11.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 960688 9.85 MB/sec execute 371 sec latency 4911.844 ms 2024-09-10T10:55:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:11 smithi103 ceph-mon[35160]: pgmap v292: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.2 MiB/s wr, 161 op/s 2024-09-10T10:55:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:11 smithi007 ceph-mon[30873]: pgmap v292: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.2 MiB/s wr, 161 op/s 2024-09-10T10:55:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:11 smithi133 ceph-mon[35282]: pgmap v292: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.2 MiB/s wr, 161 op/s 2024-09-10T10:55:12.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 963073 9.86 MB/sec execute 372 sec latency 14.728 ms 2024-09-10T10:55:13.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 964611 9.85 MB/sec execute 373 sec latency 13.940 ms 2024-09-10T10:55:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:13 smithi103 ceph-mon[35160]: pgmap v293: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-10T10:55:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:13 smithi007 ceph-mon[30873]: pgmap v293: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-10T10:55:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:13 smithi133 ceph-mon[35282]: pgmap v293: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-10T10:55:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.107+0000 7f193499f640 1 -- 172.21.15.7:0/476869373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193010c0b0 msgr2=0x7f193010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.107+0000 7f193499f640 1 --2- 172.21.15.7:0/476869373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193010c0b0 0x7f193010e550 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1928009ca0 tx=0x7f192802a950 comp rx=0 tx=0).stop 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.107+0000 7f193499f640 1 -- 172.21.15.7:0/476869373 shutdown_connections 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.107+0000 7f193499f640 1 --2- 172.21.15.7:0/476869373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193010c0b0 0x7f193010e550 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.107+0000 7f193499f640 1 --2- 172.21.15.7:0/476869373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1930103600 0x7f193010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.107+0000 7f193499f640 1 --2- 172.21.15.7:0/476869373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1930102ce0 0x7f19301030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.107+0000 7f193499f640 1 -- 172.21.15.7:0/476869373 >> 172.21.15.7:0/476869373 conn(0x7f1930077990 msgr2=0x7f19300fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.107+0000 7f193499f640 1 -- 172.21.15.7:0/476869373 shutdown_connections 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.107+0000 7f193499f640 1 -- 172.21.15.7:0/476869373 wait complete. 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f193499f640 1 Processor -- start 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f193499f640 1 -- start start 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f193499f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1930102ce0 0x7f19301982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f193499f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1930103600 0x7f1930198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f193499f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f193010c0b0 0x7f193019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f193499f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1930111ee0 con 0x7f1930102ce0 2024-09-10T10:55:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f193499f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1930111d60 con 0x7f193010c0b0 2024-09-10T10:55:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f193499f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1930112060 con 0x7f1930103600 2024-09-10T10:55:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1930102ce0 0x7f19301982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1930102ce0 0x7f19301982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41356/0 (socket says 172.21.15.7:41356) 2024-09-10T10:55:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192e59c640 1 -- 172.21.15.7:0/3293615414 learned_addr learned my addr 172.21.15.7:0/3293615414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192dd9b640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1930103600 0x7f1930198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192ed9d640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f193010c0b0 0x7f193019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192e59c640 1 -- 172.21.15.7:0/3293615414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1930103600 msgr2=0x7f1930198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192e59c640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1930103600 0x7f1930198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192e59c640 1 -- 172.21.15.7:0/3293615414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f193010c0b0 msgr2=0x7f193019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192e59c640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f193010c0b0 0x7f193019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192e59c640 1 -- 172.21.15.7:0/3293615414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193019d2e0 con 0x7f1930102ce0 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192ed9d640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f193010c0b0 0x7f193019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.108+0000 7f192e59c640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1930102ce0 0x7f19301982f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f192000ea60 tx=0x7f192000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.109+0000 7f191f7fe640 1 -- 172.21.15.7:0/3293615414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192000cde0 con 0x7f1930102ce0 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.109+0000 7f193499f640 1 -- 172.21.15.7:0/3293615414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193019d5d0 con 0x7f1930102ce0 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.109+0000 7f191f7fe640 1 -- 172.21.15.7:0/3293615414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19200035f0 con 0x7f1930102ce0 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.109+0000 7f191f7fe640 1 -- 172.21.15.7:0/3293615414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920010670 con 0x7f1930102ce0 2024-09-10T10:55:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.109+0000 7f193499f640 1 -- 172.21.15.7:0/3293615414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19301a4e60 con 0x7f1930102ce0 2024-09-10T10:55:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.111+0000 7f191f7fe640 1 -- 172.21.15.7:0/3293615414 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f19200037a0 con 0x7f1930102ce0 2024-09-10T10:55:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.111+0000 7f193781a740 1 Processor -- start 2024-09-10T10:55:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f193781a740 1 -- start start 2024-09-10T10:55:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f193781a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d129b0 0x55b481d12d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f193781a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d132d0 0x55b481d21b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f193781a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 0x55b481d244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f193781a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b481c772f0 con 0x55b481d129b0 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f193781a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b481c1f270 con 0x55b481d220c0 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f193781a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b481c732b0 con 0x55b481d132d0 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d132d0 0x55b481d21b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f191f7fe640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f19180a2da0 0x7f19180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d129b0 0x55b481d12d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d132d0 0x55b481d21b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48698/0 (socket says 172.21.15.7:48698) 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192e59c640 1 -- 172.21.15.7:0/749953096 learned_addr learned my addr 172.21.15.7:0/749953096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192dd9b640 1 --2- 172.21.15.7:0/749953096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 0x55b481d244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192ed9d640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f19180a2da0 0x7f19180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f191f7fe640 1 -- 172.21.15.7:0/3293615414 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f19200cf7e0 con 0x7f1930102ce0 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192e59c640 1 -- 172.21.15.7:0/749953096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 msgr2=0x55b481d244b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192e59c640 1 --2- 172.21.15.7:0/749953096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 0x55b481d244b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192e59c640 1 -- 172.21.15.7:0/749953096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d129b0 msgr2=0x55b481d12d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192e59c640 1 --2- 172.21.15.7:0/749953096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d129b0 0x55b481d12d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192e59c640 1 -- 172.21.15.7:0/749953096 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b481c1e340 con 0x55b481d132d0 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.112+0000 7f192dd9b640 1 --2- 172.21.15.7:0/749953096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 0x55b481d244b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.113+0000 7f192ed9d640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f19180a2da0 0x7f19180a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1928009430 tx=0x7f192802a950 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.113+0000 7f192e59c640 1 --2- 172.21.15.7:0/749953096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d132d0 0x55b481d21b80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f19200a1170 tx=0x7f1920003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cee1df5d62b4317 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.113+0000 7f191cff9640 1 -- 172.21.15.7:0/749953096 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192009bcd0 con 0x55b481d132d0 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.113+0000 7f191cff9640 1 -- 172.21.15.7:0/749953096 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19200a0080 con 0x55b481d132d0 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.113+0000 7f191cff9640 1 -- 172.21.15.7:0/749953096 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19200a02e0 con 0x55b481d132d0 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 -- 172.21.15.7:0/749953096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d132d0 msgr2=0x55b481d21b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 --2- 172.21.15.7:0/749953096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d132d0 0x55b481d21b80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f19200a1170 tx=0x7f1920003f30 comp rx=0 tx=0).stop 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 -- 172.21.15.7:0/749953096 shutdown_connections 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 --2- 172.21.15.7:0/749953096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 0x55b481d244b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 --2- 172.21.15.7:0/749953096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d132d0 0x55b481d21b80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 --2- 172.21.15.7:0/749953096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d129b0 0x55b481d12d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 -- 172.21.15.7:0/749953096 >> 172.21.15.7:0/749953096 conn(0x55b481d0f060 msgr2=0x55b481d0f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 -- 172.21.15.7:0/749953096 shutdown_connections 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 -- 172.21.15.7:0/749953096 wait complete. 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 -- start start 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 Processor -- start 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 0x55b481d1f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d132d0 0x55b481d1f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d220c0 0x55b481d1fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b481d24c90 con 0x55b481d220c0 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b481c00290 con 0x55b481d132d0 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b481d24e10 con 0x55b481d129b0 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.114+0000 7f193781a740 10 client.0 _ll_register_callbacks cb 0x55b481c31140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d220c0 0x55b481d1fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 0x55b481d1f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d220c0 0x55b481d1fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41378/0 (socket says 172.21.15.7:41378) 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 0x55b481d1f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48708/0 (socket says 172.21.15.7:48708) 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192ed9d640 1 -- 172.21.15.7:0/3320700408 learned_addr learned my addr 172.21.15.7:0/3320700408 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192dd9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d132d0 0x55b481d1f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192ed9d640 1 -- 172.21.15.7:0/3320700408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 msgr2=0x55b481d1f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192ed9d640 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 0x55b481d1f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192ed9d640 1 -- 172.21.15.7:0/3320700408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d132d0 msgr2=0x55b481d1f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192ed9d640 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d132d0 0x55b481d1f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192ed9d640 1 -- 172.21.15.7:0/3320700408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b48198a210 con 0x55b481d220c0 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192e59c640 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 0x55b481d1f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f192ed9d640 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d220c0 0x55b481d1fb30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1928035000 tx=0x7f19280095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c621b69c89787b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f1915ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f1915ffb640 1 -- 172.21.15.7:0/3320700408 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928050070 con 0x55b481d220c0 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f1915ffb640 1 -- 172.21.15.7:0/3320700408 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f192803abf0 con 0x55b481d220c0 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f193781a740 1 -- 172.21.15.7:0/3320700408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19200a0080 con 0x55b481d220c0 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f1915ffb640 1 -- 172.21.15.7:0/3320700408 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1928034040 con 0x55b481d220c0 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f193781a740 10 client.15999 fetch_fsmap learned FSMap version 39 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.115+0000 7f193781a740 1 -- 172.21.15.7:0/3320700408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b481a56620 con 0x55b481d220c0 2024-09-10T10:55:14.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f1915ffb640 1 -- 172.21.15.7:0/3320700408 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19200a0080 con 0x55b481d220c0 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f1915ffb640 1 -- 172.21.15.7:0/3320700408 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f1928049040 con 0x55b481d220c0 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f193781a740 10 client.15999 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f193781a740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f193781a740 20 client.15999 populate_metadata read hostname 'smithi007' 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f193781a740 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b481d32190 0x55b481d345a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f193781a740 4 client.15999 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f193781a740 1 -- 172.21.15.7:0/3320700408 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b481c84a20 con 0x55b481d32190 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f192e59c640 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b481d32190 0x55b481d345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f192e59c640 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b481d32190 0x55b481d345a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1939f30af21f3db5 server_cookie=abe8b292c4a7535f in_seq=0 out_seq=0 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.116+0000 7f1915ffb640 10 client.15999 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f1915ffb640 1 -- 172.21.15.7:0/3320700408 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b481c84a20 con 0x55b481d32190 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f1915ffb640 10 client.15999 handle_command_reply: tid=0 2024-09-10T10:55:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 client.15999 shutdown 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 -- 172.21.15.7:0/3320700408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d220c0 msgr2=0x55b481d1fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d220c0 0x55b481d1fb30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1928035000 tx=0x7f19280095c0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 -- 172.21.15.7:0/3320700408 shutdown_connections 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b481d32190 0x55b481d345a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d220c0 0x55b481d1fb30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d132d0 0x55b481d1f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f1915ffb640 0 client.15999 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 --2- 172.21.15.7:0/3320700408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 0x55b481d1f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 -- 172.21.15.7:0/3320700408 >> 172.21.15.7:0/3320700408 conn(0x55b481d0f060 msgr2=0x55b481d14dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 -- 172.21.15.7:0/3320700408 shutdown_connections 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.117+0000 7f193781a740 1 -- 172.21.15.7:0/3320700408 wait complete. 2024-09-10T10:55:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.118+0000 7f193781a740 20 client.15999 trim_cache size 0 max 16384 2024-09-10T10:55:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f193781a740 1 Processor -- start 2024-09-10T10:55:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f193781a740 1 -- start start 2024-09-10T10:55:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f193781a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 0x55b481d1f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f193781a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d39eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f193781a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 0x55b481d3a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f193781a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f19301adfd0 con 0x55b481d132d0 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f193781a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19301ab810 con 0x55b481d220c0 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f193781a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f19301a9680 con 0x55b481d129b0 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f192dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d39eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f192dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d39eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41382/0 (socket says 172.21.15.7:41382) 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f192dd9b640 1 -- 172.21.15.7:0/309162646 learned_addr learned my addr 172.21.15.7:0/309162646 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f192e59c640 1 --2- 172.21.15.7:0/309162646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 0x55b481d1f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.133+0000 7f192ed9d640 1 --2- 172.21.15.7:0/309162646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 0x55b481d3a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f192dd9b640 1 -- 172.21.15.7:0/309162646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 msgr2=0x55b481d1f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f192dd9b640 1 --2- 172.21.15.7:0/309162646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 0x55b481d1f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f192dd9b640 1 -- 172.21.15.7:0/309162646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 msgr2=0x55b481d3a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f192dd9b640 1 --2- 172.21.15.7:0/309162646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 0x55b481d3a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f192dd9b640 1 -- 172.21.15.7:0/309162646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b481d1d600 con 0x55b481d132d0 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f192dd9b640 1 --2- 172.21.15.7:0/309162646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d39eb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1924009c00 tx=0x7f1924005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d788a36f392db40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f1917fff640 1 -- 172.21.15.7:0/309162646 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924022070 con 0x55b481d132d0 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f1917fff640 1 -- 172.21.15.7:0/309162646 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19240120d0 con 0x55b481d132d0 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f1917fff640 1 -- 172.21.15.7:0/309162646 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924003de0 con 0x55b481d132d0 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f193781a740 1 -- 172.21.15.7:0/309162646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 msgr2=0x55b481d39eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.134+0000 7f193781a740 1 --2- 172.21.15.7:0/309162646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d39eb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1924009c00 tx=0x7f1924005f90 comp rx=0 tx=0).stop 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 -- 172.21.15.7:0/309162646 shutdown_connections 2024-09-10T10:55:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 --2- 172.21.15.7:0/309162646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d220c0 0x55b481d3a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 --2- 172.21.15.7:0/309162646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d39eb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 --2- 172.21.15.7:0/309162646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d129b0 0x55b481d1f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 -- 172.21.15.7:0/309162646 >> 172.21.15.7:0/309162646 conn(0x55b481d0f060 msgr2=0x55b481d1f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 -- 172.21.15.7:0/309162646 shutdown_connections 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 -- 172.21.15.7:0/309162646 wait complete. 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 -- start start 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 Processor -- start 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d129b0 0x55b481d2a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d1bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d220c0 0x55b481d1c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f19301a9cd0 con 0x55b481d132d0 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19301ab140 con 0x55b481d129b0 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f19301aa3a0 con 0x55b481d220c0 2024-09-10T10:55:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.135+0000 7f193781a740 10 client.0 _ll_register_callbacks cb 0x7f1924005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d1bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d129b0 0x55b481d2a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192e59c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d129b0 0x55b481d2a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:57714/0 (socket says 172.21.15.7:57714) 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192e59c640 1 -- 172.21.15.7:0/2545725431 learned_addr learned my addr 172.21.15.7:0/2545725431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d1bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41398/0 (socket says 172.21.15.7:41398) 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192ed9d640 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d220c0 0x55b481d1c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192e59c640 1 -- 172.21.15.7:0/2545725431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d220c0 msgr2=0x55b481d1c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192e59c640 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d220c0 0x55b481d1c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192e59c640 1 -- 172.21.15.7:0/2545725431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 msgr2=0x55b481d1bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192e59c640 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d1bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192e59c640 1 -- 172.21.15.7:0/2545725431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b481d9be80 con 0x55b481d129b0 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192dd9b640 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d1bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f192e59c640 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d129b0 0x55b481d2a970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f19200ccfb0 tx=0x7f192009bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd7151b43fca28b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f1915ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f1915ffb640 1 -- 172.21.15.7:0/2545725431 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192009b630 con 0x55b481d129b0 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f1915ffb640 1 -- 172.21.15.7:0/2545725431 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f1920099fd0 con 0x55b481d129b0 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f193781a740 1 -- 172.21.15.7:0/2545725431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19240120d0 con 0x55b481d129b0 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f1915ffb640 1 -- 172.21.15.7:0/2545725431 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f19200a0080 con 0x55b481d129b0 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f193781a740 10 client.25658 fetch_fsmap learned FSMap version 39 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.136+0000 7f193781a740 1 -- 172.21.15.7:0/2545725431 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b481d9d070 con 0x55b481d129b0 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f1915ffb640 1 -- 172.21.15.7:0/2545725431 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19240120d0 con 0x55b481d129b0 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f1915ffb640 1 -- 172.21.15.7:0/2545725431 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f192007aaf0 con 0x55b481d129b0 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f193781a740 10 client.25658 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:55:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f193781a740 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:55:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f193781a740 20 client.25658 populate_metadata read hostname 'smithi007' 2024-09-10T10:55:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f193781a740 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b481d9d410 0x55b481d9f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f193781a740 4 client.25658 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:55:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f193781a740 1 -- 172.21.15.7:0/2545725431 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b481c7ed70 con 0x55b481d9d410 2024-09-10T10:55:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f192dd9b640 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b481d9d410 0x55b481d9f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f1915ffb640 10 client.25658 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.137+0000 7f192dd9b640 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b481d9d410 0x55b481d9f800 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a554b12622af49 server_cookie=6bb7345462be14a9 in_seq=0 out_seq=0 2024-09-10T10:55:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f1915ffb640 1 -- 172.21.15.7:0/2545725431 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b481c7ed70 con 0x55b481d9d410 2024-09-10T10:55:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f1915ffb640 10 client.25658 handle_command_reply: tid=0 2024-09-10T10:55:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f193781a740 1 client.25658 shutdown 2024-09-10T10:55:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f193781a740 1 -- 172.21.15.7:0/2545725431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d129b0 msgr2=0x55b481d2a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f193781a740 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d129b0 0x55b481d2a970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f19200ccfb0 tx=0x7f192009bb20 comp rx=0 tx=0).stop 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f193781a740 1 -- 172.21.15.7:0/2545725431 shutdown_connections 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f193781a740 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b481d9d410 0x55b481d9f800 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f193781a740 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b481d220c0 0x55b481d1c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f193781a740 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b481d132d0 0x55b481d1bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f1915ffb640 0 client.25658 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f193781a740 1 --2- 172.21.15.7:0/2545725431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b481d129b0 0x55b481d2a970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.179+0000 7f193781a740 1 -- 172.21.15.7:0/2545725431 >> 172.21.15.7:0/2545725431 conn(0x55b481d0f060 msgr2=0x55b481d1f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193781a740 1 -- 172.21.15.7:0/2545725431 shutdown_connections 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193781a740 1 -- 172.21.15.7:0/2545725431 wait complete. 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193781a740 20 client.25658 trim_cache size 0 max 16384 2024-09-10T10:55:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:55:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:55:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:55:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:55:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193499f640 1 -- 172.21.15.7:0/3293615414 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f19180a2da0 msgr2=0x7f19180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193499f640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f19180a2da0 0x7f19180a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1928009430 tx=0x7f192802a950 comp rx=0 tx=0).stop 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193499f640 1 -- 172.21.15.7:0/3293615414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1930102ce0 msgr2=0x7f19301982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193499f640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1930102ce0 0x7f19301982f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f192000ea60 tx=0x7f192000ef20 comp rx=0 tx=0).stop 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193499f640 1 -- 172.21.15.7:0/3293615414 shutdown_connections 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193499f640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f193010c0b0 0x7f193019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193499f640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1930103600 0x7f1930198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193499f640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f19180a2da0 0x7f19180a5260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193499f640 1 --2- 172.21.15.7:0/3293615414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1930102ce0 0x7f19301982f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.180+0000 7f193499f640 1 -- 172.21.15.7:0/3293615414 >> 172.21.15.7:0/3293615414 conn(0x7f1930077990 msgr2=0x7f19300fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.181+0000 7f193499f640 1 -- 172.21.15.7:0/3293615414 shutdown_connections 2024-09-10T10:55:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:14.181+0000 7f193499f640 1 -- 172.21.15.7:0/3293615414 wait complete. 2024-09-10T10:55:14.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 966802 9.86 MB/sec execute 374 sec latency 27.636 ms 2024-09-10T10:55:15.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 967099 9.84 MB/sec execute 375 sec latency 788.765 ms 2024-09-10T10:55:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:15 smithi103 ceph-mon[35160]: pgmap v294: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-09-10T10:55:15.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:15 smithi007 ceph-mon[30873]: pgmap v294: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-09-10T10:55:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:15 smithi133 ceph-mon[35282]: pgmap v294: 97 pgs: 97 active+clean; 264 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.7 MiB/s wr, 129 op/s 2024-09-10T10:55:16.528 INFO:tasks.workunit.client.0.smithi007.stdout: 1 967167 9.81 MB/sec execute 376 sec latency 1516.376 ms 2024-09-10T10:55:17.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 967167 9.79 MB/sec execute 377 sec latency 1225.665 ms 2024-09-10T10:55:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:17 smithi103 ceph-mon[35160]: pgmap v295: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-10T10:55:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:17 smithi007 ceph-mon[30873]: pgmap v295: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-10T10:55:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:17 smithi133 ceph-mon[35282]: pgmap v295: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-10T10:55:18.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 967167 9.76 MB/sec execute 378 sec latency 2225.790 ms 2024-09-10T10:55:19.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 967167 9.73 MB/sec execute 379 sec latency 3225.912 ms 2024-09-10T10:55:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:19 smithi103 ceph-mon[35160]: pgmap v296: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-10T10:55:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:19 smithi007 ceph-mon[30873]: pgmap v296: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-10T10:55:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:19 smithi133 ceph-mon[35282]: pgmap v296: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-09-10T10:55:20.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 967167 9.71 MB/sec execute 380 sec latency 4226.032 ms 2024-09-10T10:55:21.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 967788 9.70 MB/sec execute 381 sec latency 4950.772 ms 2024-09-10T10:55:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:21 smithi103 ceph-mon[35160]: pgmap v297: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-10T10:55:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:21 smithi007 ceph-mon[30873]: pgmap v297: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-10T10:55:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:21 smithi133 ceph-mon[35282]: pgmap v297: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-10T10:55:22.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 970202 9.71 MB/sec execute 382 sec latency 13.172 ms 2024-09-10T10:55:23.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 970715 9.68 MB/sec execute 383 sec latency 567.568 ms 2024-09-10T10:55:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:23 smithi103 ceph-mon[35160]: pgmap v298: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 753 KiB/s wr, 97 op/s 2024-09-10T10:55:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:23 smithi007 ceph-mon[30873]: pgmap v298: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 753 KiB/s wr, 97 op/s 2024-09-10T10:55:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:23 smithi133 ceph-mon[35282]: pgmap v298: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 753 KiB/s wr, 97 op/s 2024-09-10T10:55:24.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 970715 9.66 MB/sec execute 384 sec latency 1567.667 ms 2024-09-10T10:55:25.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 970715 9.63 MB/sec execute 385 sec latency 2567.785 ms 2024-09-10T10:55:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:25 smithi103 ceph-mon[35160]: pgmap v299: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 753 KiB/s wr, 97 op/s 2024-09-10T10:55:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:25 smithi007 ceph-mon[30873]: pgmap v299: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 753 KiB/s wr, 97 op/s 2024-09-10T10:55:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:25 smithi133 ceph-mon[35282]: pgmap v299: 97 pgs: 97 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 753 KiB/s wr, 97 op/s 2024-09-10T10:55:26.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 971309 9.62 MB/sec execute 386 sec latency 3292.870 ms 2024-09-10T10:55:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.271+0000 7f76f64af640 1 -- 172.21.15.7:0/3190476593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76f01035e0 msgr2=0x7f76f010bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.271+0000 7f76f64af640 1 --2- 172.21.15.7:0/3190476593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76f01035e0 0x7f76f010bac0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009900 tx=0x7f76e402a480 comp rx=0 tx=0).stop 2024-09-10T10:55:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.271+0000 7f76f64af640 1 -- 172.21.15.7:0/3190476593 shutdown_connections 2024-09-10T10:55:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.271+0000 7f76f64af640 1 --2- 172.21.15.7:0/3190476593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f010c110 0x7f76f010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.271+0000 7f76f64af640 1 --2- 172.21.15.7:0/3190476593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76f01035e0 0x7f76f010bac0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.271+0000 7f76f64af640 1 --2- 172.21.15.7:0/3190476593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f0102cc0 0x7f76f01030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.271+0000 7f76f64af640 1 -- 172.21.15.7:0/3190476593 >> 172.21.15.7:0/3190476593 conn(0x7f76f00fc6b0 msgr2=0x7f76f00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.271+0000 7f76f64af640 1 -- 172.21.15.7:0/3190476593 shutdown_connections 2024-09-10T10:55:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.271+0000 7f76f64af640 1 -- 172.21.15.7:0/3190476593 wait complete. 2024-09-10T10:55:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f64af640 1 Processor -- start 2024-09-10T10:55:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f64af640 1 -- start start 2024-09-10T10:55:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f64af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f0102cc0 0x7f76f0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f64af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f01035e0 0x7f76f01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f64af640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76f010c110 0x7f76f019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f64af640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76f0111ef0 con 0x7f76f010c110 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f64af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76f0111d70 con 0x7f76f01035e0 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f64af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76f0112070 con 0x7f76f0102cc0 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f4a4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76f010c110 0x7f76f019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76effff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f0102cc0 0x7f76f0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f4a4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76f010c110 0x7f76f019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45434/0 (socket says 172.21.15.7:45434) 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76f4a4c640 1 -- 172.21.15.7:0/1450404142 learned_addr learned my addr 172.21.15.7:0/1450404142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76effff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f0102cc0 0x7f76f0198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:49866/0 (socket says 172.21.15.7:49866) 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.272+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f01035e0 0x7f76f01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76f4a4c640 1 -- 172.21.15.7:0/1450404142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f0102cc0 msgr2=0x7f76f0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76f4a4c640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f0102cc0 0x7f76f0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76f4a4c640 1 -- 172.21.15.7:0/1450404142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f01035e0 msgr2=0x7f76f01988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76f4a4c640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f01035e0 0x7f76f01988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76f4a4c640 1 -- 172.21.15.7:0/1450404142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f019d370 con 0x7f76f010c110 2024-09-10T10:55:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76effff640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f0102cc0 0x7f76f0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f01035e0 0x7f76f01988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76f4a4c640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76f010c110 0x7f76f019cc40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f76e800eb10 tx=0x7f76e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76ed7fa640 1 -- 172.21.15.7:0/1450404142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8004b20 con 0x7f76f010c110 2024-09-10T10:55:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76ed7fa640 1 -- 172.21.15.7:0/1450404142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e8004cc0 con 0x7f76f010c110 2024-09-10T10:55:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76f64af640 1 -- 172.21.15.7:0/1450404142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f019d660 con 0x7f76f010c110 2024-09-10T10:55:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76ed7fa640 1 -- 172.21.15.7:0/1450404142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8010760 con 0x7f76f010c110 2024-09-10T10:55:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.273+0000 7f76f64af640 1 -- 172.21.15.7:0/1450404142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f01a4ef0 con 0x7f76f010c110 2024-09-10T10:55:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.275+0000 7f76ed7fa640 1 -- 172.21.15.7:0/1450404142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f76e8005020 con 0x7f76f010c110 2024-09-10T10:55:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76f923e740 1 Processor -- start 2024-09-10T10:55:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76f923e740 1 -- start start 2024-09-10T10:55:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76f923e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4aefdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76ed7fa640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f76d80a2e20 0x7f76d80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76f923e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af0cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76f923e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af0f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76f923e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557d4ae622f0 con 0x557d4af0d0c0 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76f923e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d4ae0a270 con 0x557d4aefd9b0 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76f923e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557d4ae5e2b0 con 0x557d4aefe2d0 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76f4a4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af0f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76effff640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f76d80a2e20 0x7f76d80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76ef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4aefdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76ef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4aefdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:56580/0 (socket says 172.21.15.7:56580) 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.276+0000 7f76ef7fe640 1 -- 172.21.15.7:0/1851395755 learned_addr learned my addr 172.21.15.7:0/1851395755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76ed7fa640 1 -- 172.21.15.7:0/1450404142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f76e80cf910 con 0x7f76f010c110 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/1851395755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af0cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76f4a4c640 1 -- 172.21.15.7:0/1851395755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 msgr2=0x557d4af0cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76f4a4c640 1 --2- 172.21.15.7:0/1851395755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af0cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76f4a4c640 1 -- 172.21.15.7:0/1851395755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 msgr2=0x557d4aefdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76f4a4c640 1 --2- 172.21.15.7:0/1851395755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4aefdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76f4a4c640 1 -- 172.21.15.7:0/1851395755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4ae09340 con 0x557d4af0d0c0 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/1851395755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4aefdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76effff640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f76d80a2e20 0x7f76d80a52e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009fc0 tx=0x7f76e0005f90 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76f4a4c640 1 --2- 172.21.15.7:0/1851395755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af0f4b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f76e80cd030 tx=0x7f76e801c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e535155e6e207c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76d67fc640 1 -- 172.21.15.7:0/1851395755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e809b940 con 0x557d4af0d0c0 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76d67fc640 1 -- 172.21.15.7:0/1851395755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e809c500 con 0x557d4af0d0c0 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.277+0000 7f76d67fc640 1 -- 172.21.15.7:0/1851395755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8003970 con 0x557d4af0d0c0 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.278+0000 7f76f923e740 1 -- 172.21.15.7:0/1851395755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 msgr2=0x557d4af0f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.278+0000 7f76f923e740 1 --2- 172.21.15.7:0/1851395755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af0f4b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f76e80cd030 tx=0x7f76e801c590 comp rx=0 tx=0).stop 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.278+0000 7f76f923e740 1 -- 172.21.15.7:0/1851395755 shutdown_connections 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.278+0000 7f76f923e740 1 --2- 172.21.15.7:0/1851395755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af0f4b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.278+0000 7f76f923e740 1 --2- 172.21.15.7:0/1851395755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af0cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.278+0000 7f76f923e740 1 --2- 172.21.15.7:0/1851395755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4aefdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.278+0000 7f76f923e740 1 -- 172.21.15.7:0/1851395755 >> 172.21.15.7:0/1851395755 conn(0x557d4aefa060 msgr2=0x557d4aefa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.278+0000 7f76f923e740 1 -- 172.21.15.7:0/1851395755 shutdown_connections 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.278+0000 7f76f923e740 1 -- 172.21.15.7:0/1851395755 wait complete. 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.278+0000 7f76f923e740 1 -- start start 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76f923e740 1 Processor -- start 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76f923e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefd9b0 0x557d4af0a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76f923e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af0a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76f923e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4af0d0c0 0x557d4af0ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76f923e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557d4af0fc90 con 0x557d4aefd9b0 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76f923e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d4adeb290 con 0x557d4af0d0c0 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76f923e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557d4af0fe10 con 0x557d4aefe2d0 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76f923e740 10 client.0 _ll_register_callbacks cb 0x557d4ae1c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76effff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefd9b0 0x557d4af0a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76effff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefd9b0 0x557d4af0a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45450/0 (socket says 172.21.15.7:45450) 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76effff640 1 -- 172.21.15.7:0/883128128 learned_addr learned my addr 172.21.15.7:0/883128128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af0a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76f4a4c640 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4af0d0c0 0x557d4af0ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76effff640 1 -- 172.21.15.7:0/883128128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 msgr2=0x557d4af0a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76effff640 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af0a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76effff640 1 -- 172.21.15.7:0/883128128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4af0d0c0 msgr2=0x557d4af0ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76effff640 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4af0d0c0 0x557d4af0ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76effff640 1 -- 172.21.15.7:0/883128128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4ab75210 con 0x557d4aefd9b0 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76effff640 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefd9b0 0x557d4af0a0a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f76e0004650 tx=0x7f76e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ff32ed32d4fbd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.279+0000 7f76cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:55:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76cf7fe640 1 -- 172.21.15.7:0/883128128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e000f840 con 0x557d4aefd9b0 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76cf7fe640 1 -- 172.21.15.7:0/883128128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f76e000f9e0 con 0x557d4aefd9b0 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76f923e740 1 -- 172.21.15.7:0/883128128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e809c500 con 0x557d4aefd9b0 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76cf7fe640 1 -- 172.21.15.7:0/883128128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f76e000fbe0 con 0x557d4aefd9b0 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76f923e740 10 client.16017 fetch_fsmap learned FSMap version 39 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76f923e740 1 -- 172.21.15.7:0/883128128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557d4ac41620 con 0x557d4aefd9b0 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76cf7fe640 1 -- 172.21.15.7:0/883128128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e809c500 con 0x557d4aefd9b0 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76cf7fe640 1 -- 172.21.15.7:0/883128128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f76e001a070 con 0x557d4aefd9b0 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76f923e740 10 client.16017 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76f923e740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76f923e740 20 client.16017 populate_metadata read hostname 'smithi007' 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76f923e740 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557d4af1d160 0x557d4af1f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76f923e740 4 client.16017 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76f923e740 1 -- 172.21.15.7:0/883128128 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d4ae6fa20 con 0x557d4af1d160 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.280+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557d4af1d160 0x557d4af1f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.281+0000 7f76cf7fe640 10 client.16017 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.281+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557d4af1d160 0x557d4af1f570 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4bac2478f6a29dd server_cookie=c10aca5c52aee918 in_seq=0 out_seq=0 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.281+0000 7f76cf7fe640 1 -- 172.21.15.7:0/883128128 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d4ae6fa20 con 0x557d4af1d160 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.281+0000 7f76cf7fe640 10 client.16017 handle_command_reply: tid=0 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.281+0000 7f76f923e740 1 client.16017 shutdown 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 1 -- 172.21.15.7:0/883128128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefd9b0 msgr2=0x557d4af0a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefd9b0 0x557d4af0a0a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f76e0004650 tx=0x7f76e0004730 comp rx=0 tx=0).stop 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 1 -- 172.21.15.7:0/883128128 shutdown_connections 2024-09-10T10:55:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557d4af1d160 0x557d4af1f570 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4af0d0c0 0x557d4af0ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af0a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76cf7fe640 0 client.16017 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 1 --2- 172.21.15.7:0/883128128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefd9b0 0x557d4af0a0a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 1 -- 172.21.15.7:0/883128128 >> 172.21.15.7:0/883128128 conn(0x557d4aefa060 msgr2=0x557d4aeffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 1 -- 172.21.15.7:0/883128128 shutdown_connections 2024-09-10T10:55:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 1 -- 172.21.15.7:0/883128128 wait complete. 2024-09-10T10:55:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.282+0000 7f76f923e740 20 client.16017 trim_cache size 0 max 16384 2024-09-10T10:55:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.296+0000 7f76f923e740 1 Processor -- start 2024-09-10T10:55:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.296+0000 7f76f923e740 1 -- start start 2024-09-10T10:55:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76f923e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4af0a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76f923e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefe2d0 0x557d4af24e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76f923e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4af0d0c0 0x557d4af253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76f923e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01ae010 con 0x557d4aefe2d0 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76f923e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01ab850 con 0x557d4aefd9b0 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76f923e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01a96c0 con 0x557d4af0d0c0 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefe2d0 0x557d4af24e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefe2d0 0x557d4af24e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45452/0 (socket says 172.21.15.7:45452) 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76ef7fe640 1 -- 172.21.15.7:0/2664244309 learned_addr learned my addr 172.21.15.7:0/2664244309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76effff640 1 --2- 172.21.15.7:0/2664244309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4af0a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76f4a4c640 1 --2- 172.21.15.7:0/2664244309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4af0d0c0 0x557d4af253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76ef7fe640 1 -- 172.21.15.7:0/2664244309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4af0d0c0 msgr2=0x557d4af253c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/2664244309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4af0d0c0 0x557d4af253c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76ef7fe640 1 -- 172.21.15.7:0/2664244309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 msgr2=0x557d4af0a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/2664244309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4af0a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76ef7fe640 1 -- 172.21.15.7:0/2664244309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4aefe100 con 0x557d4aefe2d0 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76effff640 1 --2- 172.21.15.7:0/2664244309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4af0a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/2664244309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefe2d0 0x557d4af24e80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f76e402a9a0 tx=0x7f76e40314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68dda0e5b9a42bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.297+0000 7f76d5ffb640 1 -- 172.21.15.7:0/2664244309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4051070 con 0x557d4aefe2d0 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76d5ffb640 1 -- 172.21.15.7:0/2664244309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e4004300 con 0x557d4aefe2d0 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76d5ffb640 1 -- 172.21.15.7:0/2664244309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4035040 con 0x557d4aefe2d0 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76f923e740 1 -- 172.21.15.7:0/2664244309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefe2d0 msgr2=0x557d4af24e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76f923e740 1 --2- 172.21.15.7:0/2664244309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefe2d0 0x557d4af24e80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f76e402a9a0 tx=0x7f76e40314e0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76f923e740 1 -- 172.21.15.7:0/2664244309 shutdown_connections 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76f923e740 1 --2- 172.21.15.7:0/2664244309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4af0d0c0 0x557d4af253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76f923e740 1 --2- 172.21.15.7:0/2664244309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4aefe2d0 0x557d4af24e80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76f923e740 1 --2- 172.21.15.7:0/2664244309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4af0a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76f923e740 1 -- 172.21.15.7:0/2664244309 >> 172.21.15.7:0/2664244309 conn(0x557d4aefa060 msgr2=0x557d4af0c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76f923e740 1 -- 172.21.15.7:0/2664244309 shutdown_connections 2024-09-10T10:55:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.298+0000 7f76f923e740 1 -- 172.21.15.7:0/2664244309 wait complete. 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f923e740 1 -- start start 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f923e740 1 Processor -- start 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f923e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4af10eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f923e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af113f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f923e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af11930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f923e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01a9d10 con 0x557d4af0d0c0 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f923e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01ab180 con 0x557d4aefd9b0 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f923e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76f01aa3e0 con 0x557d4aefe2d0 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f923e740 10 client.0 _ll_register_callbacks cb 0x7f76e0004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f4a4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af11930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f4a4c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af11930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45462/0 (socket says 172.21.15.7:45462) 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76f4a4c640 1 -- 172.21.15.7:0/995114052 learned_addr learned my addr 172.21.15.7:0/995114052 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76effff640 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4af10eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.299+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af113f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76effff640 1 -- 172.21.15.7:0/995114052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 msgr2=0x557d4af113f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76effff640 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af113f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76effff640 1 -- 172.21.15.7:0/995114052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 msgr2=0x557d4af11930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76effff640 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af11930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76effff640 1 -- 172.21.15.7:0/995114052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4af86f20 con 0x557d4aefd9b0 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76f4a4c640 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af11930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76effff640 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4af10eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f76e00059f0 tx=0x7f76e000ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9835aa89f5ca5190 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:55:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76cf7fe640 1 -- 172.21.15.7:0/995114052 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e000f640 con 0x557d4aefd9b0 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76cf7fe640 1 -- 172.21.15.7:0/995114052 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f76e000f7e0 con 0x557d4aefd9b0 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76f923e740 1 -- 172.21.15.7:0/995114052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e4004300 con 0x557d4aefd9b0 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76cf7fe640 1 -- 172.21.15.7:0/995114052 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f76e003e050 con 0x557d4aefd9b0 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76f923e740 10 client.25676 fetch_fsmap learned FSMap version 39 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76f923e740 1 -- 172.21.15.7:0/995114052 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557d4af88210 con 0x557d4aefd9b0 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.300+0000 7f76cf7fe640 1 -- 172.21.15.7:0/995114052 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e4004300 con 0x557d4aefd9b0 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.301+0000 7f76cf7fe640 1 -- 172.21.15.7:0/995114052 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f76e001a070 con 0x557d4aefd9b0 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.301+0000 7f76f923e740 10 client.25676 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.301+0000 7f76f923e740 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.301+0000 7f76f923e740 20 client.25676 populate_metadata read hostname 'smithi007' 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.301+0000 7f76f923e740 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557d4af88990 0x557d4af8adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.301+0000 7f76f923e740 4 client.25676 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:55:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.301+0000 7f76f923e740 1 -- 172.21.15.7:0/995114052 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d4ae69d70 con 0x557d4af88990 2024-09-10T10:55:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.301+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557d4af88990 0x557d4af8adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.301+0000 7f76cf7fe640 10 client.25676 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.301+0000 7f76ef7fe640 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557d4af88990 0x557d4af8adb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecfba35ec55a5a09 server_cookie=3f52d21c2f7615fe in_seq=0 out_seq=0 2024-09-10T10:55:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.342+0000 7f76cf7fe640 1 -- 172.21.15.7:0/995114052 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d4ae69d70 con 0x557d4af88990 2024-09-10T10:55:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.342+0000 7f76cf7fe640 10 client.25676 handle_command_reply: tid=0 2024-09-10T10:55:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.342+0000 7f76f923e740 1 client.25676 shutdown 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 1 -- 172.21.15.7:0/995114052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 msgr2=0x557d4af10eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4af10eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f76e00059f0 tx=0x7f76e000ff90 comp rx=0 tx=0).stop 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 1 -- 172.21.15.7:0/995114052 shutdown_connections 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557d4af88990 0x557d4af8adb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557d4af0d0c0 0x557d4af11930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76cf7fe640 0 client.25676 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557d4aefe2d0 0x557d4af113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 1 --2- 172.21.15.7:0/995114052 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d4aefd9b0 0x557d4af10eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 1 -- 172.21.15.7:0/995114052 >> 172.21.15.7:0/995114052 conn(0x557d4aefa060 msgr2=0x557d4af0c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 1 -- 172.21.15.7:0/995114052 shutdown_connections 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 1 -- 172.21.15.7:0/995114052 wait complete. 2024-09-10T10:55:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.343+0000 7f76f923e740 20 client.25676 trim_cache size 0 max 16384 2024-09-10T10:55:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:55:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:55:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:55:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 -- 172.21.15.7:0/1450404142 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f76d80a2e20 msgr2=0x7f76d80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f76d80a2e20 0x7f76d80a52e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009fc0 tx=0x7f76e0005f90 comp rx=0 tx=0).stop 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 -- 172.21.15.7:0/1450404142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76f010c110 msgr2=0x7f76f019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76f010c110 0x7f76f019cc40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f76e800eb10 tx=0x7f76e800c2d0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 -- 172.21.15.7:0/1450404142 shutdown_connections 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f76d80a2e20 0x7f76d80a52e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76f010c110 0x7f76f019cc40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f01035e0 0x7f76f01988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 --2- 172.21.15.7:0/1450404142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f0102cc0 0x7f76f0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 -- 172.21.15.7:0/1450404142 >> 172.21.15.7:0/1450404142 conn(0x7f76f00fc6b0 msgr2=0x7f76f00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 -- 172.21.15.7:0/1450404142 shutdown_connections 2024-09-10T10:55:27.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:27.344+0000 7f76f64af640 1 -- 172.21.15.7:0/1450404142 wait complete. 2024-09-10T10:55:27.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 973700 9.63 MB/sec execute 387 sec latency 18.826 ms 2024-09-10T10:55:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:27 smithi103 ceph-mon[35160]: pgmap v300: 97 pgs: 97 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-09-10T10:55:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:27 smithi007 ceph-mon[30873]: pgmap v300: 97 pgs: 97 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-09-10T10:55:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:27 smithi133 ceph-mon[35282]: pgmap v300: 97 pgs: 97 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-09-10T10:55:28.529 INFO:tasks.workunit.client.0.smithi007.stdout: 1 974135 9.61 MB/sec execute 388 sec latency 579.210 ms 2024-09-10T10:55:29.530 INFO:tasks.workunit.client.0.smithi007.stdout: 1 974135 9.58 MB/sec execute 389 sec latency 1579.337 ms 2024-09-10T10:55:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:29 smithi103 ceph-mon[35160]: pgmap v301: 97 pgs: 97 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 359 KiB/s wr, 46 op/s 2024-09-10T10:55:29.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:29 smithi007 ceph-mon[30873]: pgmap v301: 97 pgs: 97 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 359 KiB/s wr, 46 op/s 2024-09-10T10:55:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:29 smithi133 ceph-mon[35282]: pgmap v301: 97 pgs: 97 active+clean; 270 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 359 KiB/s wr, 46 op/s 2024-09-10T10:55:30.530 INFO:tasks.workunit.client.0.smithi007.stdout: 1 974135 9.56 MB/sec execute 390 sec latency 2579.442 ms 2024-09-10T10:55:31.530 INFO:tasks.workunit.client.0.smithi007.stdout: 1 974191 9.53 MB/sec execute 391 sec latency 3304.775 ms 2024-09-10T10:55:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:31 smithi103 ceph-mon[35160]: pgmap v302: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 670 KiB/s wr, 96 op/s 2024-09-10T10:55:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:31 smithi007 ceph-mon[30873]: pgmap v302: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 670 KiB/s wr, 96 op/s 2024-09-10T10:55:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:31 smithi133 ceph-mon[35282]: pgmap v302: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 670 KiB/s wr, 96 op/s 2024-09-10T10:55:32.530 INFO:tasks.workunit.client.0.smithi007.stdout: 1 974191 9.51 MB/sec execute 392 sec latency 1230.084 ms 2024-09-10T10:55:33.530 INFO:tasks.workunit.client.0.smithi007.stdout: 1 974191 9.48 MB/sec execute 393 sec latency 2230.178 ms 2024-09-10T10:55:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:33 smithi103 ceph-mon[35160]: pgmap v303: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 631 KiB/s wr, 86 op/s 2024-09-10T10:55:33.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:33 smithi007 ceph-mon[30873]: pgmap v303: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 631 KiB/s wr, 86 op/s 2024-09-10T10:55:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:33 smithi133 ceph-mon[35282]: pgmap v303: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 631 KiB/s wr, 86 op/s 2024-09-10T10:55:34.530 INFO:tasks.workunit.client.0.smithi007.stdout: 1 974191 9.46 MB/sec execute 394 sec latency 3230.283 ms 2024-09-10T10:55:35.530 INFO:tasks.workunit.client.0.smithi007.stdout: 1 974191 9.44 MB/sec execute 395 sec latency 4230.406 ms 2024-09-10T10:55:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:35 smithi103 ceph-mon[35160]: pgmap v304: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 631 KiB/s wr, 86 op/s 2024-09-10T10:55:35.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:35 smithi007 ceph-mon[30873]: pgmap v304: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 631 KiB/s wr, 86 op/s 2024-09-10T10:55:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:35 smithi133 ceph-mon[35282]: pgmap v304: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 631 KiB/s wr, 86 op/s 2024-09-10T10:55:36.530 INFO:tasks.workunit.client.0.smithi007.stdout: 1 974647 9.42 MB/sec execute 396 sec latency 4956.316 ms 2024-09-10T10:55:37.530 INFO:tasks.workunit.client.0.smithi007.stdout: 1 977228 9.44 MB/sec execute 397 sec latency 13.719 ms 2024-09-10T10:55:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:37 smithi103 ceph-mon[35160]: pgmap v305: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 987 KiB/s wr, 103 op/s 2024-09-10T10:55:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:37 smithi007 ceph-mon[30873]: pgmap v305: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 987 KiB/s wr, 103 op/s 2024-09-10T10:55:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:37 smithi133 ceph-mon[35282]: pgmap v305: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 987 KiB/s wr, 103 op/s 2024-09-10T10:55:38.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 977739 9.41 MB/sec execute 398 sec latency 438.276 ms 2024-09-10T10:55:39.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 977739 9.39 MB/sec execute 399 sec latency 1438.375 ms 2024-09-10T10:55:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:39 smithi103 ceph-mon[35160]: pgmap v306: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 668 KiB/s wr, 66 op/s 2024-09-10T10:55:39.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:39 smithi007 ceph-mon[30873]: pgmap v306: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 668 KiB/s wr, 66 op/s 2024-09-10T10:55:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:39 smithi133 ceph-mon[35282]: pgmap v306: 97 pgs: 97 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 668 KiB/s wr, 66 op/s 2024-09-10T10:55:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.444+0000 7f7291ded640 1 -- 172.21.15.7:0/856475448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c101350 msgr2=0x7f728c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.444+0000 7f7291ded640 1 --2- 172.21.15.7:0/856475448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c101350 0x7f728c10d880 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7284009cd0 tx=0x7f728402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.444+0000 7f7291ded640 1 -- 172.21.15.7:0/856475448 shutdown_connections 2024-09-10T10:55:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.444+0000 7f7291ded640 1 --2- 172.21.15.7:0/856475448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c101350 0x7f728c10d880 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.444+0000 7f7291ded640 1 --2- 172.21.15.7:0/856475448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f728c1009b0 0x7f728c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.444+0000 7f7291ded640 1 --2- 172.21.15.7:0/856475448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f728c1069c0 0x7f728c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.444+0000 7f7291ded640 1 -- 172.21.15.7:0/856475448 >> 172.21.15.7:0/856475448 conn(0x7f728c077990 msgr2=0x7f728c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.444+0000 7f7291ded640 1 -- 172.21.15.7:0/856475448 shutdown_connections 2024-09-10T10:55:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.444+0000 7f7291ded640 1 -- 172.21.15.7:0/856475448 wait complete. 2024-09-10T10:55:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.445+0000 7f7291ded640 1 Processor -- start 2024-09-10T10:55:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.445+0000 7f7291ded640 1 -- start start 2024-09-10T10:55:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.445+0000 7f7291ded640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f728c1009b0 0x7f728c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.445+0000 7f7291ded640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c101350 0x7f728c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.445+0000 7f7291ded640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f728c1069c0 0x7f728c1a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.445+0000 7f7291ded640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f728c110e00 con 0x7f728c101350 2024-09-10T10:55:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.445+0000 7f7291ded640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f728c110c80 con 0x7f728c1069c0 2024-09-10T10:55:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.445+0000 7f7291ded640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f728c110f80 con 0x7f728c1009b0 2024-09-10T10:55:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.445+0000 7f7282ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c101350 0x7f728c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.445+0000 7f7282ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c101350 0x7f728c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48696/0 (socket says 172.21.15.7:48696) 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f7282ffd640 1 -- 172.21.15.7:0/605852826 learned_addr learned my addr 172.21.15.7:0/605852826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f728bfff640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f728c1069c0 0x7f728c1a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f728b7fe640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f728c1009b0 0x7f728c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f7282ffd640 1 -- 172.21.15.7:0/605852826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f728c1009b0 msgr2=0x7f728c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f7282ffd640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f728c1009b0 0x7f728c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f7282ffd640 1 -- 172.21.15.7:0/605852826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f728c1069c0 msgr2=0x7f728c1a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f7282ffd640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f728c1069c0 0x7f728c1a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f7282ffd640 1 -- 172.21.15.7:0/605852826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c1a4730 con 0x7f728c101350 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f728b7fe640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f728c1009b0 0x7f728c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f7282ffd640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c101350 0x7f728c1996b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f727400d770 tx=0x7f727400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f72897fa640 1 -- 172.21.15.7:0/605852826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274017070 con 0x7f728c101350 2024-09-10T10:55:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f7291ded640 1 -- 172.21.15.7:0/605852826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c1a4900 con 0x7f728c101350 2024-09-10T10:55:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f72897fa640 1 -- 172.21.15.7:0/605852826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7274005500 con 0x7f728c101350 2024-09-10T10:55:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f72897fa640 1 -- 172.21.15.7:0/605852826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274004e00 con 0x7f728c101350 2024-09-10T10:55:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.446+0000 7f7291ded640 1 -- 172.21.15.7:0/605852826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c1a4e20 con 0x7f728c101350 2024-09-10T10:55:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.449+0000 7f72897fa640 1 -- 172.21.15.7:0/605852826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7274004fa0 con 0x7f728c101350 2024-09-10T10:55:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.449+0000 7f7294c5d740 1 Processor -- start 2024-09-10T10:55:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.449+0000 7f7294c5d740 1 -- start start 2024-09-10T10:55:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.449+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29e2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.449+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29ed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.449+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29edcb0 0x55b7f29f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.449+0000 7f7294c5d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b7f29472f0 con 0x55b7f29e2af0 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.449+0000 7f7294c5d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b7f28ef270 con 0x55b7f29e3410 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.449+0000 7f7294c5d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b7f29432b0 con 0x55b7f29edcb0 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29e2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29e2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48704/0 (socket says 172.21.15.7:48704) 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728bfff640 1 -- 172.21.15.7:0/4053466762 learned_addr learned my addr 172.21.15.7:0/4053466762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728b7fe640 1 --2- 172.21.15.7:0/4053466762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29ed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f72897fa640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72780a2da0 0x7f72780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f7282ffd640 1 --2- 172.21.15.7:0/4053466762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29edcb0 0x55b7f29f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728bfff640 1 -- 172.21.15.7:0/4053466762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29edcb0 msgr2=0x55b7f29f00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728bfff640 1 --2- 172.21.15.7:0/4053466762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29edcb0 0x55b7f29f00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728bfff640 1 -- 172.21.15.7:0/4053466762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 msgr2=0x55b7f29ed770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728bfff640 1 --2- 172.21.15.7:0/4053466762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29ed770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728bfff640 1 -- 172.21.15.7:0/4053466762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f28ee340 con 0x55b7f29e2af0 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728b7fe640 1 --2- 172.21.15.7:0/4053466762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29ed770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f72897fa640 1 -- 172.21.15.7:0/605852826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f72740cef00 con 0x7f728c101350 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728bfff640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72780a2da0 0x7f72780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f728bfff640 1 --2- 172.21.15.7:0/4053466762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29e2ed0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7284004ab0 tx=0x7f72840315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e28b953eec527e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.450+0000 7f7281ffb640 1 -- 172.21.15.7:0/4053466762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284041040 con 0x55b7f29e2af0 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7281ffb640 1 -- 172.21.15.7:0/4053466762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284043070 con 0x55b7f29e2af0 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7281ffb640 1 -- 172.21.15.7:0/4053466762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284030c20 con 0x55b7f29e2af0 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f728bfff640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72780a2da0 0x7f72780a5260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f728404c000 tx=0x7f728404d000 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7294c5d740 1 -- 172.21.15.7:0/4053466762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 msgr2=0x55b7f29e2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7294c5d740 1 --2- 172.21.15.7:0/4053466762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29e2ed0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7284004ab0 tx=0x7f72840315b0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7294c5d740 1 -- 172.21.15.7:0/4053466762 shutdown_connections 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7294c5d740 1 --2- 172.21.15.7:0/4053466762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29edcb0 0x55b7f29f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7294c5d740 1 --2- 172.21.15.7:0/4053466762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29ed770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7294c5d740 1 --2- 172.21.15.7:0/4053466762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29e2ed0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7294c5d740 1 -- 172.21.15.7:0/4053466762 >> 172.21.15.7:0/4053466762 conn(0x55b7f29df060 msgr2=0x55b7f29df440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7294c5d740 1 -- 172.21.15.7:0/4053466762 shutdown_connections 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.451+0000 7f7294c5d740 1 -- 172.21.15.7:0/4053466762 wait complete. 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f7294c5d740 1 -- start start 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f7294c5d740 1 Processor -- start 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29eab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29e3410 0x55b7f29eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29edcb0 0x55b7f29eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f7294c5d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b7f29f0880 con 0x55b7f29e2af0 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f7294c5d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b7f28d0290 con 0x55b7f29edcb0 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f7294c5d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b7f29f0a00 con 0x55b7f29e3410 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f7294c5d740 10 client.0 _ll_register_callbacks cb 0x55b7f2901140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f728b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f728b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29eab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48712/0 (socket says 172.21.15.7:48712) 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.452+0000 7f728b7fe640 1 -- 172.21.15.7:0/2950187779 learned_addr learned my addr 172.21.15.7:0/2950187779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f728bfff640 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29edcb0 0x55b7f29eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f7282ffd640 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29e3410 0x55b7f29eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f728b7fe640 1 -- 172.21.15.7:0/2950187779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29e3410 msgr2=0x55b7f29eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f728b7fe640 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29e3410 0x55b7f29eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f728b7fe640 1 -- 172.21.15.7:0/2950187779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29edcb0 msgr2=0x55b7f29eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f728b7fe640 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29edcb0 0x55b7f29eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f728b7fe640 1 -- 172.21.15.7:0/2950187779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f265a210 con 0x55b7f29e2af0 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f728b7fe640 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29eab80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f727c00dee0 tx=0x7f727c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6272e5174072aae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f726affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f726affd640 1 -- 172.21.15.7:0/2950187779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0108a0 con 0x55b7f29e2af0 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f726affd640 1 -- 172.21.15.7:0/2950187779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f727c010a40 con 0x55b7f29e2af0 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f7294c5d740 1 -- 172.21.15.7:0/2950187779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7284043070 con 0x55b7f29e2af0 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f726affd640 1 -- 172.21.15.7:0/2950187779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f727c010c40 con 0x55b7f29e2af0 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f7294c5d740 10 client.16035 fetch_fsmap learned FSMap version 39 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.453+0000 7f7294c5d740 1 -- 172.21.15.7:0/2950187779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b7f2726620 con 0x55b7f29e2af0 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f726affd640 1 -- 172.21.15.7:0/2950187779 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284043070 con 0x55b7f29e2af0 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f726affd640 1 -- 172.21.15.7:0/2950187779 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f727c023390 con 0x55b7f29e2af0 2024-09-10T10:55:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f7294c5d740 10 client.16035 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f7294c5d740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f7294c5d740 20 client.16035 populate_metadata read hostname 'smithi007' 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f7294c5d740 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b7f29fdd60 0x55b7f2a00170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f7294c5d740 4 client.16035 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f7294c5d740 1 -- 172.21.15.7:0/2950187779 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7f2954a20 con 0x55b7f29fdd60 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f7282ffd640 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b7f29fdd60 0x55b7f2a00170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f726affd640 10 client.16035 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.454+0000 7f7282ffd640 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b7f29fdd60 0x55b7f2a00170 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4317ade402d53b53 server_cookie=854f687eab6f445a in_seq=0 out_seq=0 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.455+0000 7f726affd640 1 -- 172.21.15.7:0/2950187779 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7f2954a20 con 0x55b7f29fdd60 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.455+0000 7f726affd640 10 client.16035 handle_command_reply: tid=0 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.455+0000 7f7294c5d740 1 client.16035 shutdown 2024-09-10T10:55:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.455+0000 7f7294c5d740 1 -- 172.21.15.7:0/2950187779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 msgr2=0x55b7f29eab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.455+0000 7f7294c5d740 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29eab80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f727c00dee0 tx=0x7f727c012820 comp rx=0 tx=0).stop 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.456+0000 7f7294c5d740 1 -- 172.21.15.7:0/2950187779 shutdown_connections 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.456+0000 7f7294c5d740 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b7f29fdd60 0x55b7f2a00170 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.456+0000 7f7294c5d740 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29edcb0 0x55b7f29eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.456+0000 7f7294c5d740 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29e3410 0x55b7f29eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.456+0000 7f726affd640 0 client.16035 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.456+0000 7f7294c5d740 1 --2- 172.21.15.7:0/2950187779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29eab80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.456+0000 7f7294c5d740 1 -- 172.21.15.7:0/2950187779 >> 172.21.15.7:0/2950187779 conn(0x55b7f29df060 msgr2=0x55b7f29e4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.456+0000 7f7294c5d740 1 -- 172.21.15.7:0/2950187779 shutdown_connections 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.456+0000 7f7294c5d740 1 -- 172.21.15.7:0/2950187779 wait complete. 2024-09-10T10:55:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.456+0000 7f7294c5d740 20 client.16035 trim_cache size 0 max 16384 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.470+0000 7f7294c5d740 1 Processor -- start 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.470+0000 7f7294c5d740 1 -- start start 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29e2af0 0x55b7f29e5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29edcb0 0x55b7f2a05a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f7294c5d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1b0140 con 0x55b7f29edcb0 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f7294c5d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1ad980 con 0x55b7f29e3410 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f7294c5d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1ab7f0 con 0x55b7f29e2af0 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f728bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29edcb0 0x55b7f2a05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f728bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29edcb0 0x55b7f2a05a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48724/0 (socket says 172.21.15.7:48724) 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f728bfff640 1 -- 172.21.15.7:0/1570006535 learned_addr learned my addr 172.21.15.7:0/1570006535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f728b7fe640 1 --2- 172.21.15.7:0/1570006535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29e2af0 0x55b7f29e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f7282ffd640 1 --2- 172.21.15.7:0/1570006535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f728bfff640 1 -- 172.21.15.7:0/1570006535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29e2af0 msgr2=0x55b7f29e5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f728bfff640 1 --2- 172.21.15.7:0/1570006535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29e2af0 0x55b7f29e5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f728bfff640 1 -- 172.21.15.7:0/1570006535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 msgr2=0x55b7f29eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f728bfff640 1 --2- 172.21.15.7:0/1570006535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29eb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f728bfff640 1 -- 172.21.15.7:0/1570006535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f29ed6d0 con 0x55b7f29edcb0 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f728bfff640 1 --2- 172.21.15.7:0/1570006535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29edcb0 0x55b7f2a05a80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7284005e30 tx=0x7f72840308a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dfb245eae40462 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.471+0000 7f72817fa640 1 -- 172.21.15.7:0/1570006535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284041040 con 0x55b7f29edcb0 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f72817fa640 1 -- 172.21.15.7:0/1570006535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284036050 con 0x55b7f29edcb0 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f72817fa640 1 -- 172.21.15.7:0/1570006535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72840309b0 con 0x55b7f29edcb0 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f7294c5d740 1 -- 172.21.15.7:0/1570006535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29edcb0 msgr2=0x55b7f2a05a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f7294c5d740 1 --2- 172.21.15.7:0/1570006535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29edcb0 0x55b7f2a05a80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7284005e30 tx=0x7f72840308a0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f7294c5d740 1 -- 172.21.15.7:0/1570006535 shutdown_connections 2024-09-10T10:55:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f7294c5d740 1 --2- 172.21.15.7:0/1570006535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29edcb0 0x55b7f2a05a80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f7294c5d740 1 --2- 172.21.15.7:0/1570006535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29eb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f7294c5d740 1 --2- 172.21.15.7:0/1570006535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29e2af0 0x55b7f29e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f7294c5d740 1 -- 172.21.15.7:0/1570006535 >> 172.21.15.7:0/1570006535 conn(0x55b7f29df060 msgr2=0x55b7f29ecc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f7294c5d740 1 -- 172.21.15.7:0/1570006535 shutdown_connections 2024-09-10T10:55:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f7294c5d740 1 -- 172.21.15.7:0/1570006535 wait complete. 2024-09-10T10:55:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.472+0000 7f7294c5d740 1 -- start start 2024-09-10T10:55:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f7294c5d740 1 Processor -- start 2024-09-10T10:55:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29f1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f7294c5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29edcb0 0x55b7f29f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f7294c5d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1abe40 con 0x55b7f29e2af0 2024-09-10T10:55:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f7294c5d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1ad2b0 con 0x55b7f29e3410 2024-09-10T10:55:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f7294c5d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1ac510 con 0x55b7f29edcb0 2024-09-10T10:55:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f7294c5d740 10 client.0 _ll_register_callbacks cb 0x7f727c00d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f728b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f728b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48730/0 (socket says 172.21.15.7:48730) 2024-09-10T10:55:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f728b7fe640 1 -- 172.21.15.7:0/3039385510 learned_addr learned my addr 172.21.15.7:0/3039385510 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f728bfff640 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29edcb0 0x55b7f29f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f7282ffd640 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f728b7fe640 1 -- 172.21.15.7:0/3039385510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29edcb0 msgr2=0x55b7f29f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f728b7fe640 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29edcb0 0x55b7f29f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.473+0000 7f728b7fe640 1 -- 172.21.15.7:0/3039385510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 msgr2=0x55b7f29f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f728b7fe640 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f728b7fe640 1 -- 172.21.15.7:0/3039385510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f29f2630 con 0x55b7f29e2af0 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f728b7fe640 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29f64d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f727c00dfd0 tx=0x7f727c005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b187efa9e38ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f726affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f726affd640 1 -- 172.21.15.7:0/3039385510 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c023a70 con 0x55b7f29e2af0 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f726affd640 1 -- 172.21.15.7:0/3039385510 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f727c023c10 con 0x55b7f29e2af0 2024-09-10T10:55:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f7294c5d740 1 -- 172.21.15.7:0/3039385510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7284036050 con 0x55b7f29e2af0 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f726affd640 1 -- 172.21.15.7:0/3039385510 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f727c03a050 con 0x55b7f29e2af0 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f7294c5d740 10 client.16041 fetch_fsmap learned FSMap version 39 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f7294c5d740 1 -- 172.21.15.7:0/3039385510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b7f2a68b40 con 0x55b7f29e2af0 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f726affd640 1 -- 172.21.15.7:0/3039385510 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284036050 con 0x55b7f29e2af0 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.474+0000 7f726affd640 1 -- 172.21.15.7:0/3039385510 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f727c035030 con 0x55b7f29e2af0 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.475+0000 7f7294c5d740 10 client.16041 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.475+0000 7f7294c5d740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.475+0000 7f7294c5d740 20 client.16041 populate_metadata read hostname 'smithi007' 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.475+0000 7f7294c5d740 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b7f2a69340 0x55b7f2a6b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.475+0000 7f7294c5d740 4 client.16041 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.475+0000 7f7294c5d740 1 -- 172.21.15.7:0/3039385510 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7f294ed70 con 0x55b7f2a69340 2024-09-10T10:55:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.475+0000 7f7282ffd640 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b7f2a69340 0x55b7f2a6b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.475+0000 7f7282ffd640 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b7f2a69340 0x55b7f2a6b760 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eaa7d829b82b1fa server_cookie=342fc81f09386eca in_seq=0 out_seq=0 2024-09-10T10:55:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.475+0000 7f726affd640 10 client.16041 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.519+0000 7f726affd640 1 -- 172.21.15.7:0/3039385510 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7f294ed70 con 0x55b7f2a69340 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.519+0000 7f726affd640 10 client.16041 handle_command_reply: tid=0 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.519+0000 7f7294c5d740 1 client.16041 shutdown 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 1 -- 172.21.15.7:0/3039385510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 msgr2=0x55b7f29f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29f64d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f727c00dfd0 tx=0x7f727c005b80 comp rx=0 tx=0).stop 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 1 -- 172.21.15.7:0/3039385510 shutdown_connections 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b7f2a69340 0x55b7f2a6b760 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7f29edcb0 0x55b7f29f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7f29e3410 0x55b7f29f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 1 --2- 172.21.15.7:0/3039385510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b7f29e2af0 0x55b7f29f64d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f726affd640 0 client.16041 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 1 -- 172.21.15.7:0/3039385510 >> 172.21.15.7:0/3039385510 conn(0x55b7f29df060 msgr2=0x55b7f29dd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 1 -- 172.21.15.7:0/3039385510 shutdown_connections 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 1 -- 172.21.15.7:0/3039385510 wait complete. 2024-09-10T10:55:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7294c5d740 20 client.16041 trim_cache size 0 max 16384 2024-09-10T10:55:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:55:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:55:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:55:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7291ded640 1 -- 172.21.15.7:0/605852826 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72780a2da0 msgr2=0x7f72780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7291ded640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72780a2da0 0x7f72780a5260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f728404c000 tx=0x7f728404d000 comp rx=0 tx=0).stop 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7291ded640 1 -- 172.21.15.7:0/605852826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c101350 msgr2=0x7f728c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.520+0000 7f7291ded640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c101350 0x7f728c1996b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f727400d770 tx=0x7f727400dc30 comp rx=0 tx=0).stop 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.521+0000 7f7291ded640 1 -- 172.21.15.7:0/605852826 shutdown_connections 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.521+0000 7f7291ded640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f728c1069c0 0x7f728c1a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.521+0000 7f7291ded640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72780a2da0 0x7f72780a5260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.521+0000 7f7291ded640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f728c101350 0x7f728c1996b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.521+0000 7f7291ded640 1 --2- 172.21.15.7:0/605852826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f728c1009b0 0x7f728c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.521+0000 7f7291ded640 1 -- 172.21.15.7:0/605852826 >> 172.21.15.7:0/605852826 conn(0x7f728c077990 msgr2=0x7f728c103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.521+0000 7f7291ded640 1 -- 172.21.15.7:0/605852826 shutdown_connections 2024-09-10T10:55:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:40.521+0000 7f7291ded640 1 -- 172.21.15.7:0/605852826 wait complete. 2024-09-10T10:55:40.530 INFO:tasks.workunit.client.0.smithi007.stdout: 1 977739 9.37 MB/sec execute 400 sec latency 2438.425 ms 2024-09-10T10:55:41.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 978195 9.35 MB/sec execute 401 sec latency 3161.069 ms 2024-09-10T10:55:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:41 smithi103 ceph-mon[35160]: pgmap v307: 97 pgs: 97 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 997 KiB/s wr, 105 op/s 2024-09-10T10:55:41.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:41 smithi007 ceph-mon[30873]: pgmap v307: 97 pgs: 97 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 997 KiB/s wr, 105 op/s 2024-09-10T10:55:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:41 smithi133 ceph-mon[35282]: pgmap v307: 97 pgs: 97 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 997 KiB/s wr, 105 op/s 2024-09-10T10:55:42.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 980638 9.36 MB/sec execute 402 sec latency 14.903 ms 2024-09-10T10:55:43.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981224 9.34 MB/sec execute 403 sec latency 363.986 ms 2024-09-10T10:55:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:43 smithi103 ceph-mon[35160]: pgmap v308: 97 pgs: 97 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 685 KiB/s wr, 55 op/s 2024-09-10T10:55:43.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:43 smithi007 ceph-mon[30873]: pgmap v308: 97 pgs: 97 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 685 KiB/s wr, 55 op/s 2024-09-10T10:55:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:43 smithi133 ceph-mon[35282]: pgmap v308: 97 pgs: 97 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 685 KiB/s wr, 55 op/s 2024-09-10T10:55:44.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981224 9.32 MB/sec execute 404 sec latency 1364.093 ms 2024-09-10T10:55:45.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981224 9.30 MB/sec execute 405 sec latency 2364.200 ms 2024-09-10T10:55:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:45 smithi103 ceph-mon[35160]: pgmap v309: 97 pgs: 97 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 685 KiB/s wr, 55 op/s 2024-09-10T10:55:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:45 smithi007 ceph-mon[30873]: pgmap v309: 97 pgs: 97 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 685 KiB/s wr, 55 op/s 2024-09-10T10:55:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:45 smithi133 ceph-mon[35282]: pgmap v309: 97 pgs: 97 active+clean; 272 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 685 KiB/s wr, 55 op/s 2024-09-10T10:55:46.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981280 9.27 MB/sec execute 406 sec latency 3092.225 ms 2024-09-10T10:55:47.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981280 9.25 MB/sec execute 407 sec latency 1229.609 ms 2024-09-10T10:55:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:47 smithi103 ceph-mon[35160]: pgmap v310: 97 pgs: 97 active+clean; 273 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-09-10T10:55:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:47 smithi007 ceph-mon[30873]: pgmap v310: 97 pgs: 97 active+clean; 273 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-09-10T10:55:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:47 smithi133 ceph-mon[35282]: pgmap v310: 97 pgs: 97 active+clean; 273 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-09-10T10:55:48.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981280 9.23 MB/sec execute 408 sec latency 2229.676 ms 2024-09-10T10:55:49.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981280 9.21 MB/sec execute 409 sec latency 3229.809 ms 2024-09-10T10:55:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:49 smithi103 ceph-mon[35160]: pgmap v311: 97 pgs: 97 active+clean; 273 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 900 KiB/s wr, 97 op/s 2024-09-10T10:55:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:49 smithi007 ceph-mon[30873]: pgmap v311: 97 pgs: 97 active+clean; 273 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 900 KiB/s wr, 97 op/s 2024-09-10T10:55:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:49 smithi133 ceph-mon[35282]: pgmap v311: 97 pgs: 97 active+clean; 273 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 900 KiB/s wr, 97 op/s 2024-09-10T10:55:50.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981280 9.18 MB/sec execute 410 sec latency 4229.930 ms 2024-09-10T10:55:51.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 981736 9.17 MB/sec execute 411 sec latency 4959.954 ms 2024-09-10T10:55:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:51 smithi103 ceph-mon[35160]: pgmap v312: 97 pgs: 97 active+clean; 274 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-10T10:55:51.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:51 smithi007 ceph-mon[30873]: pgmap v312: 97 pgs: 97 active+clean; 274 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-10T10:55:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:51 smithi133 ceph-mon[35282]: pgmap v312: 97 pgs: 97 active+clean; 274 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-10T10:55:52.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 984184 9.18 MB/sec execute 412 sec latency 15.469 ms 2024-09-10T10:55:53.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 984831 9.16 MB/sec execute 413 sec latency 289.184 ms 2024-09-10T10:55:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.610+0000 7f265fb6d640 1 -- 172.21.15.7:0/930085601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2658102cc0 msgr2=0x7f26581030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.610+0000 7f265fb6d640 1 --2- 172.21.15.7:0/930085601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2658102cc0 0x7f26581030a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2654009900 tx=0x7f265402a480 comp rx=0 tx=0).stop 2024-09-10T10:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.610+0000 7f265fb6d640 1 -- 172.21.15.7:0/930085601 shutdown_connections 2024-09-10T10:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.610+0000 7f265fb6d640 1 --2- 172.21.15.7:0/930085601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f265810c110 0x7f265810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.610+0000 7f265fb6d640 1 --2- 172.21.15.7:0/930085601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26581035e0 0x7f265810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.610+0000 7f265fb6d640 1 --2- 172.21.15.7:0/930085601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2658102cc0 0x7f26581030a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.610+0000 7f265fb6d640 1 -- 172.21.15.7:0/930085601 >> 172.21.15.7:0/930085601 conn(0x7f26580fc6b0 msgr2=0x7f26580fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.610+0000 7f265fb6d640 1 -- 172.21.15.7:0/930085601 shutdown_connections 2024-09-10T10:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.610+0000 7f265fb6d640 1 -- 172.21.15.7:0/930085601 wait complete. 2024-09-10T10:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265fb6d640 1 Processor -- start 2024-09-10T10:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265fb6d640 1 -- start start 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265fb6d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2658102cc0 0x7f2658198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265fb6d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26581035e0 0x7f2658198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265fb6d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f265810c110 0x7f265819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265fb6d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2658111ef0 con 0x7f26581035e0 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265fb6d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2658111d70 con 0x7f2658102cc0 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265fb6d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2658112070 con 0x7f265810c110 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265d108640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26581035e0 0x7f2658198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265d108640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26581035e0 0x7f2658198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56250/0 (socket says 172.21.15.7:56250) 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265d108640 1 -- 172.21.15.7:0/1934684290 learned_addr learned my addr 172.21.15.7:0/1934684290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265d909640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2658102cc0 0x7f2658198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.611+0000 7f265e10a640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f265810c110 0x7f265819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f265d108640 1 -- 172.21.15.7:0/1934684290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f265810c110 msgr2=0x7f265819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f265d108640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f265810c110 0x7f265819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f265d108640 1 -- 172.21.15.7:0/1934684290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2658102cc0 msgr2=0x7f2658198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f265d108640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2658102cc0 0x7f2658198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f265d108640 1 -- 172.21.15.7:0/1934684290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265819d320 con 0x7f26581035e0 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f265d909640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2658102cc0 0x7f2658198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f265d108640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26581035e0 0x7f2658198870 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f264800d750 tx=0x7f264800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f264effd640 1 -- 172.21.15.7:0/1934684290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648017070 con 0x7f26581035e0 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f265fb6d640 1 -- 172.21.15.7:0/1934684290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265819d610 con 0x7f26581035e0 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f264effd640 1 -- 172.21.15.7:0/1934684290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2648005020 con 0x7f26581035e0 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f264effd640 1 -- 172.21.15.7:0/1934684290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648004d00 con 0x7f26581035e0 2024-09-10T10:55:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.612+0000 7f265fb6d640 1 -- 172.21.15.7:0/1934684290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581a4ea0 con 0x7f26581035e0 2024-09-10T10:55:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.614+0000 7f26629e7740 1 Processor -- start 2024-09-10T10:55:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.614+0000 7f264effd640 1 -- 172.21.15.7:0/1934684290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2648009a30 con 0x7f26581035e0 2024-09-10T10:55:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f26629e7740 1 -- start start 2024-09-10T10:55:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f26629e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4beed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f26629e7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f26629e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f26629e7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fc8b4232f0 con 0x55fc8b4bf410 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f26629e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fc8b3cb270 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f26629e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fc8b41f2b0 con 0x55fc8b4beaf0 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f265d909640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f265d909640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56264/0 (socket says 172.21.15.7:56264) 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f265d909640 1 -- 172.21.15.7:0/2673355628 learned_addr learned my addr 172.21.15.7:0/2673355628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f265e10a640 1 --2- 172.21.15.7:0/2673355628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f265d108640 1 --2- 172.21.15.7:0/2673355628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f265d909640 1 -- 172.21.15.7:0/2673355628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4beaf0 msgr2=0x55fc8b4beed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f265d909640 1 --2- 172.21.15.7:0/2673355628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4beed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f265d909640 1 -- 172.21.15.7:0/2673355628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4c9cb0 msgr2=0x55fc8b4cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f265d909640 1 --2- 172.21.15.7:0/2673355628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f265d909640 1 -- 172.21.15.7:0/2673355628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc8b3ca340 con 0x55fc8b4bf410 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.615+0000 7f264effd640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f26440a2da0 0x7f26440a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.616+0000 7f265e10a640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f26440a2da0 0x7f26440a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.616+0000 7f265d909640 1 --2- 172.21.15.7:0/2673355628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c9770 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f265402a9a0 tx=0x7f2654031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6a3dd5605d8055 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.616+0000 7f264effd640 1 -- 172.21.15.7:0/1934684290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f26480cee90 con 0x7f26581035e0 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.616+0000 7f2643fff640 1 -- 172.21.15.7:0/2673355628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654035040 con 0x55fc8b4bf410 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.616+0000 7f2643fff640 1 -- 172.21.15.7:0/2673355628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2654031a20 con 0x55fc8b4bf410 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.616+0000 7f2643fff640 1 -- 172.21.15.7:0/2673355628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654004a00 con 0x55fc8b4bf410 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 -- 172.21.15.7:0/2673355628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4bf410 msgr2=0x55fc8b4c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f265e10a640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f26440a2da0 0x7f26440a5260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2650007e30 tx=0x7f2650007600 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 --2- 172.21.15.7:0/2673355628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c9770 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f265402a9a0 tx=0x7f2654031630 comp rx=0 tx=0).stop 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 -- 172.21.15.7:0/2673355628 shutdown_connections 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 --2- 172.21.15.7:0/2673355628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 --2- 172.21.15.7:0/2673355628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c9770 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 --2- 172.21.15.7:0/2673355628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 -- 172.21.15.7:0/2673355628 >> 172.21.15.7:0/2673355628 conn(0x55fc8b4bb060 msgr2=0x55fc8b4bb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 -- 172.21.15.7:0/2673355628 shutdown_connections 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 -- 172.21.15.7:0/2673355628 wait complete. 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 -- start start 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.617+0000 7f26629e7740 1 Processor -- start 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26629e7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26629e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26629e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26629e7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fc8b4cc880 con 0x55fc8b4beaf0 2024-09-10T10:55:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26629e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fc8b3ac290 con 0x55fc8b4bf410 2024-09-10T10:55:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26629e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fc8b4cca00 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26629e7740 10 client.0 _ll_register_callbacks cb 0x55fc8b3dd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d909640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d909640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56278/0 (socket says 172.21.15.7:56278) 2024-09-10T10:55:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d909640 1 -- 172.21.15.7:0/923433666 learned_addr learned my addr 172.21.15.7:0/923433666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d108640 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265e10a640 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d909640 1 -- 172.21.15.7:0/923433666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4c9cb0 msgr2=0x55fc8b4c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d909640 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d909640 1 -- 172.21.15.7:0/923433666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4bf410 msgr2=0x55fc8b4c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d909640 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d909640 1 -- 172.21.15.7:0/923433666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc8b136210 con 0x55fc8b4beaf0 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265e10a640 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d108640 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f265d909640 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c6b00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f26540042f0 tx=0x7f26540314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211d782f27056fda server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26417fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26417fa640 1 -- 172.21.15.7:0/923433666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265404e070 con 0x55fc8b4beaf0 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26417fa640 1 -- 172.21.15.7:0/923433666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f2654003800 con 0x55fc8b4beaf0 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.618+0000 7f26629e7740 1 -- 172.21.15.7:0/923433666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2654031a20 con 0x55fc8b4beaf0 2024-09-10T10:55:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26417fa640 1 -- 172.21.15.7:0/923433666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f265405a050 con 0x55fc8b4beaf0 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26629e7740 10 client.16053 fetch_fsmap learned FSMap version 39 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26629e7740 1 -- 172.21.15.7:0/923433666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fc8b202620 con 0x55fc8b4beaf0 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26417fa640 1 -- 172.21.15.7:0/923433666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2654031a20 con 0x55fc8b4beaf0 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26417fa640 1 -- 172.21.15.7:0/923433666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f2654046ad0 con 0x55fc8b4beaf0 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26629e7740 10 client.16053 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26629e7740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26629e7740 20 client.16053 populate_metadata read hostname 'smithi007' 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26629e7740 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fc8b4d9a00 0x55fc8b4dbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26629e7740 4 client.16053 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f26629e7740 1 -- 172.21.15.7:0/923433666 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc8b430a20 con 0x55fc8b4d9a00 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.619+0000 7f265d108640 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fc8b4d9a00 0x55fc8b4dbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.620+0000 7f265d108640 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fc8b4d9a00 0x55fc8b4dbdf0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e158e53621009ce5 server_cookie=d44dc00927c28e8 in_seq=0 out_seq=0 2024-09-10T10:55:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.620+0000 7f26417fa640 10 client.16053 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26417fa640 1 -- 172.21.15.7:0/923433666 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc8b430a20 con 0x55fc8b4d9a00 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26417fa640 10 client.16053 handle_command_reply: tid=0 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 client.16053 shutdown 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 -- 172.21.15.7:0/923433666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 msgr2=0x55fc8b4c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c6b00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f26540042f0 tx=0x7f26540314e0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 -- 172.21.15.7:0/923433666 shutdown_connections 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fc8b4d9a00 0x55fc8b4dbdf0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26417fa640 0 client.16053 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 --2- 172.21.15.7:0/923433666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c6b00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 -- 172.21.15.7:0/923433666 >> 172.21.15.7:0/923433666 conn(0x55fc8b4bb060 msgr2=0x55fc8b4ba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 -- 172.21.15.7:0/923433666 shutdown_connections 2024-09-10T10:55:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 1 -- 172.21.15.7:0/923433666 wait complete. 2024-09-10T10:55:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.621+0000 7f26629e7740 20 client.16053 trim_cache size 0 max 16384 2024-09-10T10:55:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f26629e7740 1 Processor -- start 2024-09-10T10:55:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f26629e7740 1 -- start start 2024-09-10T10:55:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f26629e7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f26629e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f26629e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4e1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f26629e7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26581b0080 con 0x55fc8b4beaf0 2024-09-10T10:55:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f26629e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26581ad8c0 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f26629e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f26581ab730 con 0x55fc8b4bf410 2024-09-10T10:55:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265d909640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265d909640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56292/0 (socket says 172.21.15.7:56292) 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265d909640 1 -- 172.21.15.7:0/4178238875 learned_addr learned my addr 172.21.15.7:0/4178238875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265e10a640 1 --2- 172.21.15.7:0/4178238875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265d108640 1 --2- 172.21.15.7:0/4178238875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265d909640 1 -- 172.21.15.7:0/4178238875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4bf410 msgr2=0x55fc8b4c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265d909640 1 --2- 172.21.15.7:0/4178238875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265d909640 1 -- 172.21.15.7:0/4178238875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4c9cb0 msgr2=0x55fc8b4e1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265d909640 1 --2- 172.21.15.7:0/4178238875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4e1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265d909640 1 -- 172.21.15.7:0/4178238875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc8b4bae90 con 0x55fc8b4beaf0 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.636+0000 7f265d909640 1 --2- 172.21.15.7:0/4178238875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c7030 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2654046f20 tx=0x7f2654004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de1fdae4824003f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26437fe640 1 -- 172.21.15.7:0/4178238875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654063070 con 0x55fc8b4beaf0 2024-09-10T10:55:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26437fe640 1 -- 172.21.15.7:0/4178238875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2654005850 con 0x55fc8b4beaf0 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26437fe640 1 -- 172.21.15.7:0/4178238875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265404e040 con 0x55fc8b4beaf0 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26629e7740 1 -- 172.21.15.7:0/4178238875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 msgr2=0x55fc8b4c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26629e7740 1 --2- 172.21.15.7:0/4178238875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c7030 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2654046f20 tx=0x7f2654004560 comp rx=0 tx=0).stop 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26629e7740 1 -- 172.21.15.7:0/4178238875 shutdown_connections 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26629e7740 1 --2- 172.21.15.7:0/4178238875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4e1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26629e7740 1 --2- 172.21.15.7:0/4178238875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26629e7740 1 --2- 172.21.15.7:0/4178238875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4c7030 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26629e7740 1 -- 172.21.15.7:0/4178238875 >> 172.21.15.7:0/4178238875 conn(0x55fc8b4bb060 msgr2=0x55fc8b4c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26629e7740 1 -- 172.21.15.7:0/4178238875 shutdown_connections 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.637+0000 7f26629e7740 1 -- 172.21.15.7:0/4178238875 wait complete. 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f26629e7740 1 -- start start 2024-09-10T10:55:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f26629e7740 1 Processor -- start 2024-09-10T10:55:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f26629e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f26629e7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f26629e7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f26629e7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26581abd80 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f26629e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26581ad1f0 con 0x55fc8b4beaf0 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f26629e7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f26581ac450 con 0x55fc8b4bf410 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f26629e7740 10 client.0 _ll_register_callbacks cb 0x7f26500074a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f265e10a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f265d909640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f265e10a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56302/0 (socket says 172.21.15.7:56302) 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f265d909640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:32768/0 (socket says 172.21.15.7:32768) 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.638+0000 7f265e10a640 1 -- 172.21.15.7:0/3911513106 learned_addr learned my addr 172.21.15.7:0/3911513106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:55:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f265d108640 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f265e10a640 1 -- 172.21.15.7:0/3911513106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4bf410 msgr2=0x55fc8b4c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f265e10a640 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f265e10a640 1 -- 172.21.15.7:0/3911513106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4beaf0 msgr2=0x55fc8b4d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f265e10a640 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f265e10a640 1 -- 172.21.15.7:0/3911513106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc8b4c4450 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f265d909640 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f265e10a640 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c3c20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f265001eea0 tx=0x7f2650004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41679fb46eae8fb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f26417fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f26417fa640 1 -- 172.21.15.7:0/3911513106 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265001e260 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f26417fa640 1 -- 172.21.15.7:0/3911513106 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8564+0+0 (secure 0 0 0) 0x7f265001ca40 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f26629e7740 1 -- 172.21.15.7:0/3911513106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2654005850 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f26417fa640 1 -- 172.21.15.7:0/3911513106 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f265001cc40 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f26629e7740 10 client.16059 fetch_fsmap learned FSMap version 39 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f26629e7740 1 -- 172.21.15.7:0/3911513106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fc8b5443f0 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.639+0000 7f26417fa640 1 -- 172.21.15.7:0/3911513106 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2654005850 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.640+0000 7f26417fa640 1 -- 172.21.15.7:0/3911513106 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f265001d4a0 con 0x55fc8b4c9cb0 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.640+0000 7f26629e7740 10 client.16059 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.640+0000 7f26629e7740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.640+0000 7f26629e7740 20 client.16059 populate_metadata read hostname 'smithi007' 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.640+0000 7f26629e7740 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fc8b544ac0 0x55fc8b546ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.640+0000 7f26629e7740 4 client.16059 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.640+0000 7f26629e7740 1 -- 172.21.15.7:0/3911513106 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc8b42ad70 con 0x55fc8b544ac0 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.640+0000 7f265d909640 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fc8b544ac0 0x55fc8b546ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.640+0000 7f265d909640 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fc8b544ac0 0x55fc8b546ee0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673a3fdea58703f2 server_cookie=71ecd834211ad822 in_seq=0 out_seq=0 2024-09-10T10:55:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.640+0000 7f26417fa640 10 client.16059 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.694+0000 7f26417fa640 1 -- 172.21.15.7:0/3911513106 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc8b42ad70 con 0x55fc8b544ac0 2024-09-10T10:55:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.694+0000 7f26417fa640 10 client.16059 handle_command_reply: tid=0 2024-09-10T10:55:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.694+0000 7f26629e7740 1 client.16059 shutdown 2024-09-10T10:55:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 1 -- 172.21.15.7:0/3911513106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4c9cb0 msgr2=0x55fc8b4c3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c3c20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f265001eea0 tx=0x7f2650004a40 comp rx=0 tx=0).stop 2024-09-10T10:55:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 1 -- 172.21.15.7:0/3911513106 shutdown_connections 2024-09-10T10:55:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fc8b544ac0 0x55fc8b546ee0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc8b4c9cb0 0x55fc8b4c3c20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26417fa640 0 client.16059 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:55:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc8b4bf410 0x55fc8b4c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 1 --2- 172.21.15.7:0/3911513106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fc8b4beaf0 0x55fc8b4d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 1 -- 172.21.15.7:0/3911513106 >> 172.21.15.7:0/3911513106 conn(0x55fc8b4bb060 msgr2=0x55fc8b4c04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 1 -- 172.21.15.7:0/3911513106 shutdown_connections 2024-09-10T10:55:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 1 -- 172.21.15.7:0/3911513106 wait complete. 2024-09-10T10:55:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.695+0000 7f26629e7740 20 client.16059 trim_cache size 0 max 16384 2024-09-10T10:55:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:55:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:55:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:55:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:55:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 -- 172.21.15.7:0/1934684290 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f26440a2da0 msgr2=0x7f26440a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f26440a2da0 0x7f26440a5260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2650007e30 tx=0x7f2650007600 comp rx=0 tx=0).stop 2024-09-10T10:55:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 -- 172.21.15.7:0/1934684290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26581035e0 msgr2=0x7f2658198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26581035e0 0x7f2658198870 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f264800d750 tx=0x7f264800dc10 comp rx=0 tx=0).stop 2024-09-10T10:55:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 -- 172.21.15.7:0/1934684290 shutdown_connections 2024-09-10T10:55:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f265810c110 0x7f265819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f26440a2da0 0x7f26440a5260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26581035e0 0x7f2658198870 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 --2- 172.21.15.7:0/1934684290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2658102cc0 0x7f2658198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 -- 172.21.15.7:0/1934684290 >> 172.21.15.7:0/1934684290 conn(0x7f26580fc6b0 msgr2=0x7f26580fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 -- 172.21.15.7:0/1934684290 shutdown_connections 2024-09-10T10:55:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:55:53.696+0000 7f265fb6d640 1 -- 172.21.15.7:0/1934684290 wait complete. 2024-09-10T10:55:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:53 smithi103 ceph-mon[35160]: pgmap v313: 97 pgs: 97 active+clean; 274 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 851 KiB/s wr, 71 op/s 2024-09-10T10:55:53.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:53 smithi007 ceph-mon[30873]: pgmap v313: 97 pgs: 97 active+clean; 274 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 851 KiB/s wr, 71 op/s 2024-09-10T10:55:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:53 smithi133 ceph-mon[35282]: pgmap v313: 97 pgs: 97 active+clean; 274 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 851 KiB/s wr, 71 op/s 2024-09-10T10:55:54.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 984831 9.14 MB/sec execute 414 sec latency 1289.284 ms 2024-09-10T10:55:55.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 984831 9.12 MB/sec execute 415 sec latency 2289.391 ms 2024-09-10T10:55:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:55 smithi103 ceph-mon[35160]: pgmap v314: 97 pgs: 97 active+clean; 274 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 851 KiB/s wr, 71 op/s 2024-09-10T10:55:55.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:55 smithi007 ceph-mon[30873]: pgmap v314: 97 pgs: 97 active+clean; 274 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 851 KiB/s wr, 71 op/s 2024-09-10T10:55:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:55 smithi133 ceph-mon[35282]: pgmap v314: 97 pgs: 97 active+clean; 274 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 851 KiB/s wr, 71 op/s 2024-09-10T10:55:56.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 985272 9.10 MB/sec execute 416 sec latency 3016.895 ms 2024-09-10T10:55:57.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 987658 9.12 MB/sec execute 417 sec latency 24.213 ms 2024-09-10T10:55:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:57 smithi103 ceph-mon[35160]: pgmap v315: 97 pgs: 97 active+clean; 272 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-09-10T10:55:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:57 smithi007 ceph-mon[30873]: pgmap v315: 97 pgs: 97 active+clean; 272 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-09-10T10:55:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:57 smithi133 ceph-mon[35282]: pgmap v315: 97 pgs: 97 active+clean; 272 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-09-10T10:55:58.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 988317 9.10 MB/sec execute 418 sec latency 357.464 ms 2024-09-10T10:55:59.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 988317 9.08 MB/sec execute 419 sec latency 1357.570 ms 2024-09-10T10:55:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:55:59 smithi103 ceph-mon[35160]: pgmap v316: 97 pgs: 97 active+clean; 272 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 870 KiB/s wr, 77 op/s 2024-09-10T10:55:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:55:59 smithi007 ceph-mon[30873]: pgmap v316: 97 pgs: 97 active+clean; 272 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 870 KiB/s wr, 77 op/s 2024-09-10T10:55:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:55:59 smithi133 ceph-mon[35282]: pgmap v316: 97 pgs: 97 active+clean; 272 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 870 KiB/s wr, 77 op/s 2024-09-10T10:56:00.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 988431 9.06 MB/sec execute 420 sec latency 2082.080 ms 2024-09-10T10:56:01.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 988667 9.04 MB/sec execute 421 sec latency 933.473 ms 2024-09-10T10:56:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:01 smithi103 ceph-mon[35160]: pgmap v317: 97 pgs: 97 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-10T10:56:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:01 smithi007 ceph-mon[30873]: pgmap v317: 97 pgs: 97 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-10T10:56:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:01 smithi133 ceph-mon[35282]: pgmap v317: 97 pgs: 97 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-09-10T10:56:02.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 990562 9.04 MB/sec execute 422 sec latency 28.072 ms 2024-09-10T10:56:03.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 991451 9.04 MB/sec execute 423 sec latency 526.062 ms 2024-09-10T10:56:03.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:03 smithi103 ceph-mon[35160]: pgmap v318: 97 pgs: 97 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-09-10T10:56:03.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:03 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:56:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:03 smithi007 ceph-mon[30873]: pgmap v318: 97 pgs: 97 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-09-10T10:56:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:03 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:56:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:03 smithi133 ceph-mon[35282]: pgmap v318: 97 pgs: 97 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-09-10T10:56:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:03 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:56:04.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 991451 9.02 MB/sec execute 424 sec latency 1526.157 ms 2024-09-10T10:56:04.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:04 smithi103 ceph-mon[35160]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T10:56:04.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:04 smithi007 ceph-mon[30873]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T10:56:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:04 smithi133 ceph-mon[35282]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T10:56:05.532 INFO:tasks.workunit.client.0.smithi007.stdout: 1 991451 8.99 MB/sec execute 425 sec latency 2526.207 ms 2024-09-10T10:56:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: pgmap v320: 97 pgs: 97 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-10T10:56:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:05 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: pgmap v320: 97 pgs: 97 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-10T10:56:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:05 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: pgmap v320: 97 pgs: 97 active+clean; 275 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-10T10:56:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:05.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:05.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:05 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 991451 8.97 MB/sec execute 426 sec latency 3526.242 ms 2024-09-10T10:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.785+0000 7f5594665640 1 -- 172.21.15.7:0/150997911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f558c1011d0 msgr2=0x7f558c10de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.785+0000 7f5594665640 1 --2- 172.21.15.7:0/150997911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f558c1011d0 0x7f558c10de90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f557c009900 tx=0x7f557c02a410 comp rx=0 tx=0).stop 2024-09-10T10:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.785+0000 7f5594665640 1 -- 172.21.15.7:0/150997911 shutdown_connections 2024-09-10T10:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.785+0000 7f5594665640 1 --2- 172.21.15.7:0/150997911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f558c10e3d0 0x7f558c1107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.785+0000 7f5594665640 1 --2- 172.21.15.7:0/150997911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f558c1011d0 0x7f558c10de90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.785+0000 7f5594665640 1 --2- 172.21.15.7:0/150997911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c1008b0 0x7f558c100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.785+0000 7f5594665640 1 -- 172.21.15.7:0/150997911 >> 172.21.15.7:0/150997911 conn(0x7f558c0fc6b0 msgr2=0x7f558c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.785+0000 7f5594665640 1 -- 172.21.15.7:0/150997911 shutdown_connections 2024-09-10T10:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.785+0000 7f5594665640 1 -- 172.21.15.7:0/150997911 wait complete. 2024-09-10T10:56:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5594665640 1 Processor -- start 2024-09-10T10:56:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5594665640 1 -- start start 2024-09-10T10:56:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5594665640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f558c1008b0 0x7f558c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5594665640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c1011d0 0x7f558c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5594665640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f558c10e3d0 0x7f558c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5594665640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1140e0 con 0x7f558c1011d0 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5594665640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f558c113f60 con 0x7f558c1008b0 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5594665640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f558c114260 con 0x7f558c10e3d0 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5591c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c1011d0 0x7f558c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5591c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c1011d0 0x7f558c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48264/0 (socket says 172.21.15.7:48264) 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5591c00640 1 -- 172.21.15.7:0/1461837882 learned_addr learned my addr 172.21.15.7:0/1461837882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5592401640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f558c1008b0 0x7f558c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.786+0000 7f5592c02640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f558c10e3d0 0x7f558c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f5591c00640 1 -- 172.21.15.7:0/1461837882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f558c10e3d0 msgr2=0x7f558c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f5591c00640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f558c10e3d0 0x7f558c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f5591c00640 1 -- 172.21.15.7:0/1461837882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f558c1008b0 msgr2=0x7f558c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f5591c00640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f558c1008b0 0x7f558c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f5591c00640 1 -- 172.21.15.7:0/1461837882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c19f5e0 con 0x7f558c1011d0 2024-09-10T10:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f5592401640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f558c1008b0 0x7f558c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f5591c00640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c1011d0 0x7f558c19ab30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f557c006010 tx=0x7f557c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f55837fe640 1 -- 172.21.15.7:0/1461837882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c03f030 con 0x7f558c1011d0 2024-09-10T10:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f55837fe640 1 -- 172.21.15.7:0/1461837882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f557c004440 con 0x7f558c1011d0 2024-09-10T10:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f5594665640 1 -- 172.21.15.7:0/1461837882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c19f870 con 0x7f558c1011d0 2024-09-10T10:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f55837fe640 1 -- 172.21.15.7:0/1461837882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c035070 con 0x7f558c1011d0 2024-09-10T10:56:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.787+0000 7f5594665640 1 -- 172.21.15.7:0/1461837882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c1a7160 con 0x7f558c1011d0 2024-09-10T10:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.789+0000 7f55837fe640 1 -- 172.21.15.7:0/1461837882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f557c042030 con 0x7f558c1011d0 2024-09-10T10:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5597407740 1 Processor -- start 2024-09-10T10:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5597407740 1 -- start start 2024-09-10T10:56:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5597407740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd30ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5597407740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd3b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5597407740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 0x56113fd3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5597407740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56113fc952f0 con 0x56113fd30af0 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5597407740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56113fc3d270 con 0x56113fd3bcb0 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5597407740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56113fc912b0 con 0x56113fd31410 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5592c02640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd30ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5592c02640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd30ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48278/0 (socket says 172.21.15.7:48278) 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5592c02640 1 -- 172.21.15.7:0/3340790621 learned_addr learned my addr 172.21.15.7:0/3340790621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.790+0000 7f5591c00640 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 0x56113fd3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5592401640 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd3b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f55837fe640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f55780a2da0 0x7f55780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5592c02640 1 -- 172.21.15.7:0/3340790621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 msgr2=0x56113fd3b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5592c02640 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd3b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5592c02640 1 -- 172.21.15.7:0/3340790621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 msgr2=0x56113fd3e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5592c02640 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 0x56113fd3e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5592c02640 1 -- 172.21.15.7:0/3340790621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113fc3c340 con 0x56113fd30af0 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5592401640 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd3b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5591c00640 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 0x56113fd3e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f55837fe640 1 -- 172.21.15.7:0/1461837882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8608+0+0 (secure 0 0 0) 0x7f557c0ef820 con 0x7f558c1011d0 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5592c02640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f55780a2da0 0x7f55780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5592c02640 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd30ed0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f55840114c0 tx=0x7f55840118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae003e446c22c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5580bfe640 1 -- 172.21.15.7:0/3340790621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584017070 con 0x56113fd30af0 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5580bfe640 1 -- 172.21.15.7:0/3340790621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558400fbc0 con 0x56113fd30af0 2024-09-10T10:56:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5580bfe640 1 -- 172.21.15.7:0/3340790621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558401bbe0 con 0x56113fd30af0 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.791+0000 7f5592c02640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f55780a2da0 0x7f55780a5260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5584022000 tx=0x7f5584023000 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.792+0000 7f5597407740 1 -- 172.21.15.7:0/3340790621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 msgr2=0x56113fd30ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.792+0000 7f5597407740 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd30ed0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f55840114c0 tx=0x7f55840118c0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.792+0000 7f5597407740 1 -- 172.21.15.7:0/3340790621 shutdown_connections 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.792+0000 7f5597407740 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 0x56113fd3e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.792+0000 7f5597407740 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd3b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.792+0000 7f5597407740 1 --2- 172.21.15.7:0/3340790621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd30ed0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.792+0000 7f5597407740 1 -- 172.21.15.7:0/3340790621 >> 172.21.15.7:0/3340790621 conn(0x56113fd2d060 msgr2=0x56113fd2d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.792+0000 7f5597407740 1 -- 172.21.15.7:0/3340790621 shutdown_connections 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.792+0000 7f5597407740 1 -- 172.21.15.7:0/3340790621 wait complete. 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.792+0000 7f5597407740 1 -- start start 2024-09-10T10:56:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5597407740 1 Processor -- start 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5597407740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd30af0 0x56113fd38be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5597407740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd39120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5597407740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd3bcb0 0x56113fd39660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5597407740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56113fd3e880 con 0x56113fd3bcb0 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5597407740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56113fc1e290 con 0x56113fd30af0 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5597407740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56113fd3ea00 con 0x56113fd31410 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5597407740 10 client.0 _ll_register_callbacks cb 0x56113fc4f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5592c02640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd3bcb0 0x56113fd39660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5592c02640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd3bcb0 0x56113fd39660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48280/0 (socket says 172.21.15.7:48280) 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5592c02640 1 -- 172.21.15.7:0/499870435 learned_addr learned my addr 172.21.15.7:0/499870435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5591c00640 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd39120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5592401640 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd30af0 0x56113fd38be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5592c02640 1 -- 172.21.15.7:0/499870435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 msgr2=0x56113fd39120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5592c02640 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd39120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5592c02640 1 -- 172.21.15.7:0/499870435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd30af0 msgr2=0x56113fd38be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5592c02640 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd30af0 0x56113fd38be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5592c02640 1 -- 172.21.15.7:0/499870435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113f9a8210 con 0x56113fd3bcb0 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5591c00640 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd39120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5592c02640 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd3bcb0 0x56113fd39660 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f558401b860 tx=0x7f5584005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398d996b987119e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.793+0000 7f5575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5575ffb640 1 -- 172.21.15.7:0/499870435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584035060 con 0x56113fd3bcb0 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5575ffb640 1 -- 172.21.15.7:0/499870435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8608+0+0 (secure 0 0 0) 0x7f558400f4e0 con 0x56113fd3bcb0 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5597407740 1 -- 172.21.15.7:0/499870435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558400fbc0 con 0x56113fd3bcb0 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5575ffb640 1 -- 172.21.15.7:0/499870435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5584017040 con 0x56113fd3bcb0 2024-09-10T10:56:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5597407740 10 client.16071 fetch_fsmap learned FSMap version 39 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5597407740 1 -- 172.21.15.7:0/499870435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56113fa74620 con 0x56113fd3bcb0 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5575ffb640 1 -- 172.21.15.7:0/499870435 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558400fbc0 con 0x56113fd3bcb0 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5575ffb640 1 -- 172.21.15.7:0/499870435 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f5584026740 con 0x56113fd3bcb0 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5597407740 10 client.16071 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5597407740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5597407740 20 client.16071 populate_metadata read hostname 'smithi007' 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5597407740 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56113fd4bb90 0x56113fd4df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5597407740 4 client.16071 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.794+0000 7f5597407740 1 -- 172.21.15.7:0/499870435 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56113fca2a20 con 0x56113fd4bb90 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.795+0000 7f5592401640 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56113fd4bb90 0x56113fd4df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.795+0000 7f5592401640 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56113fd4bb90 0x56113fd4df80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa26d7b62fa090fc server_cookie=881280cc4d43f616 in_seq=0 out_seq=0 2024-09-10T10:56:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.795+0000 7f5575ffb640 10 client.16071 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5575ffb640 1 -- 172.21.15.7:0/499870435 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56113fca2a20 con 0x56113fd4bb90 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5575ffb640 10 client.16071 handle_command_reply: tid=0 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 client.16071 shutdown 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 -- 172.21.15.7:0/499870435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd3bcb0 msgr2=0x56113fd39660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd3bcb0 0x56113fd39660 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f558401b860 tx=0x7f5584005bd0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 -- 172.21.15.7:0/499870435 shutdown_connections 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56113fd4bb90 0x56113fd4df80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd3bcb0 0x56113fd39660 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd39120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5575ffb640 0 client.16071 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 --2- 172.21.15.7:0/499870435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd30af0 0x56113fd38be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 -- 172.21.15.7:0/499870435 >> 172.21.15.7:0/499870435 conn(0x56113fd2d060 msgr2=0x56113fd32d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 -- 172.21.15.7:0/499870435 shutdown_connections 2024-09-10T10:56:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 1 -- 172.21.15.7:0/499870435 wait complete. 2024-09-10T10:56:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.796+0000 7f5597407740 20 client.16071 trim_cache size 0 max 16384 2024-09-10T10:56:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.810+0000 7f5597407740 1 Processor -- start 2024-09-10T10:56:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5597407740 1 -- start start 2024-09-10T10:56:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5597407740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd30af0 0x56113fd394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5597407740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd31410 0x56113fd537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5597407740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd3bcb0 0x56113fd53ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5597407740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1b2460 con 0x56113fd31410 2024-09-10T10:56:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5597407740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1afca0 con 0x56113fd30af0 2024-09-10T10:56:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5597407740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1adaf0 con 0x56113fd3bcb0 2024-09-10T10:56:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5591c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd31410 0x56113fd537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5591c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd31410 0x56113fd537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48290/0 (socket says 172.21.15.7:48290) 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5591c00640 1 -- 172.21.15.7:0/3827434063 learned_addr learned my addr 172.21.15.7:0/3827434063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5592401640 1 --2- 172.21.15.7:0/3827434063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd30af0 0x56113fd394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5591c00640 1 -- 172.21.15.7:0/3827434063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd3bcb0 msgr2=0x56113fd53ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5591c00640 1 --2- 172.21.15.7:0/3827434063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd3bcb0 0x56113fd53ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5591c00640 1 -- 172.21.15.7:0/3827434063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd30af0 msgr2=0x56113fd394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5591c00640 1 --2- 172.21.15.7:0/3827434063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd30af0 0x56113fd394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5591c00640 1 -- 172.21.15.7:0/3827434063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113fd4e510 con 0x56113fd31410 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5591c00640 1 --2- 172.21.15.7:0/3827434063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd31410 0x56113fd537a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f557c0ecf40 tx=0x7f557c0bb780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca4808cd558b229 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.811+0000 7f5577fff640 1 -- 172.21.15.7:0/3827434063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c049040 con 0x56113fd31410 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5577fff640 1 -- 172.21.15.7:0/3827434063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f557c0c4920 con 0x56113fd31410 2024-09-10T10:56:06.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5577fff640 1 -- 172.21.15.7:0/3827434063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c0bbfc0 con 0x56113fd31410 2024-09-10T10:56:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5597407740 1 -- 172.21.15.7:0/3827434063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd31410 msgr2=0x56113fd537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5597407740 1 --2- 172.21.15.7:0/3827434063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd31410 0x56113fd537a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f557c0ecf40 tx=0x7f557c0bb780 comp rx=0 tx=0).stop 2024-09-10T10:56:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5597407740 1 -- 172.21.15.7:0/3827434063 shutdown_connections 2024-09-10T10:56:06.818 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5597407740 1 --2- 172.21.15.7:0/3827434063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd3bcb0 0x56113fd53ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5597407740 1 --2- 172.21.15.7:0/3827434063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd31410 0x56113fd537a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5597407740 1 --2- 172.21.15.7:0/3827434063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd30af0 0x56113fd394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5597407740 1 -- 172.21.15.7:0/3827434063 >> 172.21.15.7:0/3827434063 conn(0x56113fd2d060 msgr2=0x56113fd3ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5597407740 1 -- 172.21.15.7:0/3827434063 shutdown_connections 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.812+0000 7f5597407740 1 -- 172.21.15.7:0/3827434063 wait complete. 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5597407740 1 -- start start 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5597407740 1 Processor -- start 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5597407740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd35d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5597407740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd36270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5597407740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 0x56113fd367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5597407740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1ae160 con 0x56113fd30af0 2024-09-10T10:56:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5597407740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1af5d0 con 0x56113fd3bcb0 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5597407740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f558c1ae830 con 0x56113fd31410 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5597407740 10 client.0 _ll_register_callbacks cb 0x7f55840094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5592401640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd35d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5592401640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd35d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48296/0 (socket says 172.21.15.7:48296) 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5592401640 1 -- 172.21.15.7:0/662536342 learned_addr learned my addr 172.21.15.7:0/662536342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5592c02640 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 0x56113fd367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5591c00640 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd36270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5592401640 1 -- 172.21.15.7:0/662536342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 msgr2=0x56113fd36270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5592401640 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd36270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5592401640 1 -- 172.21.15.7:0/662536342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 msgr2=0x56113fd367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5592401640 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 0x56113fd367b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.813+0000 7f5592401640 1 -- 172.21.15.7:0/662536342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113fd30920 con 0x56113fd30af0 2024-09-10T10:56:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5592401640 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd35d30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x56113fd2dec0 tx=0x7f5588010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173c8773b0cfdd05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5575ffb640 1 -- 172.21.15.7:0/662536342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558802a060 con 0x56113fd30af0 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5575ffb640 1 -- 172.21.15.7:0/662536342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8608+0+0 (secure 0 0 0) 0x7f558800fcb0 con 0x56113fd30af0 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5597407740 1 -- 172.21.15.7:0/662536342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f557c0c4920 con 0x56113fd30af0 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5575ffb640 1 -- 172.21.15.7:0/662536342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5588004030 con 0x56113fd30af0 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5597407740 10 client.16077 fetch_fsmap learned FSMap version 39 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5597407740 1 -- 172.21.15.7:0/662536342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56113fdb6a10 con 0x56113fd30af0 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5575ffb640 1 -- 172.21.15.7:0/662536342 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f557c0c4920 con 0x56113fd30af0 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5575ffb640 1 -- 172.21.15.7:0/662536342 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f558801e410 con 0x56113fd30af0 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5597407740 10 client.16077 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5597407740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:56:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5597407740 20 client.16077 populate_metadata read hostname 'smithi007' 2024-09-10T10:56:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5597407740 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56113fdb7120 0x56113fdb9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5597407740 4 client.16077 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:56:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.814+0000 7f5597407740 1 -- 172.21.15.7:0/662536342 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56113fc9cd70 con 0x56113fdb7120 2024-09-10T10:56:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.815+0000 7f5591c00640 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56113fdb7120 0x56113fdb9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.815+0000 7f5591c00640 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56113fdb7120 0x56113fdb9540 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e3bb7ab4c414155 server_cookie=11603cb2c4e17838 in_seq=0 out_seq=0 2024-09-10T10:56:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.815+0000 7f5575ffb640 10 client.16077 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5575ffb640 1 -- 172.21.15.7:0/662536342 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56113fc9cd70 con 0x56113fdb7120 2024-09-10T10:56:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5575ffb640 10 client.16077 handle_command_reply: tid=0 2024-09-10T10:56:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 client.16077 shutdown 2024-09-10T10:56:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 -- 172.21.15.7:0/662536342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 msgr2=0x56113fd35d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd35d30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x56113fd2dec0 tx=0x7f5588010c70 comp rx=0 tx=0).stop 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 -- 172.21.15.7:0/662536342 shutdown_connections 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56113fdb7120 0x56113fdb9540 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56113fd3bcb0 0x56113fd367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56113fd31410 0x56113fd36270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5575ffb640 0 client.16077 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 --2- 172.21.15.7:0/662536342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56113fd30af0 0x56113fd35d30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 -- 172.21.15.7:0/662536342 >> 172.21.15.7:0/662536342 conn(0x56113fd2d060 msgr2=0x56113fd2b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 -- 172.21.15.7:0/662536342 shutdown_connections 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 1 -- 172.21.15.7:0/662536342 wait complete. 2024-09-10T10:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.871+0000 7f5597407740 20 client.16077 trim_cache size 0 max 16384 2024-09-10T10:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:56:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 -- 172.21.15.7:0/1461837882 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f55780a2da0 msgr2=0x7f55780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f55780a2da0 0x7f55780a5260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5584022000 tx=0x7f5584023000 comp rx=0 tx=0).stop 2024-09-10T10:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 -- 172.21.15.7:0/1461837882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c1011d0 msgr2=0x7f558c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c1011d0 0x7f558c19ab30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f557c006010 tx=0x7f557c004040 comp rx=0 tx=0).stop 2024-09-10T10:56:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 -- 172.21.15.7:0/1461837882 shutdown_connections 2024-09-10T10:56:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f558c10e3d0 0x7f558c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f55780a2da0 0x7f55780a5260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c1011d0 0x7f558c19ab30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 --2- 172.21.15.7:0/1461837882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f558c1008b0 0x7f558c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 -- 172.21.15.7:0/1461837882 >> 172.21.15.7:0/1461837882 conn(0x7f558c0fc6b0 msgr2=0x7f558c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 -- 172.21.15.7:0/1461837882 shutdown_connections 2024-09-10T10:56:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:06.872+0000 7f5594665640 1 -- 172.21.15.7:0/1461837882 wait complete. 2024-09-10T10:56:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-10T10:56:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: Unable to set osd_memory_target on smithi103 to 184593920: error parsing value: Value '184593920' is below minimum 939524096 2024-09-10T10:56:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:56:06.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:56:06.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: Unable to set osd_memory_target on smithi103 to 184593920: error parsing value: Value '184593920' is below minimum 939524096 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:56:06.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:56:06.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi103 to 176.0M 2024-09-10T10:56:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: Unable to set osd_memory_target on smithi103 to 184593920: error parsing value: Value '184593920' is below minimum 939524096 2024-09-10T10:56:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:56:06.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:56:06.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:56:06.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:56:07.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 991451 8.95 MB/sec execute 427 sec latency 4526.351 ms 2024-09-10T10:56:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:07 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-10T10:56:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:07 smithi103 ceph-mon[35160]: Unable to set osd_memory_target on smithi133 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-10T10:56:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:07 smithi103 ceph-mon[35160]: Adjusting osd_memory_target on smithi007 to 177.0M 2024-09-10T10:56:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:07 smithi103 ceph-mon[35160]: Unable to set osd_memory_target on smithi007 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-10T10:56:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:07 smithi103 ceph-mon[35160]: pgmap v321: 97 pgs: 97 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-09-10T10:56:07.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:07 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-10T10:56:07.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:07 smithi007 ceph-mon[30873]: Unable to set osd_memory_target on smithi133 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-10T10:56:07.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:07 smithi007 ceph-mon[30873]: Adjusting osd_memory_target on smithi007 to 177.0M 2024-09-10T10:56:07.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:07 smithi007 ceph-mon[30873]: Unable to set osd_memory_target on smithi007 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-10T10:56:07.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:07 smithi007 ceph-mon[30873]: pgmap v321: 97 pgs: 97 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-09-10T10:56:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:07 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi133 to 176.0M 2024-09-10T10:56:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:07 smithi133 ceph-mon[35282]: Unable to set osd_memory_target on smithi133 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-10T10:56:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:07 smithi133 ceph-mon[35282]: Adjusting osd_memory_target on smithi007 to 177.0M 2024-09-10T10:56:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:07 smithi133 ceph-mon[35282]: Unable to set osd_memory_target on smithi007 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-10T10:56:07.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:07 smithi133 ceph-mon[35282]: pgmap v321: 97 pgs: 97 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-09-10T10:56:08.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 991881 8.93 MB/sec execute 428 sec latency 5015.306 ms 2024-09-10T10:56:09.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 992022 8.91 MB/sec execute 429 sec latency 868.314 ms 2024-09-10T10:56:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:09 smithi103 ceph-mon[35160]: pgmap v322: 97 pgs: 97 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-09-10T10:56:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:09 smithi103 ceph-mon[35160]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T10:56:09.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:09 smithi007 ceph-mon[30873]: pgmap v322: 97 pgs: 97 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-09-10T10:56:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:09 smithi007 ceph-mon[30873]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T10:56:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:09 smithi133 ceph-mon[35282]: pgmap v322: 97 pgs: 97 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-09-10T10:56:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:09 smithi133 ceph-mon[35282]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T10:56:10.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 992022 8.89 MB/sec execute 430 sec latency 1868.433 ms 2024-09-10T10:56:11.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 992490 8.87 MB/sec execute 431 sec latency 2595.233 ms 2024-09-10T10:56:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:11 smithi103 ceph-mon[35160]: pgmap v324: 97 pgs: 97 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-09-10T10:56:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:11 smithi007 ceph-mon[30873]: pgmap v324: 97 pgs: 97 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-09-10T10:56:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:11 smithi133 ceph-mon[35282]: pgmap v324: 97 pgs: 97 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-09-10T10:56:12.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 994921 8.89 MB/sec execute 432 sec latency 22.111 ms 2024-09-10T10:56:13.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 995565 8.87 MB/sec execute 433 sec latency 359.126 ms 2024-09-10T10:56:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:13 smithi103 ceph-mon[35160]: pgmap v325: 97 pgs: 97 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-10T10:56:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:13 smithi007 ceph-mon[30873]: pgmap v325: 97 pgs: 97 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-10T10:56:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:13 smithi133 ceph-mon[35282]: pgmap v325: 97 pgs: 97 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-10T10:56:14.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 995565 8.85 MB/sec execute 434 sec latency 1359.176 ms 2024-09-10T10:56:15.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 995565 8.83 MB/sec execute 435 sec latency 2359.257 ms 2024-09-10T10:56:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:15 smithi103 ceph-mon[35160]: pgmap v326: 97 pgs: 97 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-09-10T10:56:15.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:15 smithi007 ceph-mon[30873]: pgmap v326: 97 pgs: 97 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-09-10T10:56:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:15 smithi133 ceph-mon[35282]: pgmap v326: 97 pgs: 97 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-09-10T10:56:16.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 996025 8.82 MB/sec execute 436 sec latency 3084.469 ms 2024-09-10T10:56:17.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 998458 8.83 MB/sec execute 437 sec latency 24.562 ms 2024-09-10T10:56:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:17 smithi103 ceph-mon[35160]: pgmap v327: 97 pgs: 97 active+clean; 263 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T10:56:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:17 smithi007 ceph-mon[30873]: pgmap v327: 97 pgs: 97 active+clean; 263 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T10:56:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:17 smithi133 ceph-mon[35282]: pgmap v327: 97 pgs: 97 active+clean; 263 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T10:56:18.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 998990 8.82 MB/sec execute 438 sec latency 451.007 ms 2024-09-10T10:56:19.533 INFO:tasks.workunit.client.0.smithi007.stdout: 1 998990 8.80 MB/sec execute 439 sec latency 1451.144 ms 2024-09-10T10:56:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:19 smithi103 ceph-mon[35160]: pgmap v328: 97 pgs: 97 active+clean; 263 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T10:56:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:19 smithi007 ceph-mon[30873]: pgmap v328: 97 pgs: 97 active+clean; 263 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T10:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.972+0000 7f81bbdcf640 1 -- 172.21.15.7:0/2842575354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b41069c0 msgr2=0x7f81b4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.972+0000 7f81bbdcf640 1 --2- 172.21.15.7:0/2842575354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b41069c0 0x7f81b4106da0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009930 tx=0x7f81b002a420 comp rx=0 tx=0).stop 2024-09-10T10:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.972+0000 7f81bbdcf640 1 -- 172.21.15.7:0/2842575354 shutdown_connections 2024-09-10T10:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.972+0000 7f81bbdcf640 1 --2- 172.21.15.7:0/2842575354 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b4101350 0x7f81b410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.972+0000 7f81bbdcf640 1 --2- 172.21.15.7:0/2842575354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81b41009b0 0x7f81b4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.972+0000 7f81bbdcf640 1 --2- 172.21.15.7:0/2842575354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b41069c0 0x7f81b4106da0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.972+0000 7f81bbdcf640 1 -- 172.21.15.7:0/2842575354 >> 172.21.15.7:0/2842575354 conn(0x7f81b4077990 msgr2=0x7f81b40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.972+0000 7f81bbdcf640 1 -- 172.21.15.7:0/2842575354 shutdown_connections 2024-09-10T10:56:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.972+0000 7f81bbdcf640 1 -- 172.21.15.7:0/2842575354 wait complete. 2024-09-10T10:56:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:19 smithi133 ceph-mon[35282]: pgmap v328: 97 pgs: 97 active+clean; 263 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T10:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.973+0000 7f81bbdcf640 1 Processor -- start 2024-09-10T10:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.973+0000 7f81bbdcf640 1 -- start start 2024-09-10T10:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.973+0000 7f81bbdcf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b41009b0 0x7f81b4102580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.973+0000 7f81bbdcf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b4101350 0x7f81b4102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.973+0000 7f81bbdcf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81b41069c0 0x7f81b4103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.973+0000 7f81bbdcf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4110e00 con 0x7f81b41009b0 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.973+0000 7f81bbdcf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4110c80 con 0x7f81b4101350 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.973+0000 7f81bbdcf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4110f80 con 0x7f81b41069c0 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.973+0000 7f81b9b6b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b41009b0 0x7f81b4102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81b9b6b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b41009b0 0x7f81b4102580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40162/0 (socket says 172.21.15.7:40162) 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81b9b6b640 1 -- 172.21.15.7:0/3773404307 learned_addr learned my addr 172.21.15.7:0/3773404307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81b936a640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b4101350 0x7f81b4102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81ba36c640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81b41069c0 0x7f81b4103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81b9b6b640 1 -- 172.21.15.7:0/3773404307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81b41069c0 msgr2=0x7f81b4103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81b41069c0 0x7f81b4103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81b9b6b640 1 -- 172.21.15.7:0/3773404307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b4101350 msgr2=0x7f81b4102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b4101350 0x7f81b4102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81b9b6b640 1 -- 172.21.15.7:0/3773404307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b41a6870 con 0x7f81b41009b0 2024-09-10T10:56:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b41009b0 0x7f81b4102580 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009230 tx=0x7f81b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81aaffd640 1 -- 172.21.15.7:0/3773404307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b003f030 con 0x7f81b41009b0 2024-09-10T10:56:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81aaffd640 1 -- 172.21.15.7:0/3773404307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b0003ad0 con 0x7f81b41009b0 2024-09-10T10:56:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81bbdcf640 1 -- 172.21.15.7:0/3773404307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b41a6b00 con 0x7f81b41009b0 2024-09-10T10:56:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81aaffd640 1 -- 172.21.15.7:0/3773404307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0035070 con 0x7f81b41009b0 2024-09-10T10:56:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.974+0000 7f81bbdcf640 1 -- 172.21.15.7:0/3773404307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b41a7040 con 0x7f81b41009b0 2024-09-10T10:56:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.977+0000 7f81aaffd640 1 -- 172.21.15.7:0/3773404307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f81b0042030 con 0x7f81b41009b0 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81bec46740 1 Processor -- start 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81bec46740 1 -- start start 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81bec46740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a19af0 0x562d57a19ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81bec46740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a1a410 0x562d57a24770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81bec46740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81bec46740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562d5797e2f0 con 0x562d57a24cb0 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81bec46740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562d57926270 con 0x562d57a1a410 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81bec46740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562d5797a2b0 con 0x562d57a19af0 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81b936a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81aaffd640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f81a00a2e20 0x7f81a00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81b936a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a270a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40166/0 (socket says 172.21.15.7:40166) 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81b936a640 1 -- 172.21.15.7:0/1346206771 learned_addr learned my addr 172.21.15.7:0/1346206771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.978+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/1346206771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a1a410 0x562d57a24770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f81ba36c640 1 --2- 172.21.15.7:0/1346206771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a19af0 0x562d57a19ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f81ba36c640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f81a00a2e20 0x7f81a00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f81aaffd640 1 -- 172.21.15.7:0/3773404307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7f81b00ef930 con 0x7f81b41009b0 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f81b936a640 1 -- 172.21.15.7:0/1346206771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a19af0 msgr2=0x562d57a19ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f81b936a640 1 --2- 172.21.15.7:0/1346206771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a19af0 0x562d57a19ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f81b936a640 1 -- 172.21.15.7:0/1346206771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a1a410 msgr2=0x562d57a24770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f81b936a640 1 --2- 172.21.15.7:0/1346206771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a1a410 0x562d57a24770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f81b936a640 1 -- 172.21.15.7:0/1346206771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d57925340 con 0x562d57a24cb0 2024-09-10T10:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f81b936a640 1 --2- 172.21.15.7:0/1346206771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a270a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f81a400bcb0 tx=0x7f81a40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c731e0a79a6403bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f81ba36c640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f81a00a2e20 0x7f81a00a52e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00bfd0 tx=0x7f81ac004a40 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f819ffff640 1 -- 172.21.15.7:0/1346206771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4004030 con 0x562d57a24cb0 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f819ffff640 1 -- 172.21.15.7:0/1346206771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a40041d0 con 0x562d57a24cb0 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.979+0000 7f819ffff640 1 -- 172.21.15.7:0/1346206771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a40116a0 con 0x562d57a24cb0 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.980+0000 7f81bec46740 1 -- 172.21.15.7:0/1346206771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 msgr2=0x562d57a270a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.980+0000 7f81bec46740 1 --2- 172.21.15.7:0/1346206771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a270a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f81a400bcb0 tx=0x7f81a40107a0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.980+0000 7f81bec46740 1 -- 172.21.15.7:0/1346206771 shutdown_connections 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.980+0000 7f81bec46740 1 --2- 172.21.15.7:0/1346206771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a270a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.980+0000 7f81bec46740 1 --2- 172.21.15.7:0/1346206771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a1a410 0x562d57a24770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.980+0000 7f81bec46740 1 --2- 172.21.15.7:0/1346206771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a19af0 0x562d57a19ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.980+0000 7f81bec46740 1 -- 172.21.15.7:0/1346206771 >> 172.21.15.7:0/1346206771 conn(0x562d57a16060 msgr2=0x562d57a16440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.980+0000 7f81bec46740 1 -- 172.21.15.7:0/1346206771 shutdown_connections 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.980+0000 7f81bec46740 1 -- 172.21.15.7:0/1346206771 wait complete. 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81bec46740 1 -- start start 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81bec46740 1 Processor -- start 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81bec46740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 0x562d57a21b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81bec46740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a1a410 0x562d57a220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81bec46740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 0x562d57a22600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81bec46740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562d57a27880 con 0x562d57a1a410 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81bec46740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562d57907290 con 0x562d57a19af0 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81bec46740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562d57a27a00 con 0x562d57a24cb0 2024-09-10T10:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81bec46740 10 client.0 _ll_register_callbacks cb 0x562d57938140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81b936a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a1a410 0x562d57a220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81b936a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a1a410 0x562d57a220c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40170/0 (socket says 172.21.15.7:40170) 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.981+0000 7f81b936a640 1 -- 172.21.15.7:0/926380797 learned_addr learned my addr 172.21.15.7:0/926380797 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81ba36c640 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 0x562d57a22600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 0x562d57a21b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81b936a640 1 -- 172.21.15.7:0/926380797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 msgr2=0x562d57a22600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81b936a640 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 0x562d57a22600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81b936a640 1 -- 172.21.15.7:0/926380797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 msgr2=0x562d57a21b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81b936a640 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 0x562d57a21b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81b936a640 1 -- 172.21.15.7:0/926380797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d57691210 con 0x562d57a1a410 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81ba36c640 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 0x562d57a22600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81b936a640 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a1a410 0x562d57a220c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f81a400b8e0 tx=0x7f81a400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f2ffb7d1197bc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f819d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f819d7fa640 1 -- 172.21.15.7:0/926380797 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a400be20 con 0x562d57a1a410 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f819d7fa640 1 -- 172.21.15.7:0/926380797 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7f81a40035f0 con 0x562d57a1a410 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81bec46740 1 -- 172.21.15.7:0/926380797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a40041d0 con 0x562d57a1a410 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f819d7fa640 1 -- 172.21.15.7:0/926380797 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f81a4005600 con 0x562d57a1a410 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81bec46740 10 client.16089 fetch_fsmap learned FSMap version 39 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f81bec46740 1 -- 172.21.15.7:0/926380797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562d5775d620 con 0x562d57a1a410 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.982+0000 7f819d7fa640 1 -- 172.21.15.7:0/926380797 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a40041d0 con 0x562d57a1a410 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.983+0000 7f819d7fa640 1 -- 172.21.15.7:0/926380797 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f81a402e430 con 0x562d57a1a410 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.983+0000 7f81bec46740 10 client.16089 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.983+0000 7f81bec46740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.983+0000 7f81bec46740 20 client.16089 populate_metadata read hostname 'smithi007' 2024-09-10T10:56:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.983+0000 7f81bec46740 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562d57a34bf0 0x562d57a36fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.983+0000 7f81bec46740 4 client.16089 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:56:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.983+0000 7f81bec46740 1 -- 172.21.15.7:0/926380797 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d5798ba20 con 0x562d57a34bf0 2024-09-10T10:56:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.983+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562d57a34bf0 0x562d57a36fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.983+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562d57a34bf0 0x562d57a36fe0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f42bcc92fe7170f server_cookie=a4cdaab6e1d1727d in_seq=0 out_seq=0 2024-09-10T10:56:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.983+0000 7f819d7fa640 10 client.16089 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.984+0000 7f819d7fa640 1 -- 172.21.15.7:0/926380797 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d5798ba20 con 0x562d57a34bf0 2024-09-10T10:56:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.984+0000 7f819d7fa640 10 client.16089 handle_command_reply: tid=0 2024-09-10T10:56:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.984+0000 7f81bec46740 1 client.16089 shutdown 2024-09-10T10:56:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.984+0000 7f81bec46740 1 -- 172.21.15.7:0/926380797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a1a410 msgr2=0x562d57a220c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.984+0000 7f81bec46740 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a1a410 0x562d57a220c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f81a400b8e0 tx=0x7f81a400b910 comp rx=0 tx=0).stop 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.985+0000 7f81bec46740 1 -- 172.21.15.7:0/926380797 shutdown_connections 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.985+0000 7f81bec46740 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562d57a34bf0 0x562d57a36fe0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.985+0000 7f81bec46740 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 0x562d57a22600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.985+0000 7f81bec46740 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a1a410 0x562d57a220c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.985+0000 7f819d7fa640 0 client.16089 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.985+0000 7f81bec46740 1 --2- 172.21.15.7:0/926380797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 0x562d57a21b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.985+0000 7f81bec46740 1 -- 172.21.15.7:0/926380797 >> 172.21.15.7:0/926380797 conn(0x562d57a16060 msgr2=0x562d57a1bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.985+0000 7f81bec46740 1 -- 172.21.15.7:0/926380797 shutdown_connections 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.985+0000 7f81bec46740 1 -- 172.21.15.7:0/926380797 wait complete. 2024-09-10T10:56:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:19.985+0000 7f81bec46740 20 client.16089 trim_cache size 0 max 16384 2024-09-10T10:56:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81bec46740 1 Processor -- start 2024-09-10T10:56:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81bec46740 1 -- start start 2024-09-10T10:56:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81bec46740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 0x562d57a1c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81bec46740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a1a410 0x562d57a22430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81bec46740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a3c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81bec46740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b2280 con 0x562d57a24cb0 2024-09-10T10:56:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81bec46740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41afac0 con 0x562d57a19af0 2024-09-10T10:56:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81bec46740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41ad930 con 0x562d57a1a410 2024-09-10T10:56:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81ba36c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a3c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81ba36c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a3c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40184/0 (socket says 172.21.15.7:40184) 2024-09-10T10:56:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81ba36c640 1 -- 172.21.15.7:0/3800203132 learned_addr learned my addr 172.21.15.7:0/3800203132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81b936a640 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a1a410 0x562d57a22430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 0x562d57a1c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81ba36c640 1 -- 172.21.15.7:0/3800203132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a1a410 msgr2=0x562d57a22430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81ba36c640 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a1a410 0x562d57a22430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81ba36c640 1 -- 172.21.15.7:0/3800203132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 msgr2=0x562d57a1c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81ba36c640 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 0x562d57a1c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81ba36c640 1 -- 172.21.15.7:0/3800203132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d57a15e90 con 0x562d57a24cb0 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 0x562d57a1c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.000+0000 7f81b936a640 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a1a410 0x562d57a22430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.001+0000 7f81ba36c640 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a3c890 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0109e0 tx=0x7f81ac015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb730a38ada260e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.001+0000 7f819f7fe640 1 -- 172.21.15.7:0/3800203132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac013230 con 0x562d57a24cb0 2024-09-10T10:56:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.001+0000 7f819f7fe640 1 -- 172.21.15.7:0/3800203132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81ac012450 con 0x562d57a24cb0 2024-09-10T10:56:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.001+0000 7f819f7fe640 1 -- 172.21.15.7:0/3800203132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac00e650 con 0x562d57a24cb0 2024-09-10T10:56:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 -- 172.21.15.7:0/3800203132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 msgr2=0x562d57a3c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a3c890 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0109e0 tx=0x7f81ac015070 comp rx=0 tx=0).stop 2024-09-10T10:56:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 -- 172.21.15.7:0/3800203132 shutdown_connections 2024-09-10T10:56:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a24cb0 0x562d57a3c890 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a1a410 0x562d57a22430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 --2- 172.21.15.7:0/3800203132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a19af0 0x562d57a1c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 -- 172.21.15.7:0/3800203132 >> 172.21.15.7:0/3800203132 conn(0x562d57a16060 msgr2=0x562d57a20b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 -- 172.21.15.7:0/3800203132 shutdown_connections 2024-09-10T10:56:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 -- 172.21.15.7:0/3800203132 wait complete. 2024-09-10T10:56:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 -- start start 2024-09-10T10:56:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 Processor -- start 2024-09-10T10:56:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a19af0 0x562d57a2cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a1a410 0x562d57a2d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 0x562d57a282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41adf80 con 0x562d57a19af0 2024-09-10T10:56:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41af3f0 con 0x562d57a1a410 2024-09-10T10:56:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.002+0000 7f81bec46740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41ae650 con 0x562d57a24cb0 2024-09-10T10:56:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81bec46740 10 client.0 _ll_register_callbacks cb 0x7f81ac00bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81b9b6b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a19af0 0x562d57a2cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81b9b6b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a19af0 0x562d57a2cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40192/0 (socket says 172.21.15.7:40192) 2024-09-10T10:56:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81b9b6b640 1 -- 172.21.15.7:0/3639165351 learned_addr learned my addr 172.21.15.7:0/3639165351 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81b936a640 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a1a410 0x562d57a2d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81ba36c640 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 0x562d57a282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81b9b6b640 1 -- 172.21.15.7:0/3639165351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 msgr2=0x562d57a282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 0x562d57a282f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81b9b6b640 1 -- 172.21.15.7:0/3639165351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a1a410 msgr2=0x562d57a2d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a1a410 0x562d57a2d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81b9b6b640 1 -- 172.21.15.7:0/3639165351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d57a28b20 con 0x562d57a19af0 2024-09-10T10:56:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81ba36c640 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 0x562d57a282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81b9b6b640 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a19af0 0x562d57a2cf70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f81b00b8790 tx=0x7f81b00f1d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8f451291c51ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f819d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f819d7fa640 1 -- 172.21.15.7:0/3639165351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0049040 con 0x562d57a19af0 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f819d7fa640 1 -- 172.21.15.7:0/3639165351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7f81b0003c80 con 0x562d57a19af0 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.003+0000 7f81bec46740 1 -- 172.21.15.7:0/3639165351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81ac012450 con 0x562d57a19af0 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f819d7fa640 1 -- 172.21.15.7:0/3639165351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f81b00c4a30 con 0x562d57a19af0 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f81bec46740 10 client.16095 fetch_fsmap learned FSMap version 39 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f81bec46740 1 -- 172.21.15.7:0/3639165351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562d57a9f3c0 con 0x562d57a19af0 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f819d7fa640 1 -- 172.21.15.7:0/3639165351 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81ac012450 con 0x562d57a19af0 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f819d7fa640 1 -- 172.21.15.7:0/3639165351 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7f81b0031bd0 con 0x562d57a19af0 2024-09-10T10:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f81bec46740 10 client.16095 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:56:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f81bec46740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:56:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f81bec46740 20 client.16095 populate_metadata read hostname 'smithi007' 2024-09-10T10:56:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f81bec46740 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562d57a9fac0 0x562d57aa1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f81bec46740 4 client.16095 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:56:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f81bec46740 1 -- 172.21.15.7:0/3639165351 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d57985d70 con 0x562d57a9fac0 2024-09-10T10:56:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.004+0000 7f81b936a640 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562d57a9fac0 0x562d57aa1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.005+0000 7f819d7fa640 10 client.16095 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.005+0000 7f81b936a640 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562d57a9fac0 0x562d57aa1ee0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83d15552392f3ac server_cookie=c4fbc9e3491adf29 in_seq=0 out_seq=0 2024-09-10T10:56:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.056+0000 7f819d7fa640 1 -- 172.21.15.7:0/3639165351 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d57985d70 con 0x562d57a9fac0 2024-09-10T10:56:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f819d7fa640 10 client.16095 handle_command_reply: tid=0 2024-09-10T10:56:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 client.16095 shutdown 2024-09-10T10:56:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 -- 172.21.15.7:0/3639165351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a19af0 msgr2=0x562d57a2cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a19af0 0x562d57a2cf70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f81b00b8790 tx=0x7f81b00f1d80 comp rx=0 tx=0).stop 2024-09-10T10:56:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 -- 172.21.15.7:0/3639165351 shutdown_connections 2024-09-10T10:56:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x562d57a9fac0 0x562d57aa1ee0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562d57a24cb0 0x562d57a282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562d57a1a410 0x562d57a2d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 --2- 172.21.15.7:0/3639165351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562d57a19af0 0x562d57a2cf70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f819d7fa640 0 client.16095 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 -- 172.21.15.7:0/3639165351 >> 172.21.15.7:0/3639165351 conn(0x562d57a16060 msgr2=0x562d57a20300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 -- 172.21.15.7:0/3639165351 shutdown_connections 2024-09-10T10:56:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 1 -- 172.21.15.7:0/3639165351 wait complete. 2024-09-10T10:56:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.057+0000 7f81bec46740 20 client.16095 trim_cache size 0 max 16384 2024-09-10T10:56:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:56:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:56:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:56:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:56:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 -- 172.21.15.7:0/3773404307 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f81a00a2e20 msgr2=0x7f81a00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f81a00a2e20 0x7f81a00a52e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00bfd0 tx=0x7f81ac004a40 comp rx=0 tx=0).stop 2024-09-10T10:56:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 -- 172.21.15.7:0/3773404307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b41009b0 msgr2=0x7f81b4102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b41009b0 0x7f81b4102580 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009230 tx=0x7f81b0004270 comp rx=0 tx=0).stop 2024-09-10T10:56:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 -- 172.21.15.7:0/3773404307 shutdown_connections 2024-09-10T10:56:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81b41069c0 0x7f81b4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b4101350 0x7f81b4102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f81a00a2e20 0x7f81a00a52e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 --2- 172.21.15.7:0/3773404307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b41009b0 0x7f81b4102580 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 -- 172.21.15.7:0/3773404307 >> 172.21.15.7:0/3773404307 conn(0x7f81b4077990 msgr2=0x7f81b40ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 -- 172.21.15.7:0/3773404307 shutdown_connections 2024-09-10T10:56:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:20.058+0000 7f81bbdcf640 1 -- 172.21.15.7:0/3773404307 wait complete. 2024-09-10T10:56:20.534 INFO:tasks.workunit.client.0.smithi007.stdout: 1 998990 8.78 MB/sec execute 440 sec latency 2451.276 ms 2024-09-10T10:56:21.534 INFO:tasks.workunit.client.0.smithi007.stdout: 1 999417 8.76 MB/sec execute 441 sec latency 3176.451 ms 2024-09-10T10:56:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:21 smithi103 ceph-mon[35160]: pgmap v329: 97 pgs: 97 active+clean; 263 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-10T10:56:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:21 smithi007 ceph-mon[30873]: pgmap v329: 97 pgs: 97 active+clean; 263 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-10T10:56:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:21 smithi133 ceph-mon[35282]: pgmap v329: 97 pgs: 97 active+clean; 263 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-09-10T10:56:22.534 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1001999 8.78 MB/sec execute 442 sec latency 21.504 ms 2024-09-10T10:56:23.534 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002531 8.76 MB/sec execute 443 sec latency 456.269 ms 2024-09-10T10:56:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:23 smithi103 ceph-mon[35160]: pgmap v330: 97 pgs: 97 active+clean; 263 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 901 KiB/s wr, 88 op/s 2024-09-10T10:56:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:23 smithi007 ceph-mon[30873]: pgmap v330: 97 pgs: 97 active+clean; 263 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 901 KiB/s wr, 88 op/s 2024-09-10T10:56:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:23 smithi133 ceph-mon[35282]: pgmap v330: 97 pgs: 97 active+clean; 263 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 901 KiB/s wr, 88 op/s 2024-09-10T10:56:24.534 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002531 8.74 MB/sec execute 444 sec latency 1456.369 ms 2024-09-10T10:56:25.534 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002531 8.72 MB/sec execute 445 sec latency 2456.427 ms 2024-09-10T10:56:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:25 smithi103 ceph-mon[35160]: pgmap v331: 97 pgs: 97 active+clean; 263 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 901 KiB/s wr, 88 op/s 2024-09-10T10:56:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:25 smithi007 ceph-mon[30873]: pgmap v331: 97 pgs: 97 active+clean; 263 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 901 KiB/s wr, 88 op/s 2024-09-10T10:56:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:25 smithi133 ceph-mon[35282]: pgmap v331: 97 pgs: 97 active+clean; 263 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 901 KiB/s wr, 88 op/s 2024-09-10T10:56:26.534 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002587 8.70 MB/sec execute 446 sec latency 3184.178 ms 2024-09-10T10:56:27.534 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002587 8.68 MB/sec execute 447 sec latency 1232.871 ms 2024-09-10T10:56:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:27 smithi103 ceph-mon[35160]: pgmap v332: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-10T10:56:27.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:27 smithi007 ceph-mon[30873]: pgmap v332: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-10T10:56:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:27 smithi133 ceph-mon[35282]: pgmap v332: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-10T10:56:28.534 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002587 8.66 MB/sec execute 448 sec latency 2233.012 ms 2024-09-10T10:56:29.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002587 8.64 MB/sec execute 449 sec latency 3233.139 ms 2024-09-10T10:56:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:29 smithi103 ceph-mon[35160]: pgmap v333: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 904 KiB/s wr, 99 op/s 2024-09-10T10:56:29.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:29 smithi007 ceph-mon[30873]: pgmap v333: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 904 KiB/s wr, 99 op/s 2024-09-10T10:56:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:29 smithi133 ceph-mon[35282]: pgmap v333: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 904 KiB/s wr, 99 op/s 2024-09-10T10:56:30.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002587 8.62 MB/sec execute 450 sec latency 4233.237 ms 2024-09-10T10:56:31.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002655 8.60 MB/sec execute 451 sec latency 4959.556 ms 2024-09-10T10:56:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:31 smithi103 ceph-mon[35160]: pgmap v334: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-10T10:56:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:31 smithi007 ceph-mon[30873]: pgmap v334: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-10T10:56:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:31 smithi133 ceph-mon[35282]: pgmap v334: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-10T10:56:32.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002655 8.59 MB/sec execute 452 sec latency 1225.297 ms 2024-09-10T10:56:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.155+0000 7faeb98a4640 1 -- 172.21.15.7:0/779592026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faeb4103680 msgr2=0x7faeb410bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.155+0000 7faeb98a4640 1 --2- 172.21.15.7:0/779592026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faeb4103680 0x7faeb410bb60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009900 tx=0x7fae9c02a480 comp rx=0 tx=0).stop 2024-09-10T10:56:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.155+0000 7faeb98a4640 1 -- 172.21.15.7:0/779592026 shutdown_connections 2024-09-10T10:56:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.155+0000 7faeb98a4640 1 --2- 172.21.15.7:0/779592026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faeb410c130 0x7faeb410e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.155+0000 7faeb98a4640 1 --2- 172.21.15.7:0/779592026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faeb4103680 0x7faeb410bb60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.155+0000 7faeb98a4640 1 --2- 172.21.15.7:0/779592026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb4102d60 0x7faeb4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.155+0000 7faeb98a4640 1 -- 172.21.15.7:0/779592026 >> 172.21.15.7:0/779592026 conn(0x7faeb40fc6b0 msgr2=0x7faeb40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.155+0000 7faeb98a4640 1 -- 172.21.15.7:0/779592026 shutdown_connections 2024-09-10T10:56:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.155+0000 7faeb98a4640 1 -- 172.21.15.7:0/779592026 wait complete. 2024-09-10T10:56:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeb98a4640 1 Processor -- start 2024-09-10T10:56:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeb98a4640 1 -- start start 2024-09-10T10:56:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeb98a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb4102d60 0x7faeb41982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeb98a4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faeb4103680 0x7faeb4198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeb98a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faeb410c130 0x7faeb419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeb98a4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faeb4111f00 con 0x7faeb4103680 2024-09-10T10:56:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeb98a4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faeb4111d80 con 0x7faeb4102d60 2024-09-10T10:56:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeb98a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faeb4112080 con 0x7faeb410c130 2024-09-10T10:56:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faeb4103680 0x7faeb4198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faeb4103680 0x7faeb4198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37946/0 (socket says 172.21.15.7:37946) 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeabfff640 1 -- 172.21.15.7:0/3071665046 learned_addr learned my addr 172.21.15.7:0/3071665046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeb37fe640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faeb410c130 0x7faeb419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.156+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb4102d60 0x7faeb41982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeabfff640 1 -- 172.21.15.7:0/3071665046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faeb410c130 msgr2=0x7faeb419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeabfff640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faeb410c130 0x7faeb419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeabfff640 1 -- 172.21.15.7:0/3071665046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb4102d60 msgr2=0x7faeb41982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeabfff640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb4102d60 0x7faeb41982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeabfff640 1 -- 172.21.15.7:0/3071665046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb419d2c0 con 0x7faeb4103680 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeabfff640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faeb4103680 0x7faeb4198810 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fae9c006010 tx=0x7fae9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeb88a2640 1 -- 172.21.15.7:0/3071665046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c03f030 con 0x7faeb4103680 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeb88a2640 1 -- 172.21.15.7:0/3071665046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9c003ce0 con 0x7faeb4103680 2024-09-10T10:56:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeb98a4640 1 -- 172.21.15.7:0/3071665046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb419d550 con 0x7faeb4103680 2024-09-10T10:56:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeb88a2640 1 -- 172.21.15.7:0/3071665046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c035070 con 0x7faeb4103680 2024-09-10T10:56:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.157+0000 7faeb98a4640 1 -- 172.21.15.7:0/3071665046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb41a4e40 con 0x7faeb4103680 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.160+0000 7faebc637740 1 Processor -- start 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faebc637740 1 -- start start 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb88a2640 1 -- 172.21.15.7:0/3071665046 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fae9c042030 con 0x7faeb4103680 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faebc637740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e64fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faebc637740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e65eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faebc637740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e65f0c0 0x56103e6614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faebc637740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56103e5b42f0 con 0x56103e650310 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faebc637740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56103e55c270 con 0x56103e65f0c0 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faebc637740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56103e5b02b0 con 0x56103e64f9f0 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e65eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e65eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37952/0 (socket says 172.21.15.7:37952) 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb2ffd640 1 -- 172.21.15.7:0/2881271780 learned_addr learned my addr 172.21.15.7:0/2881271780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb37fe640 1 --2- 172.21.15.7:0/2881271780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e64fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeabfff640 1 --2- 172.21.15.7:0/2881271780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e65f0c0 0x56103e6614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb2ffd640 1 -- 172.21.15.7:0/2881271780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 msgr2=0x56103e64fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/2881271780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e64fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb2ffd640 1 -- 172.21.15.7:0/2881271780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e65f0c0 msgr2=0x56103e6614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/2881271780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e65f0c0 0x56103e6614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb2ffd640 1 -- 172.21.15.7:0/2881271780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56103e55b340 con 0x56103e650310 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.161+0000 7faeb37fe640 1 --2- 172.21.15.7:0/2881271780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e64fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.162+0000 7faeb88a2640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7faea00a2e20 0x7faea00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.162+0000 7faeb37fe640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7faea00a2e20 0x7faea00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.162+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/2881271780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e65eb80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faea400cc60 tx=0x7faea400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9daf9b47713ae675 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.162+0000 7faeb88a2640 1 -- 172.21.15.7:0/3071665046 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7fae9c0ef940 con 0x7faeb4103680 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.162+0000 7faea9ffb640 1 -- 172.21.15.7:0/2881271780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4014070 con 0x56103e650310 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.162+0000 7faea9ffb640 1 -- 172.21.15.7:0/2881271780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea40047a0 con 0x56103e650310 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.162+0000 7faea9ffb640 1 -- 172.21.15.7:0/2881271780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4010790 con 0x56103e650310 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faebc637740 1 -- 172.21.15.7:0/2881271780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 msgr2=0x56103e65eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faeb37fe640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7faea00a2e20 0x7faea00a52e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faeac005fd0 tx=0x7faeac009600 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faebc637740 1 --2- 172.21.15.7:0/2881271780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e65eb80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faea400cc60 tx=0x7faea400e720 comp rx=0 tx=0).stop 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faebc637740 1 -- 172.21.15.7:0/2881271780 shutdown_connections 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faebc637740 1 --2- 172.21.15.7:0/2881271780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e65f0c0 0x56103e6614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faebc637740 1 --2- 172.21.15.7:0/2881271780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e65eb80 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faebc637740 1 --2- 172.21.15.7:0/2881271780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e64fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faebc637740 1 -- 172.21.15.7:0/2881271780 >> 172.21.15.7:0/2881271780 conn(0x56103e64c060 msgr2=0x56103e64c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faebc637740 1 -- 172.21.15.7:0/2881271780 shutdown_connections 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faebc637740 1 -- 172.21.15.7:0/2881271780 wait complete. 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.163+0000 7faebc637740 1 -- start start 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faebc637740 1 Processor -- start 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faebc637740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e65bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faebc637740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e65c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faebc637740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e65ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faebc637740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56103e661d20 con 0x56103e65f0c0 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faebc637740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56103e53d290 con 0x56103e650310 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faebc637740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56103e661ea0 con 0x56103e64f9f0 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faebc637740 10 client.0 _ll_register_callbacks cb 0x56103e56e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e65ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faeabfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e65c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e65ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37958/0 (socket says 172.21.15.7:37958) 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faeabfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e65c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:54266/0 (socket says 172.21.15.7:54266) 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faeb37fe640 1 -- 172.21.15.7:0/2988311391 learned_addr learned my addr 172.21.15.7:0/2988311391 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e65bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faeb37fe640 1 -- 172.21.15.7:0/2988311391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 msgr2=0x56103e65bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faeb37fe640 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e65bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faeb37fe640 1 -- 172.21.15.7:0/2988311391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 msgr2=0x56103e65c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.164+0000 7faeb37fe640 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e65c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7faeb37fe640 1 -- 172.21.15.7:0/2988311391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56103e2c7210 con 0x56103e65f0c0 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7faeabfff640 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e65c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:56:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7faeb37fe640 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e65ca50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faeac005f40 tx=0x7faeac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8120c8fb1aa49b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7fae92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7fae92ffd640 1 -- 172.21.15.7:0/2988311391 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac01ed30 con 0x56103e65f0c0 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7fae92ffd640 1 -- 172.21.15.7:0/2988311391 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7faeac02a480 con 0x56103e65f0c0 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7faebc637740 1 -- 172.21.15.7:0/2988311391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea40047a0 con 0x56103e65f0c0 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7fae92ffd640 1 -- 172.21.15.7:0/2988311391 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faeac009af0 con 0x56103e65f0c0 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7faebc637740 10 client.16107 fetch_fsmap learned FSMap version 39 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7faebc637740 1 -- 172.21.15.7:0/2988311391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56103e393620 con 0x56103e65f0c0 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7fae92ffd640 1 -- 172.21.15.7:0/2988311391 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea40047a0 con 0x56103e65f0c0 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.165+0000 7fae92ffd640 1 -- 172.21.15.7:0/2988311391 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7faeac031070 con 0x56103e65f0c0 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.166+0000 7faebc637740 10 client.16107 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.166+0000 7faebc637740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:56:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.166+0000 7faebc637740 20 client.16107 populate_metadata read hostname 'smithi007' 2024-09-10T10:56:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.166+0000 7faebc637740 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56103e66f0d0 0x56103e671540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.166+0000 7faebc637740 4 client.16107 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:56:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.166+0000 7faebc637740 1 -- 172.21.15.7:0/2988311391 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56103e5c1a20 con 0x56103e66f0d0 2024-09-10T10:56:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.166+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56103e66f0d0 0x56103e671540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.166+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56103e66f0d0 0x56103e671540 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75b80c4db581398b server_cookie=8ec5870d59375065 in_seq=0 out_seq=0 2024-09-10T10:56:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.166+0000 7fae92ffd640 10 client.16107 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7fae92ffd640 1 -- 172.21.15.7:0/2988311391 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56103e5c1a20 con 0x56103e66f0d0 2024-09-10T10:56:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7fae92ffd640 10 client.16107 handle_command_reply: tid=0 2024-09-10T10:56:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7faebc637740 1 client.16107 shutdown 2024-09-10T10:56:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7faebc637740 1 -- 172.21.15.7:0/2988311391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 msgr2=0x56103e65ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7faebc637740 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e65ca50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7faeac005f40 tx=0x7faeac004a40 comp rx=0 tx=0).stop 2024-09-10T10:56:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7faebc637740 1 -- 172.21.15.7:0/2988311391 shutdown_connections 2024-09-10T10:56:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7faebc637740 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56103e66f0d0 0x56103e671540 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7faebc637740 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e65ca50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7faebc637740 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e65c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7fae92ffd640 0 client.16107 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7faebc637740 1 --2- 172.21.15.7:0/2988311391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e65bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7faebc637740 1 -- 172.21.15.7:0/2988311391 >> 172.21.15.7:0/2988311391 conn(0x56103e64c060 msgr2=0x56103e651d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.167+0000 7faebc637740 1 -- 172.21.15.7:0/2988311391 shutdown_connections 2024-09-10T10:56:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.168+0000 7faebc637740 1 -- 172.21.15.7:0/2988311391 wait complete. 2024-09-10T10:56:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.168+0000 7faebc637740 20 client.16107 trim_cache size 0 max 16384 2024-09-10T10:56:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.182+0000 7faebc637740 1 Processor -- start 2024-09-10T10:56:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.182+0000 7faebc637740 1 -- start start 2024-09-10T10:56:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.182+0000 7faebc637740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e65df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faebc637740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e676e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faebc637740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e65f0c0 0x56103e677340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faebc637740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41adf20 con 0x56103e650310 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faebc637740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41ab760 con 0x56103e65f0c0 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faebc637740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41a95d0 con 0x56103e64f9f0 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e676e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e676e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37968/0 (socket says 172.21.15.7:37968) 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeabfff640 1 -- 172.21.15.7:0/4041587817 learned_addr learned my addr 172.21.15.7:0/4041587817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/4041587817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e65df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeabfff640 1 -- 172.21.15.7:0/4041587817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 msgr2=0x56103e65df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeb37fe640 1 --2- 172.21.15.7:0/4041587817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e65f0c0 0x56103e677340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeabfff640 1 --2- 172.21.15.7:0/4041587817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e65df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeabfff640 1 -- 172.21.15.7:0/4041587817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e65f0c0 msgr2=0x56103e677340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeabfff640 1 --2- 172.21.15.7:0/4041587817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e65f0c0 0x56103e677340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeabfff640 1 -- 172.21.15.7:0/4041587817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56103e65eb20 con 0x56103e650310 2024-09-10T10:56:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/4041587817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e65df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faeabfff640 1 --2- 172.21.15.7:0/4041587817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e676e00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0f52f0 tx=0x7fae9c0f2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97d079e790d0a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faea97fa640 1 -- 172.21.15.7:0/4041587817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c049040 con 0x56103e650310 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faea97fa640 1 -- 172.21.15.7:0/4041587817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9c0bc5f0 con 0x56103e650310 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.183+0000 7faea97fa640 1 -- 172.21.15.7:0/4041587817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0c4b60 con 0x56103e650310 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 -- 172.21.15.7:0/4041587817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 msgr2=0x56103e676e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 --2- 172.21.15.7:0/4041587817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e676e00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0f52f0 tx=0x7fae9c0f2f90 comp rx=0 tx=0).stop 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 -- 172.21.15.7:0/4041587817 shutdown_connections 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 --2- 172.21.15.7:0/4041587817 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e65f0c0 0x56103e677340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 --2- 172.21.15.7:0/4041587817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e650310 0x56103e676e00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 --2- 172.21.15.7:0/4041587817 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e65df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 -- 172.21.15.7:0/4041587817 >> 172.21.15.7:0/4041587817 conn(0x56103e64c060 msgr2=0x56103e65d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 -- 172.21.15.7:0/4041587817 shutdown_connections 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 -- 172.21.15.7:0/4041587817 wait complete. 2024-09-10T10:56:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 -- start start 2024-09-10T10:56:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.184+0000 7faebc637740 1 Processor -- start 2024-09-10T10:56:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faebc637740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e663600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faebc637740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e663b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faebc637740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e6d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faebc637740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41a9c20 con 0x56103e65f0c0 2024-09-10T10:56:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faebc637740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41ab090 con 0x56103e650310 2024-09-10T10:56:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faebc637740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faeb41aa2f0 con 0x56103e64f9f0 2024-09-10T10:56:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faebc637740 10 client.0 _ll_register_callbacks cb 0x7faeac009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e6d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeb37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e6d8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37982/0 (socket says 172.21.15.7:37982) 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeb37fe640 1 -- 172.21.15.7:0/2908914214 learned_addr learned my addr 172.21.15.7:0/2908914214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e663600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeabfff640 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e663b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeb37fe640 1 -- 172.21.15.7:0/2908914214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 msgr2=0x56103e663600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeb37fe640 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e663600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeb37fe640 1 -- 172.21.15.7:0/2908914214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 msgr2=0x56103e663b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeb37fe640 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e663b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeb37fe640 1 -- 172.21.15.7:0/2908914214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56103e6d9340 con 0x56103e65f0c0 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeabfff640 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e663b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7faeb37fe640 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e6d8dd0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faeac000f80 tx=0x7faeac01efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb0098133c2d4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.185+0000 7fae92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:56:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7fae92ffd640 1 -- 172.21.15.7:0/2908914214 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac01e770 con 0x56103e65f0c0 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7fae92ffd640 1 -- 172.21.15.7:0/2908914214 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7faeac004cc0 con 0x56103e65f0c0 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7faebc637740 1 -- 172.21.15.7:0/2908914214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9c0bc5f0 con 0x56103e65f0c0 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7fae92ffd640 1 -- 172.21.15.7:0/2908914214 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faeac009ab0 con 0x56103e65f0c0 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7faebc637740 10 client.16113 fetch_fsmap learned FSMap version 39 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7faebc637740 1 -- 172.21.15.7:0/2908914214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56103e6da650 con 0x56103e65f0c0 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7fae92ffd640 1 -- 172.21.15.7:0/2908914214 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae9c0bc5f0 con 0x56103e65f0c0 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7fae92ffd640 1 -- 172.21.15.7:0/2908914214 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 39) ==== 8358+0+0 (secure 0 0 0) 0x7faeac042020 con 0x56103e65f0c0 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7faebc637740 10 client.16113 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7faebc637740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7faebc637740 20 client.16113 populate_metadata read hostname 'smithi007' 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7faebc637740 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56103e6daeb0 0x56103e6dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7faebc637740 4 client.16113 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:56:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7faebc637740 1 -- 172.21.15.7:0/2908914214 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56103e5bbd70 con 0x56103e6daeb0 2024-09-10T10:56:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.186+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56103e6daeb0 0x56103e6dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.187+0000 7faeb2ffd640 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56103e6daeb0 0x56103e6dd2d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63efbc6890b79e76 server_cookie=1e2d21afd1492d7b in_seq=0 out_seq=0 2024-09-10T10:56:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.187+0000 7fae92ffd640 10 client.16113 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7fae92ffd640 1 -- 172.21.15.7:0/2908914214 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56103e5bbd70 con 0x56103e6daeb0 2024-09-10T10:56:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7fae92ffd640 10 client.16113 handle_command_reply: tid=0 2024-09-10T10:56:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 client.16113 shutdown 2024-09-10T10:56:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 -- 172.21.15.7:0/2908914214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 msgr2=0x56103e6d8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e6d8dd0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faeac000f80 tx=0x7faeac01efc0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 -- 172.21.15.7:0/2908914214 shutdown_connections 2024-09-10T10:56:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56103e6daeb0 0x56103e6dd2d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56103e65f0c0 0x56103e6d8dd0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56103e650310 0x56103e663b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7fae92ffd640 0 client.16113 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 --2- 172.21.15.7:0/2908914214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56103e64f9f0 0x56103e663600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 -- 172.21.15.7:0/2908914214 >> 172.21.15.7:0/2908914214 conn(0x56103e64c060 msgr2=0x56103e65c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 -- 172.21.15.7:0/2908914214 shutdown_connections 2024-09-10T10:56:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 1 -- 172.21.15.7:0/2908914214 wait complete. 2024-09-10T10:56:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.226+0000 7faebc637740 20 client.16113 trim_cache size 0 max 16384 2024-09-10T10:56:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:56:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:56:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:56:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 -- 172.21.15.7:0/3071665046 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7faea00a2e20 msgr2=0x7faea00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7faea00a2e20 0x7faea00a52e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faeac005fd0 tx=0x7faeac009600 comp rx=0 tx=0).stop 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 -- 172.21.15.7:0/3071665046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faeb4103680 msgr2=0x7faeb4198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faeb4103680 0x7faeb4198810 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fae9c006010 tx=0x7fae9c0042f0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 -- 172.21.15.7:0/3071665046 shutdown_connections 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faeb410c130 0x7faeb419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7faea00a2e20 0x7faea00a52e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faeb4103680 0x7faeb4198810 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 --2- 172.21.15.7:0/3071665046 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeb4102d60 0x7faeb41982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 -- 172.21.15.7:0/3071665046 >> 172.21.15.7:0/3071665046 conn(0x7faeb40fc6b0 msgr2=0x7faeb40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 -- 172.21.15.7:0/3071665046 shutdown_connections 2024-09-10T10:56:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:33.227+0000 7faeb98a4640 1 -- 172.21.15.7:0/3071665046 wait complete. 2024-09-10T10:56:33.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1002655 8.57 MB/sec execute 453 sec latency 2225.380 ms 2024-09-10T10:56:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:33 smithi103 ceph-mon[35160]: pgmap v335: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 880 KiB/s wr, 79 op/s 2024-09-10T10:56:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:33 smithi007 ceph-mon[30873]: pgmap v335: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 880 KiB/s wr, 79 op/s 2024-09-10T10:56:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:33 smithi133 ceph-mon[35282]: pgmap v335: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 880 KiB/s wr, 79 op/s 2024-09-10T10:56:34.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1003122 8.55 MB/sec execute 454 sec latency 2956.558 ms 2024-09-10T10:56:35.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1005655 8.57 MB/sec execute 455 sec latency 14.631 ms 2024-09-10T10:56:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:35 smithi103 ceph-mon[35160]: pgmap v336: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 880 KiB/s wr, 79 op/s 2024-09-10T10:56:35.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:35 smithi007 ceph-mon[30873]: pgmap v336: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 880 KiB/s wr, 79 op/s 2024-09-10T10:56:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:35 smithi133 ceph-mon[35282]: pgmap v336: 97 pgs: 97 active+clean; 265 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 880 KiB/s wr, 79 op/s 2024-09-10T10:56:36.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1006204 8.55 MB/sec execute 456 sec latency 299.109 ms 2024-09-10T10:56:37.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1006204 8.53 MB/sec execute 457 sec latency 1190.756 ms 2024-09-10T10:56:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:37 smithi103 ceph-mon[35160]: pgmap v337: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-10T10:56:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:37 smithi007 ceph-mon[30873]: pgmap v337: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-10T10:56:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:37 smithi133 ceph-mon[35282]: pgmap v337: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-10T10:56:38.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1006204 8.51 MB/sec execute 458 sec latency 2190.877 ms 2024-09-10T10:56:39.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1006204 8.50 MB/sec execute 459 sec latency 3190.967 ms 2024-09-10T10:56:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:39 smithi103 ceph-mon[35160]: pgmap v338: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 689 KiB/s wr, 57 op/s 2024-09-10T10:56:39.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:39 smithi007 ceph-mon[30873]: pgmap v338: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 689 KiB/s wr, 57 op/s 2024-09-10T10:56:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:39 smithi133 ceph-mon[35282]: pgmap v338: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 689 KiB/s wr, 57 op/s 2024-09-10T10:56:40.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1006204 8.48 MB/sec execute 460 sec latency 4191.082 ms 2024-09-10T10:56:41.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1006672 8.46 MB/sec execute 461 sec latency 4916.728 ms 2024-09-10T10:56:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:41 smithi103 ceph-mon[35160]: pgmap v339: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 947 KiB/s wr, 90 op/s 2024-09-10T10:56:41.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:41 smithi007 ceph-mon[30873]: pgmap v339: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 947 KiB/s wr, 90 op/s 2024-09-10T10:56:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:41 smithi133 ceph-mon[35282]: pgmap v339: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 947 KiB/s wr, 90 op/s 2024-09-10T10:56:42.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1009200 8.48 MB/sec execute 462 sec latency 13.851 ms 2024-09-10T10:56:42.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:42 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:56:42.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:42 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:56:42.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:42 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:56:43.535 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1010616 8.47 MB/sec execute 463 sec latency 15.716 ms 2024-09-10T10:56:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:43 smithi103 ceph-mon[35160]: pgmap v340: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 653 KiB/s wr, 71 op/s 2024-09-10T10:56:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:43 smithi007 ceph-mon[30873]: pgmap v340: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 653 KiB/s wr, 71 op/s 2024-09-10T10:56:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:43 smithi133 ceph-mon[35282]: pgmap v340: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 653 KiB/s wr, 71 op/s 2024-09-10T10:56:44.536 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1012826 8.49 MB/sec execute 464 sec latency 22.168 ms 2024-09-10T10:56:45.536 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1013171 8.47 MB/sec execute 465 sec latency 675.887 ms 2024-09-10T10:56:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:45 smithi103 ceph-mon[35160]: pgmap v341: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 653 KiB/s wr, 71 op/s 2024-09-10T10:56:45.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:45 smithi007 ceph-mon[30873]: pgmap v341: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 653 KiB/s wr, 71 op/s 2024-09-10T10:56:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:45 smithi133 ceph-mon[35282]: pgmap v341: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 653 KiB/s wr, 71 op/s 2024-09-10T10:56:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.331+0000 7f1b7e718640 1 -- 172.21.15.7:0/157991828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b781069c0 msgr2=0x7f1b78106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.331+0000 7f1b7e718640 1 --2- 172.21.15.7:0/157991828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b781069c0 0x7f1b78106da0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009930 tx=0x7f1b6802a420 comp rx=0 tx=0).stop 2024-09-10T10:56:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.331+0000 7f1b7e718640 1 -- 172.21.15.7:0/157991828 shutdown_connections 2024-09-10T10:56:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.331+0000 7f1b7e718640 1 --2- 172.21.15.7:0/157991828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b78101350 0x7f1b7810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.331+0000 7f1b7e718640 1 --2- 172.21.15.7:0/157991828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b781009b0 0x7f1b78100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.331+0000 7f1b7e718640 1 --2- 172.21.15.7:0/157991828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b781069c0 0x7f1b78106da0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.331+0000 7f1b7e718640 1 -- 172.21.15.7:0/157991828 >> 172.21.15.7:0/157991828 conn(0x7f1b78077990 msgr2=0x7f1b780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.331+0000 7f1b7e718640 1 -- 172.21.15.7:0/157991828 shutdown_connections 2024-09-10T10:56:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.331+0000 7f1b7e718640 1 -- 172.21.15.7:0/157991828 wait complete. 2024-09-10T10:56:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b7e718640 1 Processor -- start 2024-09-10T10:56:46.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b7e718640 1 -- start start 2024-09-10T10:56:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b7e718640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b781009b0 0x7f1b78198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b7e718640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b78101350 0x7f1b78198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b7e718640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b781069c0 0x7f1b7819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b7e718640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78110dc0 con 0x7f1b78101350 2024-09-10T10:56:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b7e718640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78110c40 con 0x7f1b781069c0 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b7e718640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78110f40 con 0x7f1b781009b0 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b777fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b78101350 0x7f1b78198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b777fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b78101350 0x7f1b78198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46738/0 (socket says 172.21.15.7:46738) 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b777fe640 1 -- 172.21.15.7:0/2586625529 learned_addr learned my addr 172.21.15.7:0/2586625529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b77fff640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b781009b0 0x7f1b78198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.332+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b781069c0 0x7f1b7819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b777fe640 1 -- 172.21.15.7:0/2586625529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b781009b0 msgr2=0x7f1b78198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b777fe640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b781009b0 0x7f1b78198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b777fe640 1 -- 172.21.15.7:0/2586625529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b781069c0 msgr2=0x7f1b7819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b777fe640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b781069c0 0x7f1b7819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b777fe640 1 -- 172.21.15.7:0/2586625529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7819d2f0 con 0x7f1b78101350 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b77fff640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b781009b0 0x7f1b78198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:56:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b777fe640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b78101350 0x7f1b78198840 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00e9f0 tx=0x7f1b6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b757fa640 1 -- 172.21.15.7:0/2586625529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00cd80 con 0x7f1b78101350 2024-09-10T10:56:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b757fa640 1 -- 172.21.15.7:0/2586625529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b6c010910 con 0x7f1b78101350 2024-09-10T10:56:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b7e718640 1 -- 172.21.15.7:0/2586625529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7819d5e0 con 0x7f1b78101350 2024-09-10T10:56:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b757fa640 1 -- 172.21.15.7:0/2586625529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c018990 con 0x7f1b78101350 2024-09-10T10:56:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.333+0000 7f1b7e718640 1 -- 172.21.15.7:0/2586625529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b781a4e70 con 0x7f1b78101350 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.336+0000 7f1b814b1740 1 Processor -- start 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.336+0000 7f1b757fa640 1 -- 172.21.15.7:0/2586625529 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1b6c010ac0 con 0x7f1b78101350 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.336+0000 7f1b814b1740 1 -- start start 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.336+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a399b0 0x55e027a39d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.336+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a48b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.336+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a490c0 0x55e027a4b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.336+0000 7f1b814b1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e02799e2f0 con 0x55e027a399b0 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.336+0000 7f1b814b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e027946270 con 0x55e027a3a2d0 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.336+0000 7f1b814b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e02799a2b0 con 0x55e027a490c0 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b7ccb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a399b0 0x55e027a39d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b7ccb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a399b0 0x55e027a39d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46754/0 (socket says 172.21.15.7:46754) 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/839894476 learned_addr learned my addr 172.21.15.7:0/839894476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b77fff640 1 --2- 172.21.15.7:0/839894476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a48b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b777fe640 1 --2- 172.21.15.7:0/839894476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a490c0 0x55e027a4b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b757fa640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1b600a2da0 0x7f1b600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/839894476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a490c0 msgr2=0x55e027a4b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/839894476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a490c0 0x55e027a4b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/839894476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 msgr2=0x55e027a48b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/839894476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a48b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/839894476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e027945340 con 0x55e027a399b0 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b757fa640 1 -- 172.21.15.7:0/2586625529 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7f1b6c0cf770 con 0x7f1b78101350 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1b600a2da0 0x7f1b600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/839894476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a399b0 0x55e027a39d90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009f50 tx=0x7f1b700118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe17212a48ac2c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.337+0000 7f1b5e7fc640 1 -- 172.21.15.7:0/839894476 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70004250 con 0x55e027a399b0 2024-09-10T10:56:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.338+0000 7f1b5e7fc640 1 -- 172.21.15.7:0/839894476 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b7000fe80 con 0x55e027a399b0 2024-09-10T10:56:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.338+0000 7f1b5e7fc640 1 -- 172.21.15.7:0/839894476 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7001bbd0 con 0x55e027a399b0 2024-09-10T10:56:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.338+0000 7f1b814b1740 1 -- 172.21.15.7:0/839894476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a399b0 msgr2=0x55e027a39d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.338+0000 7f1b814b1740 1 --2- 172.21.15.7:0/839894476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a399b0 0x55e027a39d90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009f50 tx=0x7f1b700118d0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.338+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1b600a2da0 0x7f1b600a5260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1b70012fd0 tx=0x7f1b70021000 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.338+0000 7f1b814b1740 1 -- 172.21.15.7:0/839894476 shutdown_connections 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.338+0000 7f1b814b1740 1 --2- 172.21.15.7:0/839894476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a490c0 0x55e027a4b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.338+0000 7f1b814b1740 1 --2- 172.21.15.7:0/839894476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a48b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.338+0000 7f1b814b1740 1 --2- 172.21.15.7:0/839894476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a399b0 0x55e027a39d90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.338+0000 7f1b814b1740 1 -- 172.21.15.7:0/839894476 >> 172.21.15.7:0/839894476 conn(0x55e027a36060 msgr2=0x55e027a36440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.339+0000 7f1b814b1740 1 -- 172.21.15.7:0/839894476 shutdown_connections 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.339+0000 7f1b814b1740 1 -- 172.21.15.7:0/839894476 wait complete. 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.339+0000 7f1b814b1740 1 -- start start 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.339+0000 7f1b814b1740 1 Processor -- start 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a399b0 0x55e027a45fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a3a2d0 0x55e027a464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a46a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b814b1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e027a4bc90 con 0x55e027a490c0 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b814b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e027927290 con 0x55e027a399b0 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b814b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e027a4be10 con 0x55e027a3a2d0 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b814b1740 10 client.0 _ll_register_callbacks cb 0x55e027958140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b7ccb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a46a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b77fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a399b0 0x55e027a45fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b77fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a399b0 0x55e027a45fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:60414/0 (socket says 172.21.15.7:60414) 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b77fff640 1 -- 172.21.15.7:0/1354475901 learned_addr learned my addr 172.21.15.7:0/1354475901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b777fe640 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a3a2d0 0x55e027a464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b77fff640 1 -- 172.21.15.7:0/1354475901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a3a2d0 msgr2=0x55e027a464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b77fff640 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a3a2d0 0x55e027a464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b77fff640 1 -- 172.21.15.7:0/1354475901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 msgr2=0x55e027a46a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b77fff640 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a46a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b77fff640 1 -- 172.21.15.7:0/1354475901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0276b1210 con 0x55e027a399b0 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b77fff640 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a399b0 0x55e027a45fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009930 tx=0x7f1b68031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb13181558dac3cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.340+0000 7f1b577fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b577fe640 1 -- 172.21.15.7:0/1354475901 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68035040 con 0x55e027a399b0 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b814b1740 1 -- 172.21.15.7:0/1354475901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e0279aba20 con 0x55e027a399b0 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b577fe640 1 -- 172.21.15.7:0/1354475901 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7f1b68004250 con 0x55e027a399b0 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b577fe640 1 -- 172.21.15.7:0/1354475901 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1b68004450 con 0x55e027a399b0 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b814b1740 10 client.25757 fetch_fsmap learned FSMap version 40 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b814b1740 1 -- 172.21.15.7:0/1354475901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e02777d620 con 0x55e027a399b0 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b577fe640 1 -- 172.21.15.7:0/1354475901 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e0279aba20 con 0x55e027a399b0 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b577fe640 1 -- 172.21.15.7:0/1354475901 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f1b6803d780 con 0x55e027a399b0 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b814b1740 10 client.25757 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:56:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b814b1740 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b814b1740 20 client.25757 populate_metadata read hostname 'smithi007' 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e027a59060 0x55e027a5b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b814b1740 4 client.25757 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b814b1740 1 -- 172.21.15.7:0/1354475901 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0279a5d70 con 0x55e027a59060 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.341+0000 7f1b777fe640 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e027a59060 0x55e027a5b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.342+0000 7f1b777fe640 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e027a59060 0x55e027a5b470 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8701b672e70accc server_cookie=42dccb24cfc7b13b in_seq=0 out_seq=0 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.342+0000 7f1b577fe640 10 client.25757 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b577fe640 1 -- 172.21.15.7:0/1354475901 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0279a5d70 con 0x55e027a59060 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b577fe640 10 client.25757 handle_command_reply: tid=0 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 client.25757 shutdown 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 -- 172.21.15.7:0/1354475901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a399b0 msgr2=0x55e027a45fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a399b0 0x55e027a45fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009930 tx=0x7f1b68031850 comp rx=0 tx=0).stop 2024-09-10T10:56:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 -- 172.21.15.7:0/1354475901 shutdown_connections 2024-09-10T10:56:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e027a59060 0x55e027a5b470 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a46a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b577fe640 0 client.25757 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a3a2d0 0x55e027a464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1354475901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a399b0 0x55e027a45fb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 -- 172.21.15.7:0/1354475901 >> 172.21.15.7:0/1354475901 conn(0x55e027a36060 msgr2=0x55e027a351c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 -- 172.21.15.7:0/1354475901 shutdown_connections 2024-09-10T10:56:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 1 -- 172.21.15.7:0/1354475901 wait complete. 2024-09-10T10:56:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.344+0000 7f1b814b1740 20 client.25757 trim_cache size 0 max 16384 2024-09-10T10:56:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b814b1740 1 Processor -- start 2024-09-10T10:56:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b814b1740 1 -- start start 2024-09-10T10:56:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 0x55e027a55650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a60d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b814b1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781adf30 con 0x55e027a490c0 2024-09-10T10:56:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b814b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781ab7a0 con 0x55e027a3a2d0 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b814b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781a9610 con 0x55e027a399b0 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b7ccb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b7ccb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46772/0 (socket says 172.21.15.7:46772) 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/625509745 learned_addr learned my addr 172.21.15.7:0/625509745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b777fe640 1 --2- 172.21.15.7:0/625509745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a60d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b77fff640 1 --2- 172.21.15.7:0/625509745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 0x55e027a55650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/625509745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 msgr2=0x55e027a55650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/625509745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 0x55e027a55650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/625509745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 msgr2=0x55e027a60d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/625509745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a60d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.359+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/625509745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e027a44f40 con 0x55e027a490c0 2024-09-10T10:56:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.360+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/625509745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a612c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009f20 tx=0x7f1b70026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4fc7916232869f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.360+0000 7f1b5dffb640 1 -- 172.21.15.7:0/625509745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70034030 con 0x55e027a490c0 2024-09-10T10:56:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.360+0000 7f1b5dffb640 1 -- 172.21.15.7:0/625509745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b7001b430 con 0x55e027a490c0 2024-09-10T10:56:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.360+0000 7f1b5dffb640 1 -- 172.21.15.7:0/625509745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70003f70 con 0x55e027a490c0 2024-09-10T10:56:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.360+0000 7f1b814b1740 1 -- 172.21.15.7:0/625509745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 msgr2=0x55e027a612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.360+0000 7f1b814b1740 1 --2- 172.21.15.7:0/625509745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a612c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009f20 tx=0x7f1b70026000 comp rx=0 tx=0).stop 2024-09-10T10:56:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 -- 172.21.15.7:0/625509745 shutdown_connections 2024-09-10T10:56:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 --2- 172.21.15.7:0/625509745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a612c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 --2- 172.21.15.7:0/625509745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a60d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 --2- 172.21.15.7:0/625509745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 0x55e027a55650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 -- 172.21.15.7:0/625509745 >> 172.21.15.7:0/625509745 conn(0x55e027a36060 msgr2=0x55e027a48030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 -- 172.21.15.7:0/625509745 shutdown_connections 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 -- 172.21.15.7:0/625509745 wait complete. 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 -- start start 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 Processor -- start 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 0x55e027a518d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a4cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a4d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781a9c60 con 0x55e027a490c0 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781ab0d0 con 0x55e027a3a2d0 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.361+0000 7f1b814b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781aa330 con 0x55e027a399b0 2024-09-10T10:56:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b814b1740 10 client.0 _ll_register_callbacks cb 0x7f1b68009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b7ccb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a4d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b77fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 0x55e027a518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b7ccb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a4d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46788/0 (socket says 172.21.15.7:46788) 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/1398554641 learned_addr learned my addr 172.21.15.7:0/1398554641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b777fe640 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a4cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/1398554641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 msgr2=0x55e027a518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 0x55e027a518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/1398554641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 msgr2=0x55e027a4cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a4cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b7ccb5640 1 -- 172.21.15.7:0/1398554641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e027a4da60 con 0x55e027a490c0 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b77fff640 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 0x55e027a518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b7ccb5640 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a4d230 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1b70006010 tx=0x7f1b7000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8d7b2cbaf01030 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b577fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b577fe640 1 -- 172.21.15.7:0/1398554641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70034040 con 0x55e027a490c0 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b577fe640 1 -- 172.21.15.7:0/1398554641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7f1b70004100 con 0x55e027a490c0 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b814b1740 1 -- 172.21.15.7:0/1398554641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b7001b430 con 0x55e027a490c0 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b577fe640 1 -- 172.21.15.7:0/1398554641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1b70041430 con 0x55e027a490c0 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b814b1740 10 client.16131 fetch_fsmap learned FSMap version 40 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.362+0000 7f1b814b1740 1 -- 172.21.15.7:0/1398554641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e027ac3e40 con 0x55e027a490c0 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b577fe640 1 -- 172.21.15.7:0/1398554641 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b7001b430 con 0x55e027a490c0 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b577fe640 1 -- 172.21.15.7:0/1398554641 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f1b70041c70 con 0x55e027a490c0 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b814b1740 10 client.16131 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b814b1740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b814b1740 20 client.16131 populate_metadata read hostname 'smithi007' 2024-09-10T10:56:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e027ac4380 0x55e027ac67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b814b1740 4 client.16131 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:56:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b814b1740 1 -- 172.21.15.7:0/1398554641 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e027922900 con 0x55e027ac4380 2024-09-10T10:56:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b77fff640 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e027ac4380 0x55e027ac67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b577fe640 10 client.16131 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.363+0000 7f1b77fff640 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e027ac4380 0x55e027ac67a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7b4bca9f6cd46f3 server_cookie=955735a6cbaf2e3 in_seq=0 out_seq=0 2024-09-10T10:56:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b577fe640 1 -- 172.21.15.7:0/1398554641 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e027922900 con 0x55e027ac4380 2024-09-10T10:56:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b577fe640 10 client.16131 handle_command_reply: tid=0 2024-09-10T10:56:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 client.16131 shutdown 2024-09-10T10:56:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 -- 172.21.15.7:0/1398554641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 msgr2=0x55e027a4d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a4d230 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1b70006010 tx=0x7f1b7000fd10 comp rx=0 tx=0).stop 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 -- 172.21.15.7:0/1398554641 shutdown_connections 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55e027ac4380 0x55e027ac67a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e027a490c0 0x55e027a4d230 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e027a3a2d0 0x55e027a4cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 --2- 172.21.15.7:0/1398554641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e027a399b0 0x55e027a518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b577fe640 0 client.16131 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 -- 172.21.15.7:0/1398554641 >> 172.21.15.7:0/1398554641 conn(0x55e027a36060 msgr2=0x55e027a349a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 -- 172.21.15.7:0/1398554641 shutdown_connections 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 1 -- 172.21.15.7:0/1398554641 wait complete. 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b814b1740 20 client.16131 trim_cache size 0 max 16384 2024-09-10T10:56:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:56:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:56:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:56:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:56:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 -- 172.21.15.7:0/2586625529 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1b600a2da0 msgr2=0x7f1b600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1b600a2da0 0x7f1b600a5260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1b70012fd0 tx=0x7f1b70021000 comp rx=0 tx=0).stop 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 -- 172.21.15.7:0/2586625529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b78101350 msgr2=0x7f1b78198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b78101350 0x7f1b78198840 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00e9f0 tx=0x7f1b6c00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 -- 172.21.15.7:0/2586625529 shutdown_connections 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b781069c0 0x7f1b7819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1b600a2da0 0x7f1b600a5260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b78101350 0x7f1b78198840 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 --2- 172.21.15.7:0/2586625529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b781009b0 0x7f1b78198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 -- 172.21.15.7:0/2586625529 >> 172.21.15.7:0/2586625529 conn(0x7f1b78077990 msgr2=0x7f1b78069980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.408+0000 7f1b7e718640 1 -- 172.21.15.7:0/2586625529 shutdown_connections 2024-09-10T10:56:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:46.409+0000 7f1b7e718640 1 -- 172.21.15.7:0/2586625529 wait complete. 2024-09-10T10:56:46.536 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1013295 8.45 MB/sec execute 466 sec latency 1401.919 ms 2024-09-10T10:56:47.536 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1013295 8.43 MB/sec execute 467 sec latency 1178.438 ms 2024-09-10T10:56:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:47 smithi103 ceph-mon[35160]: pgmap v342: 97 pgs: 97 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-09-10T10:56:47.936 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:47 smithi007 ceph-mon[30873]: pgmap v342: 97 pgs: 97 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-09-10T10:56:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:47 smithi133 ceph-mon[35282]: pgmap v342: 97 pgs: 97 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-09-10T10:56:48.536 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1013295 8.41 MB/sec execute 468 sec latency 2178.533 ms 2024-09-10T10:56:49.536 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1013295 8.40 MB/sec execute 469 sec latency 3178.634 ms 2024-09-10T10:56:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:49 smithi103 ceph-mon[35160]: pgmap v343: 97 pgs: 97 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-10T10:56:49.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:49 smithi007 ceph-mon[30873]: pgmap v343: 97 pgs: 97 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-10T10:56:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:49 smithi133 ceph-mon[35282]: pgmap v343: 97 pgs: 97 active+clean; 270 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-10T10:56:50.536 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1013295 8.38 MB/sec execute 470 sec latency 4178.764 ms 2024-09-10T10:56:51.536 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1013736 8.36 MB/sec execute 471 sec latency 4905.576 ms 2024-09-10T10:56:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:51 smithi103 ceph-mon[35160]: pgmap v344: 97 pgs: 97 active+clean; 266 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-09-10T10:56:51.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:51 smithi007 ceph-mon[30873]: pgmap v344: 97 pgs: 97 active+clean; 266 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-09-10T10:56:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:51 smithi133 ceph-mon[35282]: pgmap v344: 97 pgs: 97 active+clean; 266 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-09-10T10:56:52.536 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1016266 8.38 MB/sec execute 472 sec latency 16.455 ms 2024-09-10T10:56:53.536 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1016716 8.37 MB/sec execute 473 sec latency 540.739 ms 2024-09-10T10:56:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:53 smithi103 ceph-mon[35160]: pgmap v345: 97 pgs: 97 active+clean; 266 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T10:56:53.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:53 smithi007 ceph-mon[30873]: pgmap v345: 97 pgs: 97 active+clean; 266 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T10:56:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:53 smithi133 ceph-mon[35282]: pgmap v345: 97 pgs: 97 active+clean; 266 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T10:56:54.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1016716 8.35 MB/sec execute 474 sec latency 1540.864 ms 2024-09-10T10:56:55.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1016716 8.33 MB/sec execute 475 sec latency 2540.938 ms 2024-09-10T10:56:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:55 smithi103 ceph-mon[35160]: pgmap v346: 97 pgs: 97 active+clean; 266 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T10:56:55.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:55 smithi007 ceph-mon[30873]: pgmap v346: 97 pgs: 97 active+clean; 266 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T10:56:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:55 smithi133 ceph-mon[35282]: pgmap v346: 97 pgs: 97 active+clean; 266 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T10:56:56.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1016772 8.31 MB/sec execute 476 sec latency 3267.629 ms 2024-09-10T10:56:57.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1016772 8.30 MB/sec execute 477 sec latency 1232.364 ms 2024-09-10T10:56:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:57 smithi103 ceph-mon[35160]: pgmap v347: 97 pgs: 97 active+clean; 269 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-10T10:56:57.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:57 smithi007 ceph-mon[30873]: pgmap v347: 97 pgs: 97 active+clean; 269 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-10T10:56:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:57 smithi133 ceph-mon[35282]: pgmap v347: 97 pgs: 97 active+clean; 269 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-09-10T10:56:58.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1016772 8.28 MB/sec execute 478 sec latency 2232.468 ms 2024-09-10T10:56:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.509+0000 7f721c3d6640 1 -- 172.21.15.7:0/4214134907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72141035e0 msgr2=0x7f721410bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.509+0000 7f721c3d6640 1 --2- 172.21.15.7:0/4214134907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72141035e0 0x7f721410bac0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7204009900 tx=0x7f720402a480 comp rx=0 tx=0).stop 2024-09-10T10:56:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.509+0000 7f721c3d6640 1 -- 172.21.15.7:0/4214134907 shutdown_connections 2024-09-10T10:56:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.509+0000 7f721c3d6640 1 --2- 172.21.15.7:0/4214134907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f721410c110 0x7f721410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.509+0000 7f721c3d6640 1 --2- 172.21.15.7:0/4214134907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72141035e0 0x7f721410bac0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.509+0000 7f721c3d6640 1 --2- 172.21.15.7:0/4214134907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7214102cc0 0x7f72141030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.509+0000 7f721c3d6640 1 -- 172.21.15.7:0/4214134907 >> 172.21.15.7:0/4214134907 conn(0x7f72140fc6b0 msgr2=0x7f72140fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.509+0000 7f721c3d6640 1 -- 172.21.15.7:0/4214134907 shutdown_connections 2024-09-10T10:56:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.509+0000 7f721c3d6640 1 -- 172.21.15.7:0/4214134907 wait complete. 2024-09-10T10:56:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721c3d6640 1 Processor -- start 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721c3d6640 1 -- start start 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721c3d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7214102cc0 0x7f7214198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721c3d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72141035e0 0x7f72141989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721c3d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f721410c110 0x7f721419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721c3d6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7214111ef0 con 0x7f7214102cc0 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721c3d6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7214111d70 con 0x7f72141035e0 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721c3d6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7214112070 con 0x7f721410c110 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721a172640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7214102cc0 0x7f7214198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721a172640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7214102cc0 0x7f7214198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51706/0 (socket says 172.21.15.7:51706) 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721a172640 1 -- 172.21.15.7:0/1906609284 learned_addr learned my addr 172.21.15.7:0/1906609284 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f7219971640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72141035e0 0x7f72141989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721a973640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f721410c110 0x7f721419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.510+0000 7f721a172640 1 -- 172.21.15.7:0/1906609284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f721410c110 msgr2=0x7f721419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f721a172640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f721410c110 0x7f721419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f721a172640 1 -- 172.21.15.7:0/1906609284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72141035e0 msgr2=0x7f72141989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f721a172640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72141035e0 0x7f72141989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f721a172640 1 -- 172.21.15.7:0/1906609284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721419d450 con 0x7f7214102cc0 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f7219971640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72141035e0 0x7f72141989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f721a172640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7214102cc0 0x7f7214198460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f721000e9f0 tx=0x7f721000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f720b7fe640 1 -- 172.21.15.7:0/1906609284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721000cd80 con 0x7f7214102cc0 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f720b7fe640 1 -- 172.21.15.7:0/1906609284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7210010910 con 0x7f7214102cc0 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f721c3d6640 1 -- 172.21.15.7:0/1906609284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721419d740 con 0x7f7214102cc0 2024-09-10T10:56:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f720b7fe640 1 -- 172.21.15.7:0/1906609284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210018990 con 0x7f7214102cc0 2024-09-10T10:56:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.511+0000 7f721c3d6640 1 -- 172.21.15.7:0/1906609284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72141a4fd0 con 0x7f7214102cc0 2024-09-10T10:56:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.514+0000 7f720b7fe640 1 -- 172.21.15.7:0/1906609284 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7210010ac0 con 0x7f7214102cc0 2024-09-10T10:56:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.514+0000 7f721f253740 1 Processor -- start 2024-09-10T10:56:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.514+0000 7f721f253740 1 -- start start 2024-09-10T10:56:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.514+0000 7f721f253740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 0x559d727fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.514+0000 7f721f253740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f721f253740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 0x559d728104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f721f253740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559d727632f0 con 0x559d727ff2f0 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f721f253740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559d7270b270 con 0x559d7280e0c0 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f721f253740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559d7275f2b0 con 0x559d727fe9d0 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f721a172640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f721a172640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51708/0 (socket says 172.21.15.7:51708) 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f720b7fe640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72000a2e20 0x7f72000a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f721a172640 1 -- 172.21.15.7:0/2088173973 learned_addr learned my addr 172.21.15.7:0/2088173973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f7219971640 1 --2- 172.21.15.7:0/2088173973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 0x559d728104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f721a973640 1 --2- 172.21.15.7:0/2088173973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 0x559d727fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f721a973640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72000a2e20 0x7f72000a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f720b7fe640 1 -- 172.21.15.7:0/1906609284 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7f72100cf850 con 0x7f7214102cc0 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f7219971640 1 -- 172.21.15.7:0/2088173973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 msgr2=0x559d727fedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f7219971640 1 --2- 172.21.15.7:0/2088173973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 0x559d727fedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f7219971640 1 -- 172.21.15.7:0/2088173973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 msgr2=0x559d7280db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f7219971640 1 --2- 172.21.15.7:0/2088173973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f7219971640 1 -- 172.21.15.7:0/2088173973 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7270a340 con 0x559d7280e0c0 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.515+0000 7f721a172640 1 --2- 172.21.15.7:0/2088173973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.516+0000 7f721a973640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72000a2e20 0x7f72000a52e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f720c005c40 tx=0x7f720c005bd0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.516+0000 7f7219971640 1 --2- 172.21.15.7:0/2088173973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 0x559d728104b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7204009230 tx=0x7f7204005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc09b1205073a2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.516+0000 7f7208bfe640 1 -- 172.21.15.7:0/2088173973 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204035030 con 0x559d7280e0c0 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.516+0000 7f7208bfe640 1 -- 172.21.15.7:0/2088173973 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7204004030 con 0x559d7280e0c0 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.516+0000 7f7208bfe640 1 -- 172.21.15.7:0/2088173973 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040055a0 con 0x559d7280e0c0 2024-09-10T10:56:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 -- 172.21.15.7:0/2088173973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 msgr2=0x559d728104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 --2- 172.21.15.7:0/2088173973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 0x559d728104b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7204009230 tx=0x7f7204005e60 comp rx=0 tx=0).stop 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 -- 172.21.15.7:0/2088173973 shutdown_connections 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 --2- 172.21.15.7:0/2088173973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 0x559d728104b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 --2- 172.21.15.7:0/2088173973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 --2- 172.21.15.7:0/2088173973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 0x559d727fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 -- 172.21.15.7:0/2088173973 >> 172.21.15.7:0/2088173973 conn(0x559d727fb060 msgr2=0x559d727fb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 -- 172.21.15.7:0/2088173973 shutdown_connections 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 -- 172.21.15.7:0/2088173973 wait complete. 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 -- start start 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 Processor -- start 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d727fe9d0 0x559d7280afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d7280e0c0 0x559d7280ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559d72810c90 con 0x559d727ff2f0 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559d726ec290 con 0x559d727fe9d0 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559d72810e10 con 0x559d7280e0c0 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.517+0000 7f721f253740 10 client.0 _ll_register_callbacks cb 0x559d7271d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f7219971640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f7219971640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51722/0 (socket says 172.21.15.7:51722) 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f7219971640 1 -- 172.21.15.7:0/716975612 learned_addr learned my addr 172.21.15.7:0/716975612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f721a172640 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d727fe9d0 0x559d7280afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f721a973640 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d7280e0c0 0x559d7280ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f7219971640 1 -- 172.21.15.7:0/716975612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d7280e0c0 msgr2=0x559d7280ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f7219971640 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d7280e0c0 0x559d7280ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f7219971640 1 -- 172.21.15.7:0/716975612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d727fe9d0 msgr2=0x559d7280afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f7219971640 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d727fe9d0 0x559d7280afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f7219971640 1 -- 172.21.15.7:0/716975612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d72476210 con 0x559d727ff2f0 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f7219971640 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280b520 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7204031480 tx=0x7f72040056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f672aa176a330039 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f71fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f71fdffb640 1 -- 172.21.15.7:0/716975612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720404b070 con 0x559d727ff2f0 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f71fdffb640 1 -- 172.21.15.7:0/716975612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7f72040315b0 con 0x559d727ff2f0 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.518+0000 7f721f253740 1 -- 172.21.15.7:0/716975612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7204004030 con 0x559d727ff2f0 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f71fdffb640 1 -- 172.21.15.7:0/716975612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7204057050 con 0x559d727ff2f0 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f721f253740 10 client.16143 fetch_fsmap learned FSMap version 41 2024-09-10T10:56:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f721f253740 1 -- 172.21.15.7:0/716975612 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559d72542620 con 0x559d727ff2f0 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f71fdffb640 1 -- 172.21.15.7:0/716975612 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7204004030 con 0x559d727ff2f0 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f71fdffb640 1 -- 172.21.15.7:0/716975612 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f72040035f0 con 0x559d727ff2f0 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f721f253740 10 client.16143 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f721f253740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f721f253740 20 client.16143 populate_metadata read hostname 'smithi007' 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f721f253740 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x559d7281dee0 0x559d728202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f721f253740 4 client.16143 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f721f253740 1 -- 172.21.15.7:0/716975612 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d72770a20 con 0x559d7281dee0 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.519+0000 7f721a172640 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x559d7281dee0 0x559d728202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.520+0000 7f71fdffb640 10 client.16143 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.520+0000 7f721a172640 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x559d7281dee0 0x559d728202d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8e48f6c67cb9b server_cookie=300a676bb5ee9c26 in_seq=0 out_seq=0 2024-09-10T10:56:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f71fdffb640 1 -- 172.21.15.7:0/716975612 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d72770a20 con 0x559d7281dee0 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f71fdffb640 10 client.16143 handle_command_reply: tid=0 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 client.16143 shutdown 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 -- 172.21.15.7:0/716975612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 msgr2=0x559d7280b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280b520 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7204031480 tx=0x7f72040056d0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 -- 172.21.15.7:0/716975612 shutdown_connections 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x559d7281dee0 0x559d728202d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d7280e0c0 0x559d7280ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f71fdffb640 0 client.16143 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d7280b520 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 --2- 172.21.15.7:0/716975612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d727fe9d0 0x559d7280afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 -- 172.21.15.7:0/716975612 >> 172.21.15.7:0/716975612 conn(0x559d727fb060 msgr2=0x559d727fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 -- 172.21.15.7:0/716975612 shutdown_connections 2024-09-10T10:56:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 1 -- 172.21.15.7:0/716975612 wait complete. 2024-09-10T10:56:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.521+0000 7f721f253740 20 client.16143 trim_cache size 0 max 16384 2024-09-10T10:56:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721f253740 1 Processor -- start 2024-09-10T10:56:59.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1016772 8.26 MB/sec execute 479 sec latency 3232.513 ms 2024-09-10T10:56:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721f253740 1 -- start start 2024-09-10T10:56:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721f253740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 0x559d7280cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721f253740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d728259f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721f253740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 0x559d72825f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721f253740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72141addc0 con 0x559d727ff2f0 2024-09-10T10:56:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721f253740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72141ab660 con 0x559d7280e0c0 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721f253740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72141a94d0 con 0x559d727fe9d0 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f7219971640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d728259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f7219971640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d728259f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51738/0 (socket says 172.21.15.7:51738) 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f7219971640 1 -- 172.21.15.7:0/1652417695 learned_addr learned my addr 172.21.15.7:0/1652417695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f7219971640 1 -- 172.21.15.7:0/1652417695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 msgr2=0x559d7280cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721a172640 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 0x559d7280cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f7219971640 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 0x559d7280cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f7219971640 1 -- 172.21.15.7:0/1652417695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 msgr2=0x559d72825f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721a973640 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 0x559d72825f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f7219971640 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 0x559d72825f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f7219971640 1 -- 172.21.15.7:0/1652417695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d727ff0e0 con 0x559d727ff2f0 2024-09-10T10:56:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721a172640 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 0x559d7280cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.536+0000 7f721a973640 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 0x559d72825f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f7219971640 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d728259f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f72040314d0 tx=0x7f7204001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d15e0767a31e204 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f71fffff640 1 -- 172.21.15.7:0/1652417695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720404b070 con 0x559d727ff2f0 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f71fffff640 1 -- 172.21.15.7:0/1652417695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72040035f0 con 0x559d727ff2f0 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f71fffff640 1 -- 172.21.15.7:0/1652417695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204035040 con 0x559d727ff2f0 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f721f253740 1 -- 172.21.15.7:0/1652417695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 msgr2=0x559d728259f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f721f253740 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d728259f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f72040314d0 tx=0x7f7204001e70 comp rx=0 tx=0).stop 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f721f253740 1 -- 172.21.15.7:0/1652417695 shutdown_connections 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f721f253740 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d7280e0c0 0x559d72825f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f721f253740 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d728259f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f721f253740 1 --2- 172.21.15.7:0/1652417695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d727fe9d0 0x559d7280cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f721f253740 1 -- 172.21.15.7:0/1652417695 >> 172.21.15.7:0/1652417695 conn(0x559d727fb060 msgr2=0x559d7280c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f721f253740 1 -- 172.21.15.7:0/1652417695 shutdown_connections 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.537+0000 7f721f253740 1 -- 172.21.15.7:0/1652417695 wait complete. 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721f253740 1 -- start start 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721f253740 1 Processor -- start 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721f253740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d727fe9d0 0x559d728168d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721f253740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d72811bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721f253740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d7280e0c0 0x559d728121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721f253740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72141a9b20 con 0x559d727ff2f0 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721f253740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72141aaf90 con 0x559d727fe9d0 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721f253740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72141aa1f0 con 0x559d7280e0c0 2024-09-10T10:56:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721f253740 10 client.0 _ll_register_callbacks cb 0x7f721009bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f7219971640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d72811bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f7219971640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d72811bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51746/0 (socket says 172.21.15.7:51746) 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f7219971640 1 -- 172.21.15.7:0/114294529 learned_addr learned my addr 172.21.15.7:0/114294529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721a172640 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d727fe9d0 0x559d728168d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f721a973640 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d7280e0c0 0x559d728121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f7219971640 1 -- 172.21.15.7:0/114294529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d7280e0c0 msgr2=0x559d728121e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f7219971640 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d7280e0c0 0x559d728121e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f7219971640 1 -- 172.21.15.7:0/114294529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d727fe9d0 msgr2=0x559d728168d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f7219971640 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d727fe9d0 0x559d728168d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.538+0000 7f7219971640 1 -- 172.21.15.7:0/114294529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d72812a10 con 0x559d727ff2f0 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f7219971640 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d72811bf0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7204003a40 tx=0x7f7204005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ff9a9093fe0937 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f71fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f71fdffb640 1 -- 172.21.15.7:0/114294529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204035070 con 0x559d727ff2f0 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f71fdffb640 1 -- 172.21.15.7:0/114294529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8564+0+0 (secure 0 0 0) 0x7f72040317b0 con 0x559d727ff2f0 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f721f253740 1 -- 172.21.15.7:0/114294529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72040035f0 con 0x559d727ff2f0 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f71fdffb640 1 -- 172.21.15.7:0/114294529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f720404b040 con 0x559d727ff2f0 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f721f253740 10 client.16149 fetch_fsmap learned FSMap version 41 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f721f253740 1 -- 172.21.15.7:0/114294529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559d72888a50 con 0x559d727ff2f0 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f71fdffb640 1 -- 172.21.15.7:0/114294529 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72040035f0 con 0x559d727ff2f0 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f71fdffb640 1 -- 172.21.15.7:0/114294529 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f7204004ac0 con 0x559d727ff2f0 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f721f253740 10 client.16149 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f721f253740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f721f253740 20 client.16149 populate_metadata read hostname 'smithi007' 2024-09-10T10:56:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f721f253740 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x559d72889130 0x559d7288b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f721f253740 4 client.16149 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:56:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.539+0000 7f721f253740 1 -- 172.21.15.7:0/114294529 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d7276ad70 con 0x559d72889130 2024-09-10T10:56:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.540+0000 7f721a172640 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x559d72889130 0x559d7288b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.540+0000 7f721a172640 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x559d72889130 0x559d7288b550 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89627c311c3017c server_cookie=792ace45edef36c4 in_seq=0 out_seq=0 2024-09-10T10:56:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.540+0000 7f71fdffb640 10 client.16149 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f71fdffb640 1 -- 172.21.15.7:0/114294529 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d7276ad70 con 0x559d72889130 2024-09-10T10:56:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f71fdffb640 10 client.16149 handle_command_reply: tid=0 2024-09-10T10:56:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f721f253740 1 client.16149 shutdown 2024-09-10T10:56:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f721f253740 1 -- 172.21.15.7:0/114294529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 msgr2=0x559d72811bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f721f253740 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d72811bf0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7204003a40 tx=0x7f7204005310 comp rx=0 tx=0).stop 2024-09-10T10:56:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f721f253740 1 -- 172.21.15.7:0/114294529 shutdown_connections 2024-09-10T10:56:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f721f253740 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x559d72889130 0x559d7288b550 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f721f253740 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d7280e0c0 0x559d728121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f721f253740 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d727ff2f0 0x559d72811bf0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f71fdffb640 0 client.16149 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:56:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f721f253740 1 --2- 172.21.15.7:0/114294529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d727fe9d0 0x559d728168d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.573+0000 7f721f253740 1 -- 172.21.15.7:0/114294529 >> 172.21.15.7:0/114294529 conn(0x559d727fb060 msgr2=0x559d727fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721f253740 1 -- 172.21.15.7:0/114294529 shutdown_connections 2024-09-10T10:56:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721f253740 1 -- 172.21.15.7:0/114294529 wait complete. 2024-09-10T10:56:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721f253740 20 client.16149 trim_cache size 0 max 16384 2024-09-10T10:56:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:56:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:56:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:56:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:56:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 -- 172.21.15.7:0/1906609284 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72000a2e20 msgr2=0x7f72000a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72000a2e20 0x7f72000a52e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f720c005c40 tx=0x7f720c005bd0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 -- 172.21.15.7:0/1906609284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7214102cc0 msgr2=0x7f7214198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7214102cc0 0x7f7214198460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f721000e9f0 tx=0x7f721000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 -- 172.21.15.7:0/1906609284 shutdown_connections 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f721410c110 0x7f721419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72141035e0 0x7f72141989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f72000a2e20 0x7f72000a52e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 --2- 172.21.15.7:0/1906609284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7214102cc0 0x7f7214198460 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 -- 172.21.15.7:0/1906609284 >> 172.21.15.7:0/1906609284 conn(0x7f72140fc6b0 msgr2=0x7f72140fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 -- 172.21.15.7:0/1906609284 shutdown_connections 2024-09-10T10:56:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:56:59.574+0000 7f721c3d6640 1 -- 172.21.15.7:0/1906609284 wait complete. 2024-09-10T10:56:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:59 smithi103 ceph-mon[35160]: pgmap v348: 97 pgs: 97 active+clean; 269 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 848 KiB/s wr, 79 op/s 2024-09-10T10:56:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:56:59 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:56:59.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:59 smithi007 ceph-mon[30873]: pgmap v348: 97 pgs: 97 active+clean; 269 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 848 KiB/s wr, 79 op/s 2024-09-10T10:56:59.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:56:59 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:56:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:59 smithi133 ceph-mon[35282]: pgmap v348: 97 pgs: 97 active+clean; 269 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 848 KiB/s wr, 79 op/s 2024-09-10T10:56:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:56:59 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:57:00.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1017018 8.24 MB/sec execute 480 sec latency 4049.717 ms 2024-09-10T10:57:01.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1018527 8.24 MB/sec execute 481 sec latency 27.064 ms 2024-09-10T10:57:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:01 smithi103 ceph-mon[35160]: pgmap v349: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 921 KiB/s wr, 103 op/s 2024-09-10T10:57:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:01 smithi007 ceph-mon[30873]: pgmap v349: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 921 KiB/s wr, 103 op/s 2024-09-10T10:57:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:01 smithi133 ceph-mon[35282]: pgmap v349: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 921 KiB/s wr, 103 op/s 2024-09-10T10:57:02.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1020259 8.25 MB/sec execute 482 sec latency 11.595 ms 2024-09-10T10:57:03.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1020264 8.23 MB/sec execute 483 sec latency 998.295 ms 2024-09-10T10:57:03.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:03 smithi103 ceph-mon[35160]: pgmap v350: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 611 KiB/s wr, 67 op/s 2024-09-10T10:57:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:03 smithi007 ceph-mon[30873]: pgmap v350: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 611 KiB/s wr, 67 op/s 2024-09-10T10:57:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:03 smithi133 ceph-mon[35282]: pgmap v350: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 611 KiB/s wr, 67 op/s 2024-09-10T10:57:04.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1020264 8.21 MB/sec execute 484 sec latency 1998.386 ms 2024-09-10T10:57:04.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:04 smithi103 ceph-mon[35160]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T10:57:04.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:04 smithi007 ceph-mon[30873]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T10:57:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:04 smithi133 ceph-mon[35282]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T10:57:05.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1020264 8.20 MB/sec execute 485 sec latency 2998.499 ms 2024-09-10T10:57:05.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:05 smithi103 ceph-mon[35160]: pgmap v352: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 734 KiB/s wr, 80 op/s 2024-09-10T10:57:05.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:05 smithi007 ceph-mon[30873]: pgmap v352: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 734 KiB/s wr, 80 op/s 2024-09-10T10:57:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:05 smithi133 ceph-mon[35282]: pgmap v352: 97 pgs: 97 active+clean; 262 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 734 KiB/s wr, 80 op/s 2024-09-10T10:57:06.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1020320 8.18 MB/sec execute 486 sec latency 3724.141 ms 2024-09-10T10:57:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:57:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:57:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:57:06.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:57:06.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:57:06.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:57:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:57:06.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:57:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:57:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:57:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:57:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:57:07.537 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1020320 8.16 MB/sec execute 487 sec latency 1237.627 ms 2024-09-10T10:57:07.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:07 smithi103 ceph-mon[35160]: pgmap v353: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 628 KiB/s wr, 115 op/s 2024-09-10T10:57:07.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:07 smithi007 ceph-mon[30873]: pgmap v353: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 628 KiB/s wr, 115 op/s 2024-09-10T10:57:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:07 smithi133 ceph-mon[35282]: pgmap v353: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 628 KiB/s wr, 115 op/s 2024-09-10T10:57:08.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1020320 8.15 MB/sec execute 488 sec latency 2237.723 ms 2024-09-10T10:57:09.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1020320 8.13 MB/sec execute 489 sec latency 3237.863 ms 2024-09-10T10:57:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:09 smithi103 ceph-mon[35160]: pgmap v354: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 628 KiB/s wr, 115 op/s 2024-09-10T10:57:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:09 smithi103 ceph-mon[35160]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:57:09.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:09 smithi007 ceph-mon[30873]: pgmap v354: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 628 KiB/s wr, 115 op/s 2024-09-10T10:57:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:09 smithi007 ceph-mon[30873]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:57:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:09 smithi133 ceph-mon[35282]: pgmap v354: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 628 KiB/s wr, 115 op/s 2024-09-10T10:57:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:09 smithi133 ceph-mon[35282]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:57:10.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1020320 8.11 MB/sec execute 490 sec latency 4238.004 ms 2024-09-10T10:57:11.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1020776 8.10 MB/sec execute 491 sec latency 4961.179 ms 2024-09-10T10:57:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:11 smithi103 ceph-mon[35160]: pgmap v356: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 800 KiB/s wr, 131 op/s 2024-09-10T10:57:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:11 smithi007 ceph-mon[30873]: pgmap v356: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 800 KiB/s wr, 131 op/s 2024-09-10T10:57:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:11 smithi133 ceph-mon[35282]: pgmap v356: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 800 KiB/s wr, 131 op/s 2024-09-10T10:57:12.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1023363 8.12 MB/sec execute 492 sec latency 13.440 ms 2024-09-10T10:57:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.671+0000 7f29f5f42640 1 -- 172.21.15.7:0/2104426291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f0101240 msgr2=0x7f29f010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.671+0000 7f29f5f42640 1 --2- 172.21.15.7:0/2104426291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f0101240 0x7f29f010de70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009900 tx=0x7f29d802a410 comp rx=0 tx=0).stop 2024-09-10T10:57:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.671+0000 7f29f5f42640 1 -- 172.21.15.7:0/2104426291 shutdown_connections 2024-09-10T10:57:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.671+0000 7f29f5f42640 1 --2- 172.21.15.7:0/2104426291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29f010e440 0x7f29f0110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.671+0000 7f29f5f42640 1 --2- 172.21.15.7:0/2104426291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f0101240 0x7f29f010de70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.671+0000 7f29f5f42640 1 --2- 172.21.15.7:0/2104426291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f0100890 0x7f29f0100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.671+0000 7f29f5f42640 1 -- 172.21.15.7:0/2104426291 >> 172.21.15.7:0/2104426291 conn(0x7f29f0077990 msgr2=0x7f29f00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.671+0000 7f29f5f42640 1 -- 172.21.15.7:0/2104426291 shutdown_connections 2024-09-10T10:57:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.671+0000 7f29f5f42640 1 -- 172.21.15.7:0/2104426291 wait complete. 2024-09-10T10:57:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29f5f42640 1 Processor -- start 2024-09-10T10:57:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29f5f42640 1 -- start start 2024-09-10T10:57:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29f5f42640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29f0100890 0x7f29f019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29f5f42640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f0101240 0x7f29f019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29f5f42640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f010e440 0x7f29f019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29f5f42640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01140a0 con 0x7f29f010e440 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29f5f42640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0113f20 con 0x7f29f0101240 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29f5f42640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0114220 con 0x7f29f0100890 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29effff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f010e440 0x7f29f019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29ef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29f0100890 0x7f29f019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29ef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29f0100890 0x7f29f019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:43976/0 (socket says 172.21.15.7:43976) 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29effff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f010e440 0x7f29f019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53348/0 (socket says 172.21.15.7:53348) 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29ef7fe640 1 -- 172.21.15.7:0/1707906632 learned_addr learned my addr 172.21.15.7:0/1707906632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.672+0000 7f29e6ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f0101240 0x7f29f019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29effff640 1 -- 172.21.15.7:0/1707906632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29f0100890 msgr2=0x7f29f019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29effff640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29f0100890 0x7f29f019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29effff640 1 -- 172.21.15.7:0/1707906632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f0101240 msgr2=0x7f29f019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29effff640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f0101240 0x7f29f019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29effff640 1 -- 172.21.15.7:0/1707906632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f019f560 con 0x7f29f010e440 2024-09-10T10:57:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29effff640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f010e440 0x7f29f019ee30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f29e8005fd0 tx=0x7f29e800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29ed7fa640 1 -- 172.21.15.7:0/1707906632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e800cdf0 con 0x7f29f010e440 2024-09-10T10:57:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29ed7fa640 1 -- 172.21.15.7:0/1707906632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e8005020 con 0x7f29f010e440 2024-09-10T10:57:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29ed7fa640 1 -- 172.21.15.7:0/1707906632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80106d0 con 0x7f29f010e440 2024-09-10T10:57:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29f5f42640 1 -- 172.21.15.7:0/1707906632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f019f850 con 0x7f29f010e440 2024-09-10T10:57:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.673+0000 7f29f5f42640 1 -- 172.21.15.7:0/1707906632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f01a70e0 con 0x7f29f010e440 2024-09-10T10:57:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.676+0000 7f29ed7fa640 1 -- 172.21.15.7:0/1707906632 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f29e80054d0 con 0x7f29f010e440 2024-09-10T10:57:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.676+0000 7f29f8cda740 1 Processor -- start 2024-09-10T10:57:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.676+0000 7f29f8cda740 1 -- start start 2024-09-10T10:57:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.676+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.676+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813dc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.676+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813dccb0 0x55fe813df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.676+0000 7f29f8cda740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fe813362f0 con 0x55fe813d1af0 2024-09-10T10:57:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.676+0000 7f29f8cda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fe812de270 con 0x55fe813d2410 2024-09-10T10:57:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.676+0000 7f29f8cda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fe813322b0 con 0x55fe813dccb0 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53350/0 (socket says 172.21.15.7:53350) 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e6ffd640 1 -- 172.21.15.7:0/1777323304 learned_addr learned my addr 172.21.15.7:0/1777323304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/1777323304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813dccb0 0x55fe813df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29ef7fe640 1 --2- 172.21.15.7:0/1777323304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813dc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29ed7fa640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f29dc0a2da0 0x7f29dc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e6ffd640 1 -- 172.21.15.7:0/1777323304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813dccb0 msgr2=0x55fe813df0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/1777323304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813dccb0 0x55fe813df0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e6ffd640 1 -- 172.21.15.7:0/1777323304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 msgr2=0x55fe813dc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/1777323304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813dc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e6ffd640 1 -- 172.21.15.7:0/1777323304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe812dd340 con 0x55fe813d1af0 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29effff640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f29dc0a2da0 0x7f29dc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/1777323304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d1ed0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009900 tx=0x7f29d8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c316a354fa46f50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29ed7fa640 1 -- 172.21.15.7:0/1707906632 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f29e80cfb30 con 0x7f29f010e440 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.677+0000 7f29e5ffb640 1 -- 172.21.15.7:0/1777323304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8035040 con 0x55fe813d1af0 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29e5ffb640 1 -- 172.21.15.7:0/1777323304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d8005500 con 0x55fe813d1af0 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29e5ffb640 1 -- 172.21.15.7:0/1777323304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8004bc0 con 0x55fe813d1af0 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29f8cda740 1 -- 172.21.15.7:0/1777323304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 msgr2=0x55fe813d1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29effff640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f29dc0a2da0 0x7f29dc0a5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f29e80cd300 tx=0x7f29e80bc040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29f8cda740 1 --2- 172.21.15.7:0/1777323304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d1ed0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009900 tx=0x7f29d8000bc0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29f8cda740 1 -- 172.21.15.7:0/1777323304 shutdown_connections 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29f8cda740 1 --2- 172.21.15.7:0/1777323304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813dccb0 0x55fe813df0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29f8cda740 1 --2- 172.21.15.7:0/1777323304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813dc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29f8cda740 1 --2- 172.21.15.7:0/1777323304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d1ed0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29f8cda740 1 -- 172.21.15.7:0/1777323304 >> 172.21.15.7:0/1777323304 conn(0x55fe813ce060 msgr2=0x55fe813ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29f8cda740 1 -- 172.21.15.7:0/1777323304 shutdown_connections 2024-09-10T10:57:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.678+0000 7f29f8cda740 1 -- 172.21.15.7:0/1777323304 wait complete. 2024-09-10T10:57:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29f8cda740 1 -- start start 2024-09-10T10:57:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29f8cda740 1 Processor -- start 2024-09-10T10:57:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813da0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813dccb0 0x55fe813da610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29f8cda740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fe813df880 con 0x55fe813d1af0 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29f8cda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fe812bf290 con 0x55fe813d2410 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29f8cda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fe813dfa00 con 0x55fe813dccb0 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29f8cda740 10 client.0 _ll_register_callbacks cb 0x55fe812f0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53366/0 (socket says 172.21.15.7:53366) 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29ef7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813da0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.679+0000 7f29e6ffd640 1 -- 172.21.15.7:0/2641467286 learned_addr learned my addr 172.21.15.7:0/2641467286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813dccb0 0x55fe813da610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29e6ffd640 1 -- 172.21.15.7:0/2641467286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813dccb0 msgr2=0x55fe813da610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813dccb0 0x55fe813da610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29e6ffd640 1 -- 172.21.15.7:0/2641467286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 msgr2=0x55fe813da0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813da0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29e6ffd640 1 -- 172.21.15.7:0/2641467286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe81049210 con 0x55fe813d1af0 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d9b90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f29d80048a0 tx=0x7f29d8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3317d7b37845ec18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29ceffd640 1 -- 172.21.15.7:0/2641467286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8041070 con 0x55fe813d1af0 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29ceffd640 1 -- 172.21.15.7:0/2641467286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f29d8004940 con 0x55fe813d1af0 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29f8cda740 1 -- 172.21.15.7:0/2641467286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d8005500 con 0x55fe813d1af0 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29ceffd640 1 -- 172.21.15.7:0/2641467286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f29d8035040 con 0x55fe813d1af0 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29f8cda740 10 client.16161 fetch_fsmap learned FSMap version 41 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29f8cda740 1 -- 172.21.15.7:0/2641467286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fe81115620 con 0x55fe813d1af0 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29ceffd640 1 -- 172.21.15.7:0/2641467286 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d8005500 con 0x55fe813d1af0 2024-09-10T10:57:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29ceffd640 1 -- 172.21.15.7:0/2641467286 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f29d8055030 con 0x55fe813d1af0 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29f8cda740 10 client.16161 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.680+0000 7f29f8cda740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.681+0000 7f29f8cda740 20 client.16161 populate_metadata read hostname 'smithi007' 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.681+0000 7f29f8cda740 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fe813eca80 0x55fe813eee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.681+0000 7f29f8cda740 4 client.16161 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.681+0000 7f29f8cda740 1 -- 172.21.15.7:0/2641467286 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe81343a20 con 0x55fe813eca80 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.681+0000 7f29ef7fe640 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fe813eca80 0x55fe813eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.681+0000 7f29ef7fe640 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fe813eca80 0x55fe813eee70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cfee24a8f7b9324 server_cookie=39430afbdc00dee1 in_seq=0 out_seq=0 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.681+0000 7f29ceffd640 10 client.16161 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29ceffd640 1 -- 172.21.15.7:0/2641467286 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe81343a20 con 0x55fe813eca80 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29ceffd640 10 client.16161 handle_command_reply: tid=0 2024-09-10T10:57:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 client.16161 shutdown 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 -- 172.21.15.7:0/2641467286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 msgr2=0x55fe813d9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d9b90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f29d80048a0 tx=0x7f29d8005740 comp rx=0 tx=0).stop 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 -- 172.21.15.7:0/2641467286 shutdown_connections 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fe813eca80 0x55fe813eee70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813dccb0 0x55fe813da610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813da0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29ceffd640 0 client.16161 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 --2- 172.21.15.7:0/2641467286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813d9b90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 -- 172.21.15.7:0/2641467286 >> 172.21.15.7:0/2641467286 conn(0x55fe813ce060 msgr2=0x55fe813d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 -- 172.21.15.7:0/2641467286 shutdown_connections 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.682+0000 7f29f8cda740 1 -- 172.21.15.7:0/2641467286 wait complete. 2024-09-10T10:57:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.683+0000 7f29f8cda740 20 client.16161 trim_cache size 0 max 16384 2024-09-10T10:57:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29f8cda740 1 Processor -- start 2024-09-10T10:57:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29f8cda740 1 -- start start 2024-09-10T10:57:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813da440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813d2410 0x55fe813f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813dccb0 0x55fe813f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29f8cda740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b2280 con 0x55fe813d1af0 2024-09-10T10:57:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29f8cda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01afb20 con 0x55fe813dccb0 2024-09-10T10:57:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29f8cda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01ad990 con 0x55fe813d2410 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813da440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29ef7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813d2410 0x55fe813f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813da440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53382/0 (socket says 172.21.15.7:53382) 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29e6ffd640 1 -- 172.21.15.7:0/2516932545 learned_addr learned my addr 172.21.15.7:0/2516932545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/2516932545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813dccb0 0x55fe813f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29e6ffd640 1 -- 172.21.15.7:0/2516932545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813d2410 msgr2=0x55fe813f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/2516932545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813d2410 0x55fe813f1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29e6ffd640 1 -- 172.21.15.7:0/2516932545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813dccb0 msgr2=0x55fe813f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.699+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/2516932545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813dccb0 0x55fe813f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.700+0000 7f29e6ffd640 1 -- 172.21.15.7:0/2516932545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe813d2240 con 0x55fe813d1af0 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.700+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/2516932545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813da440 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55fe812d9590 tx=0x7f29d802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbb2da354773425 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.700+0000 7f29e57fa640 1 -- 172.21.15.7:0/2516932545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8041030 con 0x55fe813d1af0 2024-09-10T10:57:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.700+0000 7f29e57fa640 1 -- 172.21.15.7:0/2516932545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d80054f0 con 0x55fe813d1af0 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.700+0000 7f29e57fa640 1 -- 172.21.15.7:0/2516932545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8004e70 con 0x55fe813d1af0 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 -- 172.21.15.7:0/2516932545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 msgr2=0x55fe813da440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 --2- 172.21.15.7:0/2516932545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813da440 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x55fe812d9590 tx=0x7f29d802a410 comp rx=0 tx=0).stop 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 -- 172.21.15.7:0/2516932545 shutdown_connections 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 --2- 172.21.15.7:0/2516932545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813dccb0 0x55fe813f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 --2- 172.21.15.7:0/2516932545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813d2410 0x55fe813f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 --2- 172.21.15.7:0/2516932545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813d1af0 0x55fe813da440 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 -- 172.21.15.7:0/2516932545 >> 172.21.15.7:0/2516932545 conn(0x55fe813ce060 msgr2=0x55fe813dbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 -- 172.21.15.7:0/2516932545 shutdown_connections 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 -- 172.21.15.7:0/2516932545 wait complete. 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 -- start start 2024-09-10T10:57:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.701+0000 7f29f8cda740 1 Processor -- start 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813d1af0 0x55fe813f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29f8cda740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813dccb0 0x55fe813ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29f8cda740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01adfe0 con 0x55fe813dccb0 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29f8cda740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01af450 con 0x55fe813d2410 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29f8cda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01ae6b0 con 0x55fe813d1af0 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29f8cda740 10 client.0 _ll_register_callbacks cb 0x7f29e000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813dccb0 0x55fe813ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29e6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813dccb0 0x55fe813ef550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53390/0 (socket says 172.21.15.7:53390) 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29e6ffd640 1 -- 172.21.15.7:0/732450851 learned_addr learned my addr 172.21.15.7:0/732450851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813d1af0 0x55fe813f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29ef7fe640 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29e6ffd640 1 -- 172.21.15.7:0/732450851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813d1af0 msgr2=0x55fe813f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813d1af0 0x55fe813f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29e6ffd640 1 -- 172.21.15.7:0/732450851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 msgr2=0x55fe813f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29e6ffd640 1 -- 172.21.15.7:0/732450851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe813efd30 con 0x55fe813dccb0 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29ef7fe640 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29e6ffd640 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813dccb0 0x55fe813ef550 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f29d8040040 tx=0x7f29d8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea01242439eb662 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29ceffd640 1 -- 172.21.15.7:0/732450851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d80052f0 con 0x55fe813dccb0 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29ceffd640 1 -- 172.21.15.7:0/732450851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f29d804d660 con 0x55fe813dccb0 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.702+0000 7f29f8cda740 1 -- 172.21.15.7:0/732450851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d80054f0 con 0x55fe813dccb0 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29ceffd640 1 -- 172.21.15.7:0/732450851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f29d804d860 con 0x55fe813dccb0 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29f8cda740 10 client.16167 fetch_fsmap learned FSMap version 41 2024-09-10T10:57:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29f8cda740 1 -- 172.21.15.7:0/732450851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fe81456f80 con 0x55fe813dccb0 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29ceffd640 1 -- 172.21.15.7:0/732450851 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d80054f0 con 0x55fe813dccb0 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29ceffd640 1 -- 172.21.15.7:0/732450851 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 41) ==== 8350+0+0 (secure 0 0 0) 0x7f29d806b070 con 0x55fe813dccb0 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29f8cda740 10 client.16167 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29f8cda740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29f8cda740 20 client.16167 populate_metadata read hostname 'smithi007' 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29f8cda740 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fe813eca80 0x55fe814594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29f8cda740 4 client.16167 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29f8cda740 1 -- 172.21.15.7:0/732450851 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe8133dd70 con 0x55fe813eca80 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.703+0000 7f29ef7fe640 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fe813eca80 0x55fe814594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.704+0000 7f29ceffd640 10 client.16167 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.704+0000 7f29ef7fe640 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fe813eca80 0x55fe814594c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57063ea259dca256 server_cookie=bb55bfda87f91202 in_seq=0 out_seq=0 2024-09-10T10:57:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.739+0000 7f29ceffd640 1 -- 172.21.15.7:0/732450851 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe8133dd70 con 0x55fe813eca80 2024-09-10T10:57:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.739+0000 7f29ceffd640 10 client.16167 handle_command_reply: tid=0 2024-09-10T10:57:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.739+0000 7f29f8cda740 1 client.16167 shutdown 2024-09-10T10:57:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.739+0000 7f29f8cda740 1 -- 172.21.15.7:0/732450851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813dccb0 msgr2=0x55fe813ef550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.739+0000 7f29f8cda740 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813dccb0 0x55fe813ef550 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f29d8040040 tx=0x7f29d8004a50 comp rx=0 tx=0).stop 2024-09-10T10:57:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.739+0000 7f29f8cda740 1 -- 172.21.15.7:0/732450851 shutdown_connections 2024-09-10T10:57:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.739+0000 7f29f8cda740 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fe813eca80 0x55fe814594c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.740+0000 7f29f8cda740 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fe813dccb0 0x55fe813ef550 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.740+0000 7f29f8cda740 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fe813d2410 0x55fe813f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.740+0000 7f29f8cda740 1 --2- 172.21.15.7:0/732450851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fe813d1af0 0x55fe813f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.740+0000 7f29ceffd640 0 client.16167 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.740+0000 7f29f8cda740 1 -- 172.21.15.7:0/732450851 >> 172.21.15.7:0/732450851 conn(0x55fe813ce060 msgr2=0x55fe813e7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.740+0000 7f29f8cda740 1 -- 172.21.15.7:0/732450851 shutdown_connections 2024-09-10T10:57:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.740+0000 7f29f8cda740 1 -- 172.21.15.7:0/732450851 wait complete. 2024-09-10T10:57:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.740+0000 7f29f8cda740 20 client.16167 trim_cache size 0 max 16384 2024-09-10T10:57:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:57:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:57:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:57:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:57:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.740+0000 7f29f5f42640 1 -- 172.21.15.7:0/1707906632 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f29dc0a2da0 msgr2=0x7f29dc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.740+0000 7f29f5f42640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f29dc0a2da0 0x7f29dc0a5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f29e80cd300 tx=0x7f29e80bc040 comp rx=0 tx=0).stop 2024-09-10T10:57:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.741+0000 7f29f5f42640 1 -- 172.21.15.7:0/1707906632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f010e440 msgr2=0x7f29f019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.741+0000 7f29f5f42640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f010e440 0x7f29f019ee30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f29e8005fd0 tx=0x7f29e800ef10 comp rx=0 tx=0).stop 2024-09-10T10:57:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.741+0000 7f29f5f42640 1 -- 172.21.15.7:0/1707906632 shutdown_connections 2024-09-10T10:57:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.741+0000 7f29f5f42640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f29dc0a2da0 0x7f29dc0a5260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.741+0000 7f29f5f42640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f010e440 0x7f29f019ee30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.741+0000 7f29f5f42640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29f0101240 0x7f29f019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.741+0000 7f29f5f42640 1 --2- 172.21.15.7:0/1707906632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29f0100890 0x7f29f019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.741+0000 7f29f5f42640 1 -- 172.21.15.7:0/1707906632 >> 172.21.15.7:0/1707906632 conn(0x7f29f0077990 msgr2=0x7f29f00ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.741+0000 7f29f5f42640 1 -- 172.21.15.7:0/1707906632 shutdown_connections 2024-09-10T10:57:12.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:12.741+0000 7f29f5f42640 1 -- 172.21.15.7:0/1707906632 wait complete. 2024-09-10T10:57:13.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1023929 8.10 MB/sec execute 493 sec latency 425.383 ms 2024-09-10T10:57:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:13 smithi103 ceph-mon[35160]: pgmap v357: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 771 KiB/s wr, 126 op/s 2024-09-10T10:57:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:13 smithi007 ceph-mon[30873]: pgmap v357: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 771 KiB/s wr, 126 op/s 2024-09-10T10:57:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:13 smithi133 ceph-mon[35282]: pgmap v357: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 771 KiB/s wr, 126 op/s 2024-09-10T10:57:14.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1023929 8.09 MB/sec execute 494 sec latency 1425.501 ms 2024-09-10T10:57:15.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1023929 8.07 MB/sec execute 495 sec latency 2425.632 ms 2024-09-10T10:57:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:15 smithi103 ceph-mon[35160]: pgmap v358: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 640 KiB/s wr, 105 op/s 2024-09-10T10:57:15.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:15 smithi007 ceph-mon[30873]: pgmap v358: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 640 KiB/s wr, 105 op/s 2024-09-10T10:57:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:15 smithi133 ceph-mon[35282]: pgmap v358: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 640 KiB/s wr, 105 op/s 2024-09-10T10:57:16.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1024359 8.06 MB/sec execute 496 sec latency 3152.266 ms 2024-09-10T10:57:17.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1026884 8.08 MB/sec execute 497 sec latency 16.923 ms 2024-09-10T10:57:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:17 smithi103 ceph-mon[35160]: pgmap v359: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 820 KiB/s wr, 87 op/s 2024-09-10T10:57:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:17 smithi007 ceph-mon[30873]: pgmap v359: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 820 KiB/s wr, 87 op/s 2024-09-10T10:57:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:17 smithi133 ceph-mon[35282]: pgmap v359: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 820 KiB/s wr, 87 op/s 2024-09-10T10:57:18.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1028218 8.07 MB/sec execute 498 sec latency 13.982 ms 2024-09-10T10:57:19.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1030552 8.08 MB/sec execute 499 sec latency 17.715 ms 2024-09-10T10:57:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:19 smithi103 ceph-mon[35160]: pgmap v360: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 820 KiB/s wr, 87 op/s 2024-09-10T10:57:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:19 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:57:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:19 smithi007 ceph-mon[30873]: pgmap v360: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 820 KiB/s wr, 87 op/s 2024-09-10T10:57:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:19 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:57:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:19 smithi133 ceph-mon[35282]: pgmap v360: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 467 KiB/s rd, 820 KiB/s wr, 87 op/s 2024-09-10T10:57:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:19 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:57:20.538 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1032058 8.08 MB/sec execute 500 sec latency 15.921 ms 2024-09-10T10:57:21.539 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1034166 8.09 MB/sec execute 501 sec latency 19.173 ms 2024-09-10T10:57:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:21 smithi103 ceph-mon[35160]: pgmap v361: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-10T10:57:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:21 smithi007 ceph-mon[30873]: pgmap v361: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-10T10:57:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:21 smithi133 ceph-mon[35282]: pgmap v361: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-10T10:57:22.539 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1034500 8.07 MB/sec execute 502 sec latency 704.289 ms 2024-09-10T10:57:23.539 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1034500 8.06 MB/sec execute 503 sec latency 1704.394 ms 2024-09-10T10:57:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:23 smithi103 ceph-mon[35160]: pgmap v362: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-09-10T10:57:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:23 smithi007 ceph-mon[30873]: pgmap v362: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-09-10T10:57:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:23 smithi133 ceph-mon[35282]: pgmap v362: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-09-10T10:57:24.539 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1034500 8.04 MB/sec execute 504 sec latency 2704.486 ms 2024-09-10T10:57:25.539 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1034500 8.02 MB/sec execute 505 sec latency 3704.591 ms 2024-09-10T10:57:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.841+0000 7f1316677640 1 -- 172.21.15.7:0/133572422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131010c0b0 msgr2=0x7f131010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.841+0000 7f1316677640 1 --2- 172.21.15.7:0/133572422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131010c0b0 0x7f131010e550 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1308009ca0 tx=0x7f130802a950 comp rx=0 tx=0).stop 2024-09-10T10:57:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.841+0000 7f1316677640 1 -- 172.21.15.7:0/133572422 shutdown_connections 2024-09-10T10:57:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.841+0000 7f1316677640 1 --2- 172.21.15.7:0/133572422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131010c0b0 0x7f131010e550 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.841+0000 7f1316677640 1 --2- 172.21.15.7:0/133572422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1310103600 0x7f131010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.841+0000 7f1316677640 1 --2- 172.21.15.7:0/133572422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310102ce0 0x7f13101030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.841+0000 7f1316677640 1 -- 172.21.15.7:0/133572422 >> 172.21.15.7:0/133572422 conn(0x7f1310077990 msgr2=0x7f13100fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.841+0000 7f1316677640 1 -- 172.21.15.7:0/133572422 shutdown_connections 2024-09-10T10:57:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.841+0000 7f1316677640 1 -- 172.21.15.7:0/133572422 wait complete. 2024-09-10T10:57:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.842+0000 7f1316677640 1 Processor -- start 2024-09-10T10:57:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.842+0000 7f1316677640 1 -- start start 2024-09-10T10:57:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.842+0000 7f1316677640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1310102ce0 0x7f131019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.842+0000 7f1316677640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310103600 0x7f131019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.842+0000 7f1316677640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131010c0b0 0x7f131019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.842+0000 7f1316677640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1310111eb0 con 0x7f131010c0b0 2024-09-10T10:57:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.842+0000 7f1316677640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1310111d30 con 0x7f1310102ce0 2024-09-10T10:57:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.842+0000 7f1316677640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1310112030 con 0x7f1310103600 2024-09-10T10:57:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.842+0000 7f1314c14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131010c0b0 0x7f131019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f1314c14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131010c0b0 0x7f131019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36516/0 (socket says 172.21.15.7:36516) 2024-09-10T10:57:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f1314c14640 1 -- 172.21.15.7:0/2590027678 learned_addr learned my addr 172.21.15.7:0/2590027678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f130ffff640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1310102ce0 0x7f131019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f130f7fe640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310103600 0x7f131019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f1314c14640 1 -- 172.21.15.7:0/2590027678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310103600 msgr2=0x7f131019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f1314c14640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310103600 0x7f131019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f1314c14640 1 -- 172.21.15.7:0/2590027678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1310102ce0 msgr2=0x7f131019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f1314c14640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1310102ce0 0x7f131019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f1314c14640 1 -- 172.21.15.7:0/2590027678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131019f560 con 0x7f131010c0b0 2024-09-10T10:57:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f1314c14640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131010c0b0 0x7f131019ee30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f130802ae70 tx=0x7f130802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:25.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f130d7fa640 1 -- 172.21.15.7:0/2590027678 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308034040 con 0x7f131010c0b0 2024-09-10T10:57:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f130d7fa640 1 -- 172.21.15.7:0/2590027678 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1308030920 con 0x7f131010c0b0 2024-09-10T10:57:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f1316677640 1 -- 172.21.15.7:0/2590027678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131019f7f0 con 0x7f131010c0b0 2024-09-10T10:57:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f130d7fa640 1 -- 172.21.15.7:0/2590027678 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308045480 con 0x7f131010c0b0 2024-09-10T10:57:25.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.843+0000 7f1316677640 1 -- 172.21.15.7:0/2590027678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13101a70e0 con 0x7f131010c0b0 2024-09-10T10:57:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f130d7fa640 1 -- 172.21.15.7:0/2590027678 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1308037020 con 0x7f131010c0b0 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f1319409740 1 Processor -- start 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f1319409740 1 -- start start 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f1319409740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7852af0 0x55f9d7852ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f1319409740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7853410 0x55f9d785d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f1319409740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d78600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f1319409740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f9d77b72f0 con 0x55f9d7852af0 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f1319409740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f9d775f270 con 0x55f9d785dcb0 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f1319409740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f9d77b32b0 con 0x55f9d7853410 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f130f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7852af0 0x55f9d7852ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f130f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7852af0 0x55f9d7852ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36530/0 (socket says 172.21.15.7:36530) 2024-09-10T10:57:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f130f7fe640 1 -- 172.21.15.7:0/1743354268 learned_addr learned my addr 172.21.15.7:0/1743354268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.846+0000 7f130ffff640 1 --2- 172.21.15.7:0/1743354268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7853410 0x55f9d785d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f130f7fe640 1 --2- 172.21.15.7:0/1743354268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d78600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f130d7fa640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f12f80a2e20 0x7f12f80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f130f7fe640 1 -- 172.21.15.7:0/1743354268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7853410 msgr2=0x55f9d785d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f130f7fe640 1 --2- 172.21.15.7:0/1743354268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7853410 0x55f9d785d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f130f7fe640 1 -- 172.21.15.7:0/1743354268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 msgr2=0x55f9d78600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f130f7fe640 1 --2- 172.21.15.7:0/1743354268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d78600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f130f7fe640 1 -- 172.21.15.7:0/1743354268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d775e340 con 0x55f9d7852af0 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f1314c14640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f12f80a2e20 0x7f12f80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f130f7fe640 1 --2- 172.21.15.7:0/1743354268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7852af0 0x55f9d7852ed0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f130400bf70 tx=0x7f130400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d958a8f335dac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f130d7fa640 1 -- 172.21.15.7:0/2590027678 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f13080f0100 con 0x7f131010c0b0 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f12f67fc640 1 -- 172.21.15.7:0/1743354268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130401e640 con 0x55f9d7852af0 2024-09-10T10:57:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f12f67fc640 1 -- 172.21.15.7:0/1743354268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1304010910 con 0x55f9d7852af0 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.847+0000 7f12f67fc640 1 -- 172.21.15.7:0/1743354268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130401d780 con 0x55f9d7852af0 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1319409740 1 -- 172.21.15.7:0/1743354268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7852af0 msgr2=0x55f9d7852ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1319409740 1 --2- 172.21.15.7:0/1743354268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7852af0 0x55f9d7852ed0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f130400bf70 tx=0x7f130400b7c0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1314c14640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f12f80a2e20 0x7f12f80a52e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13080eda80 tx=0x7f13080b8a40 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1319409740 1 -- 172.21.15.7:0/1743354268 shutdown_connections 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1319409740 1 --2- 172.21.15.7:0/1743354268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d78600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1319409740 1 --2- 172.21.15.7:0/1743354268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7853410 0x55f9d785d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1319409740 1 --2- 172.21.15.7:0/1743354268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7852af0 0x55f9d7852ed0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1319409740 1 -- 172.21.15.7:0/1743354268 >> 172.21.15.7:0/1743354268 conn(0x55f9d784f060 msgr2=0x55f9d784f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1319409740 1 -- 172.21.15.7:0/1743354268 shutdown_connections 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1319409740 1 -- 172.21.15.7:0/1743354268 wait complete. 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.848+0000 7f1319409740 1 -- start start 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f1319409740 1 Processor -- start 2024-09-10T10:57:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f1319409740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d785ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f1319409740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d785b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f1319409740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d785b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f1319409740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f9d7860880 con 0x55f9d7853410 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f1319409740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f9d7740290 con 0x55f9d785dcb0 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f1319409740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f9d7860a00 con 0x55f9d7852af0 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f1319409740 10 client.0 _ll_register_callbacks cb 0x55f9d7771140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d785b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d785b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36534/0 (socket says 172.21.15.7:36534) 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130ffff640 1 -- 172.21.15.7:0/3996500706 learned_addr learned my addr 172.21.15.7:0/3996500706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130f7fe640 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d785b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130f7fe640 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d785ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130ffff640 1 -- 172.21.15.7:0/3996500706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 msgr2=0x55f9d785ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130ffff640 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d785ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130ffff640 1 -- 172.21.15.7:0/3996500706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 msgr2=0x55f9d785b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130ffff640 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d785b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130ffff640 1 -- 172.21.15.7:0/3996500706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d74ca210 con 0x55f9d7853410 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.849+0000 7f130f7fe640 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d785ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f130ffff640 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d785b070 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f130000b920 tx=0x7f130000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7184dc3acd1b96be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f12ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f12ef7fe640 1 -- 172.21.15.7:0/3996500706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300003ea0 con 0x55f9d7853410 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f12ef7fe640 1 -- 172.21.15.7:0/3996500706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f130001b640 con 0x55f9d7853410 2024-09-10T10:57:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f1319409740 1 -- 172.21.15.7:0/3996500706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1304010910 con 0x55f9d7853410 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f12ef7fe640 1 -- 172.21.15.7:0/3996500706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f130001b840 con 0x55f9d7853410 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f1319409740 10 client.16179 fetch_fsmap learned FSMap version 42 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f1319409740 1 -- 172.21.15.7:0/3996500706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f9d7596620 con 0x55f9d7853410 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f12ef7fe640 1 -- 172.21.15.7:0/3996500706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1304010910 con 0x55f9d7853410 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f12ef7fe640 1 -- 172.21.15.7:0/3996500706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f1300019020 con 0x55f9d7853410 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f1319409740 10 client.16179 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f1319409740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f1319409740 20 client.16179 populate_metadata read hostname 'smithi007' 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f1319409740 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f9d786dc60 0x55f9d7870050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f1319409740 4 client.16179 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.850+0000 7f1319409740 1 -- 172.21.15.7:0/3996500706 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9d77c4a20 con 0x55f9d786dc60 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.851+0000 7f130f7fe640 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f9d786dc60 0x55f9d7870050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.851+0000 7f12ef7fe640 10 client.16179 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.851+0000 7f130f7fe640 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f9d786dc60 0x55f9d7870050 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d9761a104c1789 server_cookie=ce33c30d17c96cc2 in_seq=0 out_seq=0 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f12ef7fe640 1 -- 172.21.15.7:0/3996500706 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9d77c4a20 con 0x55f9d786dc60 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f12ef7fe640 10 client.16179 handle_command_reply: tid=0 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 client.16179 shutdown 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 -- 172.21.15.7:0/3996500706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 msgr2=0x55f9d785b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d785b070 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f130000b920 tx=0x7f130000bde0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 -- 172.21.15.7:0/3996500706 shutdown_connections 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f9d786dc60 0x55f9d7870050 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d785b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d785b070 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 --2- 172.21.15.7:0/3996500706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d785ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f12ef7fe640 0 client.16179 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 -- 172.21.15.7:0/3996500706 >> 172.21.15.7:0/3996500706 conn(0x55f9d784f060 msgr2=0x55f9d7854dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 -- 172.21.15.7:0/3996500706 shutdown_connections 2024-09-10T10:57:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 1 -- 172.21.15.7:0/3996500706 wait complete. 2024-09-10T10:57:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.852+0000 7f1319409740 20 client.16179 trim_cache size 0 max 16384 2024-09-10T10:57:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.867+0000 7f1319409740 1 Processor -- start 2024-09-10T10:57:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.867+0000 7f1319409740 1 -- start start 2024-09-10T10:57:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.867+0000 7f1319409740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d785b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.867+0000 7f1319409740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d7853410 0x55f9d7875910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.867+0000 7f1319409740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d785dcb0 0x55f9d7875e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.867+0000 7f1319409740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13101b2230 con 0x55f9d785dcb0 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.867+0000 7f1319409740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13101afb00 con 0x55f9d7853410 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.867+0000 7f1319409740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f13101ad970 con 0x55f9d7852af0 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.867+0000 7f130f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d785dcb0 0x55f9d7875e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d785dcb0 0x55f9d7875e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36548/0 (socket says 172.21.15.7:36548) 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130f7fe640 1 -- 172.21.15.7:0/3303551235 learned_addr learned my addr 172.21.15.7:0/3303551235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130f7fe640 1 --2- 172.21.15.7:0/3303551235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d785b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130ffff640 1 --2- 172.21.15.7:0/3303551235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d7853410 0x55f9d7875910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130f7fe640 1 -- 172.21.15.7:0/3303551235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 msgr2=0x55f9d785b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130f7fe640 1 --2- 172.21.15.7:0/3303551235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d785b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130f7fe640 1 -- 172.21.15.7:0/3303551235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d7853410 msgr2=0x55f9d7875910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130f7fe640 1 --2- 172.21.15.7:0/3303551235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d7853410 0x55f9d7875910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130f7fe640 1 -- 172.21.15.7:0/3303551235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d7870680 con 0x55f9d785dcb0 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130ffff640 1 --2- 172.21.15.7:0/3303551235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d7853410 0x55f9d7875910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f130f7fe640 1 --2- 172.21.15.7:0/3303551235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d785dcb0 0x55f9d7875e50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f130401d450 tx=0x7f130400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58b60f28ce98921 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f12f5ffb640 1 -- 172.21.15.7:0/3303551235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304005050 con 0x55f9d785dcb0 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f12f5ffb640 1 -- 172.21.15.7:0/3303551235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1304010720 con 0x55f9d785dcb0 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.868+0000 7f12f5ffb640 1 -- 172.21.15.7:0/3303551235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040049c0 con 0x55f9d785dcb0 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 -- 172.21.15.7:0/3303551235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d785dcb0 msgr2=0x55f9d7875e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 --2- 172.21.15.7:0/3303551235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d785dcb0 0x55f9d7875e50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f130401d450 tx=0x7f130400b7c0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 -- 172.21.15.7:0/3303551235 shutdown_connections 2024-09-10T10:57:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 --2- 172.21.15.7:0/3303551235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d785dcb0 0x55f9d7875e50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 --2- 172.21.15.7:0/3303551235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d7853410 0x55f9d7875910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 --2- 172.21.15.7:0/3303551235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d785b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 -- 172.21.15.7:0/3303551235 >> 172.21.15.7:0/3303551235 conn(0x55f9d784f060 msgr2=0x55f9d785cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 -- 172.21.15.7:0/3303551235 shutdown_connections 2024-09-10T10:57:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 -- 172.21.15.7:0/3303551235 wait complete. 2024-09-10T10:57:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 -- start start 2024-09-10T10:57:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.869+0000 7f1319409740 1 Processor -- start 2024-09-10T10:57:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f1319409740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d78664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f1319409740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d7861830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f1319409740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d7861db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f1319409740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13101adfc0 con 0x55f9d7853410 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f1319409740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13101af430 con 0x55f9d785dcb0 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f1319409740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f13101ae690 con 0x55f9d7852af0 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f1319409740 10 client.0 _ll_register_callbacks cb 0x7f1300004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d7861830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d7861830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36562/0 (socket says 172.21.15.7:36562) 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130ffff640 1 -- 172.21.15.7:0/92683171 learned_addr learned my addr 172.21.15.7:0/92683171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130f7fe640 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d78664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130f7fe640 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d7861db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130ffff640 1 -- 172.21.15.7:0/92683171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 msgr2=0x55f9d78664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130ffff640 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d78664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130ffff640 1 -- 172.21.15.7:0/92683171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 msgr2=0x55f9d7861db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130ffff640 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d7861db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130ffff640 1 -- 172.21.15.7:0/92683171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d784ee90 con 0x55f9d7853410 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130f7fe640 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d78664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f130ffff640 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d7861830 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1300000c00 tx=0x7f1300003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90682f478e51a0d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f12ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.870+0000 7f12ef7fe640 1 -- 172.21.15.7:0/92683171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300027040 con 0x55f9d7853410 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f12ef7fe640 1 -- 172.21.15.7:0/92683171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f130001b9e0 con 0x55f9d7853410 2024-09-10T10:57:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f1319409740 1 -- 172.21.15.7:0/92683171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1304010720 con 0x55f9d7853410 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f12ef7fe640 1 -- 172.21.15.7:0/92683171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1300038050 con 0x55f9d7853410 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f1319409740 10 client.16185 fetch_fsmap learned FSMap version 42 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f1319409740 1 -- 172.21.15.7:0/92683171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f9d78d87c0 con 0x55f9d7853410 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f12ef7fe640 1 -- 172.21.15.7:0/92683171 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1304010720 con 0x55f9d7853410 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f12ef7fe640 1 -- 172.21.15.7:0/92683171 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f1300033030 con 0x55f9d7853410 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f1319409740 10 client.16185 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f1319409740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f1319409740 20 client.16185 populate_metadata read hostname 'smithi007' 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f1319409740 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f9d78d8fc0 0x55f9d78db3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f1319409740 4 client.16185 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:57:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f1319409740 1 -- 172.21.15.7:0/92683171 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9d77bed70 con 0x55f9d78d8fc0 2024-09-10T10:57:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.871+0000 7f130f7fe640 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f9d78d8fc0 0x55f9d78db3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.872+0000 7f130f7fe640 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f9d78d8fc0 0x55f9d78db3e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523ca731cf184568 server_cookie=9e9d28ffc56b483e in_seq=0 out_seq=0 2024-09-10T10:57:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.872+0000 7f12ef7fe640 10 client.16185 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:25 smithi103 ceph-mon[35160]: pgmap v363: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-09-10T10:57:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f12ef7fe640 1 -- 172.21.15.7:0/92683171 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9d77bed70 con 0x55f9d78d8fc0 2024-09-10T10:57:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f12ef7fe640 10 client.16185 handle_command_reply: tid=0 2024-09-10T10:57:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 client.16185 shutdown 2024-09-10T10:57:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 -- 172.21.15.7:0/92683171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 msgr2=0x55f9d7861830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d7861830 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1300000c00 tx=0x7f1300003a40 comp rx=0 tx=0).stop 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 -- 172.21.15.7:0/92683171 shutdown_connections 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55f9d78d8fc0 0x55f9d78db3e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f9d785dcb0 0x55f9d7861db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d7853410 0x55f9d7861830 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 --2- 172.21.15.7:0/92683171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9d7852af0 0x55f9d78664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 -- 172.21.15.7:0/92683171 >> 172.21.15.7:0/92683171 conn(0x55f9d784f060 msgr2=0x55f9d784d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f12ef7fe640 0 client.16185 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 -- 172.21.15.7:0/92683171 shutdown_connections 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 1 -- 172.21.15.7:0/92683171 wait complete. 2024-09-10T10:57:25.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.931+0000 7f1319409740 20 client.16185 trim_cache size 0 max 16384 2024-09-10T10:57:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:57:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:57:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:57:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:57:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 -- 172.21.15.7:0/2590027678 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f12f80a2e20 msgr2=0x7f12f80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f12f80a2e20 0x7f12f80a52e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13080eda80 tx=0x7f13080b8a40 comp rx=0 tx=0).stop 2024-09-10T10:57:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 -- 172.21.15.7:0/2590027678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131010c0b0 msgr2=0x7f131019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131010c0b0 0x7f131019ee30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f130802ae70 tx=0x7f130802f5d0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 -- 172.21.15.7:0/2590027678 shutdown_connections 2024-09-10T10:57:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f12f80a2e20 0x7f12f80a52e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f131010c0b0 0x7f131019ee30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1310103600 0x7f131019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 --2- 172.21.15.7:0/2590027678 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1310102ce0 0x7f131019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 -- 172.21.15.7:0/2590027678 >> 172.21.15.7:0/2590027678 conn(0x7f1310077990 msgr2=0x7f1310100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:25.935 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:25 smithi007 ceph-mon[30873]: pgmap v363: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-09-10T10:57:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 -- 172.21.15.7:0/2590027678 shutdown_connections 2024-09-10T10:57:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:25.932+0000 7f1316677640 1 -- 172.21.15.7:0/2590027678 wait complete. 2024-09-10T10:57:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:25 smithi133 ceph-mon[35282]: pgmap v363: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-09-10T10:57:26.539 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1034568 8.01 MB/sec execute 506 sec latency 4424.332 ms 2024-09-10T10:57:27.539 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1034568 7.99 MB/sec execute 507 sec latency 1234.994 ms 2024-09-10T10:57:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:27 smithi103 ceph-mon[35160]: pgmap v364: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-09-10T10:57:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:27 smithi007 ceph-mon[30873]: pgmap v364: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-09-10T10:57:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:27 smithi133 ceph-mon[35282]: pgmap v364: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-09-10T10:57:28.539 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1034568 7.98 MB/sec execute 508 sec latency 2235.133 ms 2024-09-10T10:57:29.539 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1034568 7.96 MB/sec execute 509 sec latency 3235.265 ms 2024-09-10T10:57:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:29 smithi103 ceph-mon[35160]: pgmap v365: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-10T10:57:29.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:29 smithi007 ceph-mon[30873]: pgmap v365: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-10T10:57:29.989 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:29 smithi133 ceph-mon[35282]: pgmap v365: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-10T10:57:30.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1034568 7.95 MB/sec execute 510 sec latency 4235.375 ms 2024-09-10T10:57:31.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1035001 7.93 MB/sec execute 511 sec latency 4961.270 ms 2024-09-10T10:57:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:31 smithi103 ceph-mon[35160]: pgmap v366: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-09-10T10:57:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:31 smithi007 ceph-mon[30873]: pgmap v366: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-09-10T10:57:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:31 smithi133 ceph-mon[35282]: pgmap v366: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-09-10T10:57:32.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1037538 7.95 MB/sec execute 512 sec latency 15.675 ms 2024-09-10T10:57:33.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1038112 7.94 MB/sec execute 513 sec latency 407.649 ms 2024-09-10T10:57:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:33 smithi103 ceph-mon[35160]: pgmap v367: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 745 KiB/s wr, 78 op/s 2024-09-10T10:57:33.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:33 smithi007 ceph-mon[30873]: pgmap v367: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 745 KiB/s wr, 78 op/s 2024-09-10T10:57:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:33 smithi133 ceph-mon[35282]: pgmap v367: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 745 KiB/s wr, 78 op/s 2024-09-10T10:57:34.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1038112 7.92 MB/sec execute 514 sec latency 1407.750 ms 2024-09-10T10:57:35.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1038112 7.91 MB/sec execute 515 sec latency 2407.869 ms 2024-09-10T10:57:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:35 smithi103 ceph-mon[35160]: pgmap v368: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 745 KiB/s wr, 78 op/s 2024-09-10T10:57:35.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:35 smithi007 ceph-mon[30873]: pgmap v368: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 745 KiB/s wr, 78 op/s 2024-09-10T10:57:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:35 smithi133 ceph-mon[35282]: pgmap v368: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 745 KiB/s wr, 78 op/s 2024-09-10T10:57:36.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1038542 7.89 MB/sec execute 516 sec latency 3130.834 ms 2024-09-10T10:57:37.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1041004 7.91 MB/sec execute 517 sec latency 16.204 ms 2024-09-10T10:57:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:37 smithi103 ceph-mon[35160]: pgmap v369: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.0 MiB/s wr, 117 op/s 2024-09-10T10:57:37.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:37 smithi007 ceph-mon[30873]: pgmap v369: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.0 MiB/s wr, 117 op/s 2024-09-10T10:57:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:37 smithi133 ceph-mon[35282]: pgmap v369: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.0 MiB/s wr, 117 op/s 2024-09-10T10:57:38.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1041537 7.90 MB/sec execute 518 sec latency 459.731 ms 2024-09-10T10:57:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.041+0000 7f62ebaac640 1 -- 172.21.15.7:0/1133865584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e4102480 msgr2=0x7f62e410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.041+0000 7f62ebaac640 1 --2- 172.21.15.7:0/1133865584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e4102480 0x7f62e410e9b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009cd0 tx=0x7f62dc02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.041+0000 7f62ebaac640 1 -- 172.21.15.7:0/1133865584 shutdown_connections 2024-09-10T10:57:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.041+0000 7f62ebaac640 1 --2- 172.21.15.7:0/1133865584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e4102480 0x7f62e410e9b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.041+0000 7f62ebaac640 1 --2- 172.21.15.7:0/1133865584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62e4101ae0 0x7f62e4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.041+0000 7f62ebaac640 1 --2- 172.21.15.7:0/1133865584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e4107b80 0x7f62e4107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.041+0000 7f62ebaac640 1 -- 172.21.15.7:0/1133865584 >> 172.21.15.7:0/1133865584 conn(0x7f62e40fd7e0 msgr2=0x7f62e40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.041+0000 7f62ebaac640 1 -- 172.21.15.7:0/1133865584 shutdown_connections 2024-09-10T10:57:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.041+0000 7f62ebaac640 1 -- 172.21.15.7:0/1133865584 wait complete. 2024-09-10T10:57:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62ebaac640 1 Processor -- start 2024-09-10T10:57:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62ebaac640 1 -- start start 2024-09-10T10:57:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62ebaac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e4101ae0 0x7f62e4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62ebaac640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e4102480 0x7f62e4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62ebaac640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62e4107b80 0x7f62e419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62ebaac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4111ef0 con 0x7f62e4101ae0 2024-09-10T10:57:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62ebaac640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4111d70 con 0x7f62e4102480 2024-09-10T10:57:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62ebaac640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4112070 con 0x7f62e4107b80 2024-09-10T10:57:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62e9848640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e4101ae0 0x7f62e4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62e9047640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e4102480 0x7f62e4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62e9848640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e4101ae0 0x7f62e4198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45290/0 (socket says 172.21.15.7:45290) 2024-09-10T10:57:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62e9848640 1 -- 172.21.15.7:0/2889879487 learned_addr learned my addr 172.21.15.7:0/2889879487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.042+0000 7f62ea049640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62e4107b80 0x7f62e419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62e9047640 1 -- 172.21.15.7:0/2889879487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62e4107b80 msgr2=0x7f62e419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62e9047640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62e4107b80 0x7f62e419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62e9047640 1 -- 172.21.15.7:0/2889879487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e4101ae0 msgr2=0x7f62e4198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62e9047640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e4101ae0 0x7f62e4198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62e9047640 1 -- 172.21.15.7:0/2889879487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e419d300 con 0x7f62e4102480 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62ea049640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62e4107b80 0x7f62e419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62e9848640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e4101ae0 0x7f62e4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62e9047640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e4102480 0x7f62e4198850 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e9f0 tx=0x7f62d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62daffd640 1 -- 172.21.15.7:0/2889879487 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d400ccc0 con 0x7f62e4102480 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62daffd640 1 -- 172.21.15.7:0/2889879487 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d4005020 con 0x7f62e4102480 2024-09-10T10:57:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62ebaac640 1 -- 172.21.15.7:0/2889879487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e419d5f0 con 0x7f62e4102480 2024-09-10T10:57:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62daffd640 1 -- 172.21.15.7:0/2889879487 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40049e0 con 0x7f62e4102480 2024-09-10T10:57:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.043+0000 7f62ebaac640 1 -- 172.21.15.7:0/2889879487 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e41a4e80 con 0x7f62e4102480 2024-09-10T10:57:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.046+0000 7f62ee84b740 1 Processor -- start 2024-09-10T10:57:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.046+0000 7f62daffd640 1 -- 172.21.15.7:0/2889879487 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f62d4005560 con 0x7f62e4102480 2024-09-10T10:57:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.046+0000 7f62ee84b740 1 -- start start 2024-09-10T10:57:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.046+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fafed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.046+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.046+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fbd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.046+0000 7f62ee84b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561746f142f0 con 0x561746fbacb0 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.046+0000 7f62ee84b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561746ebc270 con 0x561746fb0410 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.046+0000 7f62ee84b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561746f102b0 con 0x561746fafaf0 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62e9047640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fbd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62ea049640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fafed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62ea049640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fafed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:37294/0 (socket says 172.21.15.7:37294) 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62e9848640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62e9047640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fbd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45298/0 (socket says 172.21.15.7:45298) 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62ea049640 1 -- 172.21.15.7:0/241635867 learned_addr learned my addr 172.21.15.7:0/241635867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62daffd640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62d00a2da0 0x7f62d00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62e9047640 1 -- 172.21.15.7:0/241635867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 msgr2=0x561746fafed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62e9047640 1 --2- 172.21.15.7:0/241635867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fafed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62e9047640 1 -- 172.21.15.7:0/241635867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 msgr2=0x561746fba770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62e9047640 1 --2- 172.21.15.7:0/241635867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fba770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62e9047640 1 -- 172.21.15.7:0/241635867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561746ebb340 con 0x561746fbacb0 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62ea049640 1 --2- 172.21.15.7:0/241635867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fafed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62e9848640 1 --2- 172.21.15.7:0/241635867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62ea049640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62d00a2da0 0x7f62d00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62daffd640 1 -- 172.21.15.7:0/2889879487 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f62d40cf4e0 con 0x7f62e4102480 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62e9047640 1 --2- 172.21.15.7:0/241635867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fbd0a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f62d40ccba0 tx=0x7f62d401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cf2b34790040c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.047+0000 7f62cffff640 1 -- 172.21.15.7:0/241635867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d409b2d0 con 0x561746fbacb0 2024-09-10T10:57:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62cffff640 1 -- 172.21.15.7:0/241635867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d40051d0 con 0x561746fbacb0 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62cffff640 1 -- 172.21.15.7:0/241635867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4066560 con 0x561746fbacb0 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62ea049640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62d00a2da0 0x7f62d00a5260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009430 tx=0x7f62dc03f040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62ee84b740 1 -- 172.21.15.7:0/241635867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 msgr2=0x561746fbd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62ee84b740 1 --2- 172.21.15.7:0/241635867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fbd0a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f62d40ccba0 tx=0x7f62d401f000 comp rx=0 tx=0).stop 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62ee84b740 1 -- 172.21.15.7:0/241635867 shutdown_connections 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62ee84b740 1 --2- 172.21.15.7:0/241635867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fbd0a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62ee84b740 1 --2- 172.21.15.7:0/241635867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62ee84b740 1 --2- 172.21.15.7:0/241635867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fafed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62ee84b740 1 -- 172.21.15.7:0/241635867 >> 172.21.15.7:0/241635867 conn(0x561746fac060 msgr2=0x561746fac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62ee84b740 1 -- 172.21.15.7:0/241635867 shutdown_connections 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.048+0000 7f62ee84b740 1 -- 172.21.15.7:0/241635867 wait complete. 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.049+0000 7f62ee84b740 1 -- start start 2024-09-10T10:57:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.049+0000 7f62ee84b740 1 Processor -- start 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.049+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.049+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fb8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.049+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fbacb0 0x561746fb8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.049+0000 7f62ee84b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561746fbd880 con 0x561746fafaf0 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.049+0000 7f62ee84b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561746e9d290 con 0x561746fb0410 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.049+0000 7f62ee84b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561746fbda00 con 0x561746fbacb0 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.049+0000 7f62ee84b740 10 client.0 _ll_register_callbacks cb 0x561746ece140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.049+0000 7f62e9848640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62e9848640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45308/0 (socket says 172.21.15.7:45308) 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62e9848640 1 -- 172.21.15.7:0/1250604242 learned_addr learned my addr 172.21.15.7:0/1250604242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62e9047640 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fb8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62ea049640 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fbacb0 0x561746fb8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62e9848640 1 -- 172.21.15.7:0/1250604242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fbacb0 msgr2=0x561746fb8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62e9848640 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fbacb0 0x561746fb8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62e9848640 1 -- 172.21.15.7:0/1250604242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 msgr2=0x561746fb8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62e9848640 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fb8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62e9848640 1 -- 172.21.15.7:0/1250604242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561746c27210 con 0x561746fafaf0 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62e9848640 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb7ad0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f62e00118a0 tx=0x7f62e0011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350fd3bcefeda55c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62cd7fa640 1 -- 172.21.15.7:0/1250604242 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0010c60 con 0x561746fafaf0 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62cd7fa640 1 -- 172.21.15.7:0/1250604242 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f62e0010e00 con 0x561746fafaf0 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62ee84b740 1 -- 172.21.15.7:0/1250604242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d40051d0 con 0x561746fafaf0 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62cd7fa640 1 -- 172.21.15.7:0/1250604242 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f62e001a410 con 0x561746fafaf0 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62ee84b740 10 client.16197 fetch_fsmap learned FSMap version 42 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.050+0000 7f62ee84b740 1 -- 172.21.15.7:0/1250604242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561746cf3620 con 0x561746fafaf0 2024-09-10T10:57:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62cd7fa640 1 -- 172.21.15.7:0/1250604242 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d40051d0 con 0x561746fafaf0 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62cd7fa640 1 -- 172.21.15.7:0/1250604242 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f62e00217a0 con 0x561746fafaf0 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62ee84b740 10 client.16197 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62ee84b740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62ee84b740 20 client.16197 populate_metadata read hostname 'smithi007' 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62ee84b740 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561746fcab70 0x561746fccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62ee84b740 4 client.16197 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62ee84b740 1 -- 172.21.15.7:0/1250604242 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561746f21a20 con 0x561746fcab70 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62e9047640 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561746fcab70 0x561746fccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62e9047640 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561746fcab70 0x561746fccfa0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbd9e268cbdb01d3 server_cookie=2420179add5b8a83 in_seq=0 out_seq=0 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.051+0000 7f62cd7fa640 10 client.16197 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62cd7fa640 1 -- 172.21.15.7:0/1250604242 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561746f21a20 con 0x561746fcab70 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62cd7fa640 10 client.16197 handle_command_reply: tid=0 2024-09-10T10:57:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 client.16197 shutdown 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 -- 172.21.15.7:0/1250604242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 msgr2=0x561746fb7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb7ad0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f62e00118a0 tx=0x7f62e0011d60 comp rx=0 tx=0).stop 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 -- 172.21.15.7:0/1250604242 shutdown_connections 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561746fcab70 0x561746fccfa0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fbacb0 0x561746fb8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62cd7fa640 0 client.16197 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fb8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 --2- 172.21.15.7:0/1250604242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb7ad0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 -- 172.21.15.7:0/1250604242 >> 172.21.15.7:0/1250604242 conn(0x561746fac060 msgr2=0x561746fb1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 -- 172.21.15.7:0/1250604242 shutdown_connections 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.052+0000 7f62ee84b740 1 -- 172.21.15.7:0/1250604242 wait complete. 2024-09-10T10:57:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.053+0000 7f62ee84b740 20 client.16197 trim_cache size 0 max 16384 2024-09-10T10:57:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.069+0000 7f62ee84b740 1 Processor -- start 2024-09-10T10:57:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ee84b740 1 -- start start 2024-09-10T10:57:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fb6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fb8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fd2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ee84b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41b0180 con 0x561746fbacb0 2024-09-10T10:57:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ee84b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41ada50 con 0x561746fb0410 2024-09-10T10:57:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ee84b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41ab8c0 con 0x561746fafaf0 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ea049640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fd2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ea049640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fd2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45320/0 (socket says 172.21.15.7:45320) 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ea049640 1 -- 172.21.15.7:0/1366681783 learned_addr learned my addr 172.21.15.7:0/1366681783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62e9848640 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fb6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62e9047640 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fb8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ea049640 1 -- 172.21.15.7:0/1366681783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 msgr2=0x561746fb6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ea049640 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fb6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ea049640 1 -- 172.21.15.7:0/1366681783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 msgr2=0x561746fb8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ea049640 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fb8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ea049640 1 -- 172.21.15.7:0/1366681783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561746fabe90 con 0x561746fbacb0 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62e9047640 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fb8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62e9848640 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fb6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:57:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.070+0000 7f62ea049640 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fd2900 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f62dc031560 tx=0x7f62dc041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecc290de0076fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.071+0000 7f62cf7fe640 1 -- 172.21.15.7:0/1366681783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc003d70 con 0x561746fbacb0 2024-09-10T10:57:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.071+0000 7f62cf7fe640 1 -- 172.21.15.7:0/1366681783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62dc036040 con 0x561746fbacb0 2024-09-10T10:57:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.071+0000 7f62cf7fe640 1 -- 172.21.15.7:0/1366681783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc030b40 con 0x561746fbacb0 2024-09-10T10:57:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.071+0000 7f62ee84b740 1 -- 172.21.15.7:0/1366681783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 msgr2=0x561746fd2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.071+0000 7f62ee84b740 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fd2900 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f62dc031560 tx=0x7f62dc041000 comp rx=0 tx=0).stop 2024-09-10T10:57:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.071+0000 7f62ee84b740 1 -- 172.21.15.7:0/1366681783 shutdown_connections 2024-09-10T10:57:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.071+0000 7f62ee84b740 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fbacb0 0x561746fd2900 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.071+0000 7f62ee84b740 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fb8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.071+0000 7f62ee84b740 1 --2- 172.21.15.7:0/1366681783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fafaf0 0x561746fb6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.072+0000 7f62ee84b740 1 -- 172.21.15.7:0/1366681783 >> 172.21.15.7:0/1366681783 conn(0x561746fac060 msgr2=0x561746fb9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.072+0000 7f62ee84b740 1 -- 172.21.15.7:0/1366681783 shutdown_connections 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.072+0000 7f62ee84b740 1 -- 172.21.15.7:0/1366681783 wait complete. 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.072+0000 7f62ee84b740 1 -- start start 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.072+0000 7f62ee84b740 1 Processor -- start 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fc2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62ee84b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fbacb0 0x561746fc3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62ee84b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41abf10 con 0x561746fafaf0 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62ee84b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41ad380 con 0x561746fb0410 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62ee84b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41ac5e0 con 0x561746fbacb0 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62ee84b740 10 client.0 _ll_register_callbacks cb 0x7f62e000be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62e9848640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62e9848640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45330/0 (socket says 172.21.15.7:45330) 2024-09-10T10:57:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62e9848640 1 -- 172.21.15.7:0/389445282 learned_addr learned my addr 172.21.15.7:0/389445282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62e9047640 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fc2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62ea049640 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fbacb0 0x561746fc3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62e9848640 1 -- 172.21.15.7:0/389445282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fbacb0 msgr2=0x561746fc3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62e9848640 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fbacb0 0x561746fc3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62e9848640 1 -- 172.21.15.7:0/389445282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 msgr2=0x561746fc2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62e9848640 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fc2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62e9848640 1 -- 172.21.15.7:0/389445282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561746fc3840 con 0x561746fafaf0 2024-09-10T10:57:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62e9848640 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb5450 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f62e0021dd0 tx=0x7f62e0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4069be9f3b1dbfa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:57:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62cd7fa640 1 -- 172.21.15.7:0/389445282 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0021e50 con 0x561746fafaf0 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62cd7fa640 1 -- 172.21.15.7:0/389445282 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f62e00055f0 con 0x561746fafaf0 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.073+0000 7f62ee84b740 1 -- 172.21.15.7:0/389445282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62dc036040 con 0x561746fafaf0 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62cd7fa640 1 -- 172.21.15.7:0/389445282 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f62e0039050 con 0x561746fafaf0 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62ee84b740 10 client.16203 fetch_fsmap learned FSMap version 42 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62ee84b740 1 -- 172.21.15.7:0/389445282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5617470358d0 con 0x561746fafaf0 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62cd7fa640 1 -- 172.21.15.7:0/389445282 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62dc036040 con 0x561746fafaf0 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62cd7fa640 1 -- 172.21.15.7:0/389445282 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f62e0034030 con 0x561746fafaf0 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62ee84b740 10 client.16203 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62ee84b740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62ee84b740 20 client.16203 populate_metadata read hostname 'smithi007' 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62ee84b740 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561747036070 0x561747038490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62ee84b740 4 client.16203 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:57:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62ee84b740 1 -- 172.21.15.7:0/389445282 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561746f1bd70 con 0x561747036070 2024-09-10T10:57:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.074+0000 7f62e9047640 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561747036070 0x561747038490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.075+0000 7f62cd7fa640 10 client.16203 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.075+0000 7f62e9047640 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561747036070 0x561747038490 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c72586b1c766b1 server_cookie=3a84e52cebb10555 in_seq=0 out_seq=0 2024-09-10T10:57:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.116+0000 7f62cd7fa640 1 -- 172.21.15.7:0/389445282 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561746f1bd70 con 0x561747036070 2024-09-10T10:57:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.116+0000 7f62cd7fa640 10 client.16203 handle_command_reply: tid=0 2024-09-10T10:57:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.116+0000 7f62ee84b740 1 client.16203 shutdown 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 1 -- 172.21.15.7:0/389445282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 msgr2=0x561746fb5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb5450 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f62e0021dd0 tx=0x7f62e0005270 comp rx=0 tx=0).stop 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 1 -- 172.21.15.7:0/389445282 shutdown_connections 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x561747036070 0x561747038490 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561746fbacb0 0x561746fc3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561746fb0410 0x561746fc2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62cd7fa640 0 client.16203 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 1 --2- 172.21.15.7:0/389445282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561746fafaf0 0x561746fb5450 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 1 -- 172.21.15.7:0/389445282 >> 172.21.15.7:0/389445282 conn(0x561746fac060 msgr2=0x561746faa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 1 -- 172.21.15.7:0/389445282 shutdown_connections 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 1 -- 172.21.15.7:0/389445282 wait complete. 2024-09-10T10:57:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ee84b740 20 client.16203 trim_cache size 0 max 16384 2024-09-10T10:57:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:57:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:57:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:57:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.117+0000 7f62ebaac640 1 -- 172.21.15.7:0/2889879487 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62d00a2da0 msgr2=0x7f62d00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62d00a2da0 0x7f62d00a5260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009430 tx=0x7f62dc03f040 comp rx=0 tx=0).stop 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 -- 172.21.15.7:0/2889879487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e4102480 msgr2=0x7f62e4198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e4102480 0x7f62e4198850 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e9f0 tx=0x7f62d400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 -- 172.21.15.7:0/2889879487 shutdown_connections 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62e4107b80 0x7f62e419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e4102480 0x7f62e4198850 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f62d00a2da0 0x7f62d00a5260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 --2- 172.21.15.7:0/2889879487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e4101ae0 0x7f62e4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 -- 172.21.15.7:0/2889879487 >> 172.21.15.7:0/2889879487 conn(0x7f62e40fd7e0 msgr2=0x7f62e40fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 -- 172.21.15.7:0/2889879487 shutdown_connections 2024-09-10T10:57:39.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:39.118+0000 7f62ebaac640 1 -- 172.21.15.7:0/2889879487 wait complete. 2024-09-10T10:57:39.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1041537 7.88 MB/sec execute 519 sec latency 1459.810 ms 2024-09-10T10:57:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:39 smithi103 ceph-mon[35160]: pgmap v370: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 707 KiB/s wr, 75 op/s 2024-09-10T10:57:39.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:39 smithi007 ceph-mon[30873]: pgmap v370: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 707 KiB/s wr, 75 op/s 2024-09-10T10:57:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:39 smithi133 ceph-mon[35282]: pgmap v370: 97 pgs: 97 active+clean; 274 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 707 KiB/s wr, 75 op/s 2024-09-10T10:57:40.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1041537 7.87 MB/sec execute 520 sec latency 2459.949 ms 2024-09-10T10:57:41.540 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1041593 7.85 MB/sec execute 521 sec latency 3179.137 ms 2024-09-10T10:57:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:41 smithi103 ceph-mon[35160]: pgmap v371: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-09-10T10:57:41.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:41 smithi007 ceph-mon[30873]: pgmap v371: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-09-10T10:57:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:41 smithi133 ceph-mon[35282]: pgmap v371: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.3 MiB/s wr, 135 op/s 2024-09-10T10:57:42.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1041593 7.84 MB/sec execute 522 sec latency 1232.699 ms 2024-09-10T10:57:43.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1041593 7.82 MB/sec execute 523 sec latency 2232.817 ms 2024-09-10T10:57:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:43 smithi103 ceph-mon[35160]: pgmap v372: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 961 KiB/s wr, 98 op/s 2024-09-10T10:57:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:43 smithi007 ceph-mon[30873]: pgmap v372: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 961 KiB/s wr, 98 op/s 2024-09-10T10:57:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:43 smithi133 ceph-mon[35282]: pgmap v372: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 961 KiB/s wr, 98 op/s 2024-09-10T10:57:44.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1041593 7.81 MB/sec execute 524 sec latency 3232.925 ms 2024-09-10T10:57:45.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1041593 7.79 MB/sec execute 525 sec latency 4233.049 ms 2024-09-10T10:57:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:45 smithi103 ceph-mon[35160]: pgmap v373: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 961 KiB/s wr, 98 op/s 2024-09-10T10:57:45.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:45 smithi007 ceph-mon[30873]: pgmap v373: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 961 KiB/s wr, 98 op/s 2024-09-10T10:57:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:45 smithi133 ceph-mon[35282]: pgmap v373: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 961 KiB/s wr, 98 op/s 2024-09-10T10:57:46.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1042034 7.78 MB/sec execute 526 sec latency 4956.283 ms 2024-09-10T10:57:47.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1044524 7.80 MB/sec execute 527 sec latency 14.867 ms 2024-09-10T10:57:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:47 smithi103 ceph-mon[35160]: pgmap v374: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 995 KiB/s wr, 104 op/s 2024-09-10T10:57:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:47 smithi007 ceph-mon[30873]: pgmap v374: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 995 KiB/s wr, 104 op/s 2024-09-10T10:57:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:47 smithi133 ceph-mon[35282]: pgmap v374: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 995 KiB/s wr, 104 op/s 2024-09-10T10:57:48.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045081 7.78 MB/sec execute 528 sec latency 419.491 ms 2024-09-10T10:57:49.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045081 7.77 MB/sec execute 529 sec latency 1419.605 ms 2024-09-10T10:57:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:49 smithi103 ceph-mon[35160]: pgmap v375: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 675 KiB/s wr, 66 op/s 2024-09-10T10:57:49.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:49 smithi007 ceph-mon[30873]: pgmap v375: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 675 KiB/s wr, 66 op/s 2024-09-10T10:57:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:49 smithi133 ceph-mon[35282]: pgmap v375: 97 pgs: 97 active+clean; 273 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 675 KiB/s wr, 66 op/s 2024-09-10T10:57:50.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045081 7.75 MB/sec execute 530 sec latency 2419.702 ms 2024-09-10T10:57:51.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045137 7.74 MB/sec execute 531 sec latency 3152.962 ms 2024-09-10T10:57:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:51 smithi103 ceph-mon[35160]: pgmap v376: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-10T10:57:51.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:51 smithi007 ceph-mon[30873]: pgmap v376: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-10T10:57:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:51 smithi133 ceph-mon[35282]: pgmap v376: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-10T10:57:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.217+0000 7f0400959640 1 -- 172.21.15.7:0/1098707931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc1009e0 msgr2=0x7f03fc100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.217+0000 7f0400959640 1 --2- 172.21.15.7:0/1098707931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc1009e0 0x7f03fc100e40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009930 tx=0x7f03e402a420 comp rx=0 tx=0).stop 2024-09-10T10:57:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.217+0000 7f0400959640 1 -- 172.21.15.7:0/1098707931 shutdown_connections 2024-09-10T10:57:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.217+0000 7f0400959640 1 --2- 172.21.15.7:0/1098707931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc101380 0x7f03fc10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.217+0000 7f0400959640 1 --2- 172.21.15.7:0/1098707931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc1009e0 0x7f03fc100e40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.217+0000 7f0400959640 1 --2- 172.21.15.7:0/1098707931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc1069f0 0x7f03fc106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.217+0000 7f0400959640 1 -- 172.21.15.7:0/1098707931 >> 172.21.15.7:0/1098707931 conn(0x7f03fc0fc6c0 msgr2=0x7f03fc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.217+0000 7f0400959640 1 -- 172.21.15.7:0/1098707931 shutdown_connections 2024-09-10T10:57:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.217+0000 7f0400959640 1 -- 172.21.15.7:0/1098707931 wait complete. 2024-09-10T10:57:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.217+0000 7f0400959640 1 Processor -- start 2024-09-10T10:57:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f0400959640 1 -- start start 2024-09-10T10:57:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f0400959640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc1009e0 0x7f03fc1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f0400959640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc101380 0x7f03fc197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f0400959640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc1069f0 0x7f03fc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f0400959640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc110e30 con 0x7f03fc101380 2024-09-10T10:57:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f0400959640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc110cb0 con 0x7f03fc1009e0 2024-09-10T10:57:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f0400959640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc110fb0 con 0x7f03fc1069f0 2024-09-10T10:57:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f03f1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc101380 0x7f03fc197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f03f1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc101380 0x7f03fc197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54260/0 (socket says 172.21.15.7:54260) 2024-09-10T10:57:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f03f1d9b640 1 -- 172.21.15.7:0/3485406396 learned_addr learned my addr 172.21.15.7:0/3485406396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f03fa59c640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc1009e0 0x7f03fc1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.218+0000 7f03fad9d640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc1069f0 0x7f03fc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f03f1d9b640 1 -- 172.21.15.7:0/3485406396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc1069f0 msgr2=0x7f03fc19bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f03f1d9b640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc1069f0 0x7f03fc19bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f03f1d9b640 1 -- 172.21.15.7:0/3485406396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc1009e0 msgr2=0x7f03fc1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f03f1d9b640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc1009e0 0x7f03fc1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f03f1d9b640 1 -- 172.21.15.7:0/3485406396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc19c2d0 con 0x7f03fc101380 2024-09-10T10:57:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f03f1d9b640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc101380 0x7f03fc197820 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009230 tx=0x7f03e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f03f3fff640 1 -- 172.21.15.7:0/3485406396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e403f030 con 0x7f03fc101380 2024-09-10T10:57:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f0400959640 1 -- 172.21.15.7:0/3485406396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc19c560 con 0x7f03fc101380 2024-09-10T10:57:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f03f3fff640 1 -- 172.21.15.7:0/3485406396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03e4003ad0 con 0x7f03fc101380 2024-09-10T10:57:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f03f3fff640 1 -- 172.21.15.7:0/3485406396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4035070 con 0x7f03fc101380 2024-09-10T10:57:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.219+0000 7f0400959640 1 -- 172.21.15.7:0/3485406396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc1a3e50 con 0x7f03fc101380 2024-09-10T10:57:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.222+0000 7f04037d1740 1 Processor -- start 2024-09-10T10:57:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f04037d1740 1 -- start start 2024-09-10T10:57:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03f3fff640 1 -- 172.21.15.7:0/3485406396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f03e4042030 con 0x7f03fc101380 2024-09-10T10:57:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f04037d1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce231ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f04037d1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f04037d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce23f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f04037d1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a5ce1962f0 con 0x55a5ce232410 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f04037d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5ce13e270 con 0x55a5ce23ccb0 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f04037d1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a5ce1922b0 con 0x55a5ce231af0 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54270/0 (socket says 172.21.15.7:54270) 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03fa59c640 1 -- 172.21.15.7:0/35075 learned_addr learned my addr 172.21.15.7:0/35075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03fad9d640 1 --2- 172.21.15.7:0/35075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce231ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03f1d9b640 1 --2- 172.21.15.7:0/35075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce23f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03fa59c640 1 -- 172.21.15.7:0/35075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 msgr2=0x55a5ce231ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03fa59c640 1 --2- 172.21.15.7:0/35075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce231ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03fa59c640 1 -- 172.21.15.7:0/35075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 msgr2=0x55a5ce23f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03fa59c640 1 --2- 172.21.15.7:0/35075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce23f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03fa59c640 1 -- 172.21.15.7:0/35075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ce13d340 con 0x55a5ce232410 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.223+0000 7f03f1d9b640 1 --2- 172.21.15.7:0/35075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce23f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.224+0000 7f03f3fff640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03e80a2da0 0x7f03e80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.224+0000 7f03fa59c640 1 --2- 172.21.15.7:0/35075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23c770 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009c70 tx=0x7f03ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66217fa129fff88b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.224+0000 7f03fad9d640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03e80a2da0 0x7f03e80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.224+0000 7f03f3fff640 1 -- 172.21.15.7:0/3485406396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f03e40efae0 con 0x7f03fc101380 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.224+0000 7f03f0d99640 1 -- 172.21.15.7:0/35075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec004230 con 0x55a5ce232410 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.224+0000 7f03f0d99640 1 -- 172.21.15.7:0/35075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03ec0043d0 con 0x55a5ce232410 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.224+0000 7f03f0d99640 1 -- 172.21.15.7:0/35075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec019820 con 0x55a5ce232410 2024-09-10T10:57:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 -- 172.21.15.7:0/35075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 msgr2=0x55a5ce23c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f03fad9d640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03e80a2da0 0x7f03e80a5260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f03f4005fd0 tx=0x7f03f4009600 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 --2- 172.21.15.7:0/35075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23c770 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009c70 tx=0x7f03ec011810 comp rx=0 tx=0).stop 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 -- 172.21.15.7:0/35075 shutdown_connections 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 --2- 172.21.15.7:0/35075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce23f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 --2- 172.21.15.7:0/35075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23c770 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 --2- 172.21.15.7:0/35075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce231ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 -- 172.21.15.7:0/35075 >> 172.21.15.7:0/35075 conn(0x55a5ce22e060 msgr2=0x55a5ce22e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 -- 172.21.15.7:0/35075 shutdown_connections 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 -- 172.21.15.7:0/35075 wait complete. 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 -- start start 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.225+0000 7f04037d1740 1 Processor -- start 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f04037d1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce239bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f04037d1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f04037d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce23a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f04037d1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a5ce23f880 con 0x55a5ce232410 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f04037d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a5ce11f290 con 0x55a5ce23ccb0 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f04037d1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a5ce23fa00 con 0x55a5ce231af0 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f04037d1740 10 client.0 _ll_register_callbacks cb 0x55a5ce150140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03f1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03f1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54274/0 (socket says 172.21.15.7:54274) 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03f1d9b640 1 -- 172.21.15.7:0/2364531261 learned_addr learned my addr 172.21.15.7:0/2364531261 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03fa59c640 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce239bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03fad9d640 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce23a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03f1d9b640 1 -- 172.21.15.7:0/2364531261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 msgr2=0x55a5ce239bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03f1d9b640 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce239bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03f1d9b640 1 -- 172.21.15.7:0/2364531261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 msgr2=0x55a5ce23a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03f1d9b640 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce23a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03f1d9b640 1 -- 172.21.15.7:0/2364531261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5cdea9210 con 0x55a5ce232410 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03f1d9b640 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23a0f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f03e40bbcd0 tx=0x7f03e40bbbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36e93c8f39a5ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03e1ffb640 1 -- 172.21.15.7:0/2364531261 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4049050 con 0x55a5ce232410 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f03e1ffb640 1 -- 172.21.15.7:0/2364531261 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f03e4003c80 con 0x55a5ce232410 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.226+0000 7f04037d1740 1 -- 172.21.15.7:0/2364531261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03ec0043d0 con 0x55a5ce232410 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f03e1ffb640 1 -- 172.21.15.7:0/2364531261 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f03e40bbfd0 con 0x55a5ce232410 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f04037d1740 10 client.16215 fetch_fsmap learned FSMap version 42 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f04037d1740 1 -- 172.21.15.7:0/2364531261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a5cdf75620 con 0x55a5ce232410 2024-09-10T10:57:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f03e1ffb640 1 -- 172.21.15.7:0/2364531261 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03ec0043d0 con 0x55a5ce232410 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f03e1ffb640 1 -- 172.21.15.7:0/2364531261 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f03e40b8640 con 0x55a5ce232410 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f04037d1740 10 client.16215 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f04037d1740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f04037d1740 20 client.16215 populate_metadata read hostname 'smithi007' 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f04037d1740 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a5ce24cbd0 0x55a5ce24f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f04037d1740 4 client.16215 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f04037d1740 1 -- 172.21.15.7:0/2364531261 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5ce1a3a20 con 0x55a5ce24cbd0 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.227+0000 7f03fa59c640 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a5ce24cbd0 0x55a5ce24f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.228+0000 7f03fa59c640 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a5ce24cbd0 0x55a5ce24f000 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6250255fee4c5931 server_cookie=e4bad0df36e63a37 in_seq=0 out_seq=0 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.228+0000 7f03e1ffb640 10 client.16215 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.228+0000 7f03e1ffb640 1 -- 172.21.15.7:0/2364531261 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5ce1a3a20 con 0x55a5ce24cbd0 2024-09-10T10:57:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.228+0000 7f03e1ffb640 10 client.16215 handle_command_reply: tid=0 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 client.16215 shutdown 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 -- 172.21.15.7:0/2364531261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 msgr2=0x55a5ce23a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23a0f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f03e40bbcd0 tx=0x7f03e40bbbd0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 -- 172.21.15.7:0/2364531261 shutdown_connections 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a5ce24cbd0 0x55a5ce24f000 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce23a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce232410 0x55a5ce23a0f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 --2- 172.21.15.7:0/2364531261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce239bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f03e1ffb640 0 client.16215 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 -- 172.21.15.7:0/2364531261 >> 172.21.15.7:0/2364531261 conn(0x55a5ce22e060 msgr2=0x55a5ce233d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 -- 172.21.15.7:0/2364531261 shutdown_connections 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 1 -- 172.21.15.7:0/2364531261 wait complete. 2024-09-10T10:57:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.229+0000 7f04037d1740 20 client.16215 trim_cache size 0 max 16384 2024-09-10T10:57:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f04037d1740 1 Processor -- start 2024-09-10T10:57:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f04037d1740 1 -- start start 2024-09-10T10:57:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f04037d1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce231af0 0x55a5ce234560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f04037d1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce232410 0x55a5ce23a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f04037d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce254910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f04037d1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1aeff0 con 0x55a5ce231af0 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f04037d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ac8c0 con 0x55a5ce23ccb0 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f04037d1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1aa730 con 0x55a5ce232410 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f03fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce231af0 0x55a5ce234560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f03fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce231af0 0x55a5ce234560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54282/0 (socket says 172.21.15.7:54282) 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f03fa59c640 1 -- 172.21.15.7:0/3810712470 learned_addr learned my addr 172.21.15.7:0/3810712470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f03f1d9b640 1 --2- 172.21.15.7:0/3810712470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce232410 0x55a5ce23a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f03fa59c640 1 -- 172.21.15.7:0/3810712470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce232410 msgr2=0x55a5ce23a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f03fa59c640 1 --2- 172.21.15.7:0/3810712470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce232410 0x55a5ce23a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f03fa59c640 1 -- 172.21.15.7:0/3810712470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 msgr2=0x55a5ce254910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f03fad9d640 1 --2- 172.21.15.7:0/3810712470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce254910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f03fa59c640 1 --2- 172.21.15.7:0/3810712470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce254910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.244+0000 7f03fa59c640 1 -- 172.21.15.7:0/3810712470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ce23c700 con 0x55a5ce231af0 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f03fa59c640 1 --2- 172.21.15.7:0/3810712470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce231af0 0x55a5ce234560 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009c70 tx=0x7f03ec0193e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc457483f6b60877 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f03e3fff640 1 -- 172.21.15.7:0/3810712470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec005070 con 0x55a5ce231af0 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f03e3fff640 1 -- 172.21.15.7:0/3810712470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03ec02b070 con 0x55a5ce231af0 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f03e3fff640 1 -- 172.21.15.7:0/3810712470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0194f0 con 0x55a5ce231af0 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f04037d1740 1 -- 172.21.15.7:0/3810712470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce231af0 msgr2=0x55a5ce234560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f04037d1740 1 --2- 172.21.15.7:0/3810712470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce231af0 0x55a5ce234560 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009c70 tx=0x7f03ec0193e0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f04037d1740 1 -- 172.21.15.7:0/3810712470 shutdown_connections 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f04037d1740 1 --2- 172.21.15.7:0/3810712470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce23ccb0 0x55a5ce254910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f04037d1740 1 --2- 172.21.15.7:0/3810712470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce232410 0x55a5ce23a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f04037d1740 1 --2- 172.21.15.7:0/3810712470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce231af0 0x55a5ce234560 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f04037d1740 1 -- 172.21.15.7:0/3810712470 >> 172.21.15.7:0/3810712470 conn(0x55a5ce22e060 msgr2=0x55a5ce23bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f04037d1740 1 -- 172.21.15.7:0/3810712470 shutdown_connections 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.245+0000 7f04037d1740 1 -- 172.21.15.7:0/3810712470 wait complete. 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f04037d1740 1 -- start start 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f04037d1740 1 Processor -- start 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f04037d1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce236ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f04037d1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce232410 0x55a5ce237020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f04037d1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce23ccb0 0x55a5ce237560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f04037d1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1aad80 con 0x55a5ce23ccb0 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f04037d1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ac1f0 con 0x55a5ce232410 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f04037d1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1ab450 con 0x55a5ce231af0 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f04037d1740 10 client.0 _ll_register_callbacks cb 0x7f03e4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f03fad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce23ccb0 0x55a5ce237560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f03fad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce23ccb0 0x55a5ce237560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54296/0 (socket says 172.21.15.7:54296) 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f03fad9d640 1 -- 172.21.15.7:0/3526465185 learned_addr learned my addr 172.21.15.7:0/3526465185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f03fa59c640 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce236ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.246+0000 7f03f1d9b640 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce232410 0x55a5ce237020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03fad9d640 1 -- 172.21.15.7:0/3526465185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 msgr2=0x55a5ce236ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03fad9d640 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce236ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03fad9d640 1 -- 172.21.15.7:0/3526465185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce232410 msgr2=0x55a5ce237020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03fad9d640 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce232410 0x55a5ce237020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03fad9d640 1 -- 172.21.15.7:0/3526465185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ce2b8a00 con 0x55a5ce23ccb0 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03fad9d640 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce23ccb0 0x55a5ce237560 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f03f4012a20 tx=0x7f03f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1edb7efb3da1124 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03e1ffb640 1 -- 172.21.15.7:0/3526465185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f402a440 con 0x55a5ce23ccb0 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03e1ffb640 1 -- 172.21.15.7:0/3526465185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8564+0+0 (secure 0 0 0) 0x7f03f401f450 con 0x55a5ce23ccb0 2024-09-10T10:57:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f04037d1740 1 -- 172.21.15.7:0/3526465185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03ec02b070 con 0x55a5ce23ccb0 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03e1ffb640 1 -- 172.21.15.7:0/3526465185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f03f401f650 con 0x55a5ce23ccb0 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f04037d1740 10 client.16221 fetch_fsmap learned FSMap version 42 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f04037d1740 1 -- 172.21.15.7:0/3526465185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a5ce2b9d50 con 0x55a5ce23ccb0 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.247+0000 7f03e1ffb640 1 -- 172.21.15.7:0/3526465185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03ec02b070 con 0x55a5ce23ccb0 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.248+0000 7f03e1ffb640 1 -- 172.21.15.7:0/3526465185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f03f4028070 con 0x55a5ce23ccb0 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.248+0000 7f04037d1740 10 client.16221 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.248+0000 7f04037d1740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.248+0000 7f04037d1740 20 client.16221 populate_metadata read hostname 'smithi007' 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.248+0000 7f04037d1740 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a5ce2ba520 0x55a5ce2bc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.248+0000 7f04037d1740 4 client.16221 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.248+0000 7f04037d1740 1 -- 172.21.15.7:0/3526465185 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5ce19dd70 con 0x55a5ce2ba520 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.248+0000 7f03fa59c640 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a5ce2ba520 0x55a5ce2bc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:57:52.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.248+0000 7f03fa59c640 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a5ce2ba520 0x55a5ce2bc940 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab79f42bf935d5cd server_cookie=d6e06c4bf44d9ede in_seq=0 out_seq=0 2024-09-10T10:57:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.248+0000 7f03e1ffb640 10 client.16221 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f03e1ffb640 1 -- 172.21.15.7:0/3526465185 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5ce19dd70 con 0x55a5ce2ba520 2024-09-10T10:57:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f03e1ffb640 10 client.16221 handle_command_reply: tid=0 2024-09-10T10:57:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 client.16221 shutdown 2024-09-10T10:57:52.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 -- 172.21.15.7:0/3526465185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce23ccb0 msgr2=0x55a5ce237560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce23ccb0 0x55a5ce237560 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f03f4012a20 tx=0x7f03f4004a40 comp rx=0 tx=0).stop 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 -- 172.21.15.7:0/3526465185 shutdown_connections 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a5ce2ba520 0x55a5ce2bc940 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a5ce23ccb0 0x55a5ce237560 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f03e1ffb640 0 client.16221 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a5ce232410 0x55a5ce237020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 --2- 172.21.15.7:0/3526465185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5ce231af0 0x55a5ce236ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 -- 172.21.15.7:0/3526465185 >> 172.21.15.7:0/3526465185 conn(0x55a5ce22e060 msgr2=0x55a5ce22c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 -- 172.21.15.7:0/3526465185 shutdown_connections 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 1 -- 172.21.15.7:0/3526465185 wait complete. 2024-09-10T10:57:52.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.288+0000 7f04037d1740 20 client.16221 trim_cache size 0 max 16384 2024-09-10T10:57:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:57:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:57:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:57:52.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 -- 172.21.15.7:0/3485406396 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03e80a2da0 msgr2=0x7f03e80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03e80a2da0 0x7f03e80a5260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f03f4005fd0 tx=0x7f03f4009600 comp rx=0 tx=0).stop 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 -- 172.21.15.7:0/3485406396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc101380 msgr2=0x7f03fc197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc101380 0x7f03fc197820 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009230 tx=0x7f03e4004270 comp rx=0 tx=0).stop 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 -- 172.21.15.7:0/3485406396 shutdown_connections 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03fc1069f0 0x7f03fc19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03e80a2da0 0x7f03e80a5260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc101380 0x7f03fc197820 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 --2- 172.21.15.7:0/3485406396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc1009e0 0x7f03fc1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 -- 172.21.15.7:0/3485406396 >> 172.21.15.7:0/3485406396 conn(0x7f03fc0fc6c0 msgr2=0x7f03fc0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:57:52.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 -- 172.21.15.7:0/3485406396 shutdown_connections 2024-09-10T10:57:52.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:57:52.289+0000 7f0400959640 1 -- 172.21.15.7:0/3485406396 wait complete. 2024-09-10T10:57:52.541 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045137 7.73 MB/sec execute 532 sec latency 1229.249 ms 2024-09-10T10:57:53.542 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045137 7.71 MB/sec execute 533 sec latency 2229.381 ms 2024-09-10T10:57:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:53 smithi103 ceph-mon[35160]: pgmap v377: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 601 KiB/s wr, 56 op/s 2024-09-10T10:57:53.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:53 smithi007 ceph-mon[30873]: pgmap v377: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 601 KiB/s wr, 56 op/s 2024-09-10T10:57:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:53 smithi133 ceph-mon[35282]: pgmap v377: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 601 KiB/s wr, 56 op/s 2024-09-10T10:57:54.542 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045137 7.70 MB/sec execute 534 sec latency 3229.479 ms 2024-09-10T10:57:55.542 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045137 7.68 MB/sec execute 535 sec latency 4229.592 ms 2024-09-10T10:57:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:55 smithi103 ceph-mon[35160]: pgmap v378: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 601 KiB/s wr, 56 op/s 2024-09-10T10:57:55.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:55 smithi007 ceph-mon[30873]: pgmap v378: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 601 KiB/s wr, 56 op/s 2024-09-10T10:57:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:55 smithi133 ceph-mon[35282]: pgmap v378: 97 pgs: 97 active+clean; 276 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 601 KiB/s wr, 56 op/s 2024-09-10T10:57:56.542 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045206 7.67 MB/sec execute 536 sec latency 4951.575 ms 2024-09-10T10:57:57.542 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045206 7.65 MB/sec execute 537 sec latency 1224.950 ms 2024-09-10T10:57:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:57 smithi103 ceph-mon[35160]: pgmap v379: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-09-10T10:57:57.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:57 smithi007 ceph-mon[30873]: pgmap v379: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-09-10T10:57:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:57 smithi133 ceph-mon[35282]: pgmap v379: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-09-10T10:57:58.542 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045206 7.64 MB/sec execute 538 sec latency 2225.028 ms 2024-09-10T10:57:59.542 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045206 7.63 MB/sec execute 539 sec latency 3225.168 ms 2024-09-10T10:57:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:57:59 smithi103 ceph-mon[35160]: pgmap v380: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-09-10T10:57:59.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:57:59 smithi007 ceph-mon[30873]: pgmap v380: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-09-10T10:57:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:57:59 smithi133 ceph-mon[35282]: pgmap v380: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-09-10T10:58:00.542 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1045329 7.61 MB/sec execute 540 sec latency 4133.717 ms 2024-09-10T10:58:01.542 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1046823 7.61 MB/sec execute 541 sec latency 28.258 ms 2024-09-10T10:58:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:01 smithi103 ceph-mon[35160]: pgmap v381: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-09-10T10:58:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:01 smithi007 ceph-mon[30873]: pgmap v381: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-09-10T10:58:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:01 smithi133 ceph-mon[35282]: pgmap v381: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-09-10T10:58:02.542 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1048631 7.62 MB/sec execute 542 sec latency 11.077 ms 2024-09-10T10:58:03.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1048755 7.60 MB/sec execute 543 sec latency 896.344 ms 2024-09-10T10:58:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:03 smithi007 ceph-mon[30873]: pgmap v382: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 855 KiB/s wr, 51 op/s 2024-09-10T10:58:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:03 smithi133 ceph-mon[35282]: pgmap v382: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 855 KiB/s wr, 51 op/s 2024-09-10T10:58:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:03 smithi103 ceph-mon[35160]: pgmap v382: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 855 KiB/s wr, 51 op/s 2024-09-10T10:58:04.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1048755 7.59 MB/sec execute 544 sec latency 1896.489 ms 2024-09-10T10:58:04.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:04 smithi007 ceph-mon[30873]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:58:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:04 smithi133 ceph-mon[35282]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:58:05.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:04 smithi103 ceph-mon[35160]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:58:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.389+0000 7f880ed78640 1 -- 172.21.15.7:0/1918915870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8808101370 msgr2=0x7f880810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.389+0000 7f880ed78640 1 --2- 172.21.15.7:0/1918915870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8808101370 0x7f880810d8a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8800009cd0 tx=0x7f880002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 -- 172.21.15.7:0/1918915870 shutdown_connections 2024-09-10T10:58:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 --2- 172.21.15.7:0/1918915870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8808101370 0x7f880810d8a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 --2- 172.21.15.7:0/1918915870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88081009d0 0x7f8808100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 --2- 172.21.15.7:0/1918915870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88081069e0 0x7f8808106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 -- 172.21.15.7:0/1918915870 >> 172.21.15.7:0/1918915870 conn(0x7f88080fc6b0 msgr2=0x7f88080fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 -- 172.21.15.7:0/1918915870 shutdown_connections 2024-09-10T10:58:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 -- 172.21.15.7:0/1918915870 wait complete. 2024-09-10T10:58:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 Processor -- start 2024-09-10T10:58:05.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 -- start start 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88081009d0 0x7f8808198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.390+0000 7f880ed78640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8808101370 0x7f88081988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880ed78640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88081069e0 0x7f880819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880ed78640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8808110e20 con 0x7f88081009d0 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880ed78640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8808110ca0 con 0x7f88081069e0 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880ed78640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8808110fa0 con 0x7f8808101370 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880cb14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88081009d0 0x7f8808198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880cb14640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88081009d0 0x7f8808198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45712/0 (socket says 172.21.15.7:45712) 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880cb14640 1 -- 172.21.15.7:0/477671367 learned_addr learned my addr 172.21.15.7:0/477671367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f8807fff640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8808101370 0x7f88081988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880d315640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88081069e0 0x7f880819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880cb14640 1 -- 172.21.15.7:0/477671367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8808101370 msgr2=0x7f88081988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880cb14640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8808101370 0x7f88081988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880cb14640 1 -- 172.21.15.7:0/477671367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88081069e0 msgr2=0x7f880819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880cb14640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88081069e0 0x7f880819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880cb14640 1 -- 172.21.15.7:0/477671367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880819d380 con 0x7f88081009d0 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f8807fff640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8808101370 0x7f88081988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f880cb14640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88081009d0 0x7f8808198390 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00e9f0 tx=0x7f87fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.391+0000 7f8805ffb640 1 -- 172.21.15.7:0/477671367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00cd80 con 0x7f88081009d0 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.392+0000 7f880ed78640 1 -- 172.21.15.7:0/477671367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880819d670 con 0x7f88081009d0 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.392+0000 7f8805ffb640 1 -- 172.21.15.7:0/477671367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc010910 con 0x7f88081009d0 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.392+0000 7f8805ffb640 1 -- 172.21.15.7:0/477671367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0188f0 con 0x7f88081009d0 2024-09-10T10:58:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.392+0000 7f880ed78640 1 -- 172.21.15.7:0/477671367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88081a4f00 con 0x7f88081009d0 2024-09-10T10:58:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.394+0000 7f8805ffb640 1 -- 172.21.15.7:0/477671367 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f87fc010ac0 con 0x7f88081009d0 2024-09-10T10:58:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.394+0000 7f8811bf1740 1 Processor -- start 2024-09-10T10:58:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.394+0000 7f8811bf1740 1 -- start start 2024-09-10T10:58:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8811bf1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa7b6422f0 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8811bf1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fa7b5ea270 con 0x55fa7b6ddaf0 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8811bf1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fa7b63e2b0 con 0x55fa7b6de410 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8807fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8807fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45728/0 (socket says 172.21.15.7:45728) 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8807fff640 1 -- 172.21.15.7:0/2309311258 learned_addr learned my addr 172.21.15.7:0/2309311258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8805ffb640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f87f80a2e20 0x7f87f80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f880cb14640 1 --2- 172.21.15.7:0/2309311258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f880d315640 1 --2- 172.21.15.7:0/2309311258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8807fff640 1 -- 172.21.15.7:0/2309311258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 msgr2=0x55fa7b6e8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8807fff640 1 --2- 172.21.15.7:0/2309311258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6e8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8807fff640 1 -- 172.21.15.7:0/2309311258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 msgr2=0x55fa7b6dded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8807fff640 1 --2- 172.21.15.7:0/2309311258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6dded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8807fff640 1 -- 172.21.15.7:0/2309311258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7b5e9340 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f880d315640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f87f80a2e20 0x7f87f80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.395+0000 7f8805ffb640 1 -- 172.21.15.7:0/477671367 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8608+0+0 (secure 0 0 0) 0x7f87fc0cf860 con 0x7f88081009d0 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.396+0000 7f8807fff640 1 --2- 172.21.15.7:0/2309311258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6eb0a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009cf0 tx=0x7f87f40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b7f6f93492c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.396+0000 7f87f2ffd640 1 -- 172.21.15.7:0/2309311258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4012450 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.396+0000 7f87f2ffd640 1 -- 172.21.15.7:0/2309311258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f4012ad0 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.396+0000 7f87f2ffd640 1 -- 172.21.15.7:0/2309311258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f400fc50 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 -- 172.21.15.7:0/2309311258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 msgr2=0x55fa7b6eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f880d315640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f87f80a2e20 0x7f87f80a52e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8800009a60 tx=0x7f88000095c0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 --2- 172.21.15.7:0/2309311258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6eb0a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009cf0 tx=0x7f87f40117b0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 -- 172.21.15.7:0/2309311258 shutdown_connections 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 --2- 172.21.15.7:0/2309311258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6eb0a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 --2- 172.21.15.7:0/2309311258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 --2- 172.21.15.7:0/2309311258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 -- 172.21.15.7:0/2309311258 >> 172.21.15.7:0/2309311258 conn(0x55fa7b6da060 msgr2=0x55fa7b6da440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 -- 172.21.15.7:0/2309311258 shutdown_connections 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 -- 172.21.15.7:0/2309311258 wait complete. 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 -- start start 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.397+0000 7f8811bf1740 1 Processor -- start 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6e6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f8811bf1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa7b6eb880 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f8811bf1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fa7b5cb290 con 0x55fa7b6ddaf0 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f8811bf1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fa7b6eba00 con 0x55fa7b6de410 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f8811bf1740 10 client.0 _ll_register_callbacks cb 0x55fa7b5fc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880d315640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880d315640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6e65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45738/0 (socket says 172.21.15.7:45738) 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880d315640 1 -- 172.21.15.7:0/3470495867 learned_addr learned my addr 172.21.15.7:0/3470495867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f8807fff640 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880cb14640 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880d315640 1 -- 172.21.15.7:0/3470495867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 msgr2=0x55fa7b6e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880d315640 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880d315640 1 -- 172.21.15.7:0/3470495867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 msgr2=0x55fa7b6e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880d315640 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6e5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880d315640 1 -- 172.21.15.7:0/3470495867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7b355210 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f8807fff640 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880cb14640 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f880d315640 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6e65a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8800009c60 tx=0x7f8800005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b857d53a3b835cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.398+0000 7f87e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f87e7fff640 1 -- 172.21.15.7:0/3470495867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880004c070 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f87e7fff640 1 -- 172.21.15.7:0/3470495867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8608+0+0 (secure 0 0 0) 0x7f8800004450 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f8811bf1740 1 -- 172.21.15.7:0/3470495867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f4012ad0 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f87e7fff640 1 -- 172.21.15.7:0/3470495867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8800043070 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f8811bf1740 10 client.16233 fetch_fsmap learned FSMap version 42 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f8811bf1740 1 -- 172.21.15.7:0/3470495867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fa7b421620 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f87e7fff640 1 -- 172.21.15.7:0/3470495867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f4012ad0 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f87e7fff640 1 -- 172.21.15.7:0/3470495867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f8800030070 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f8811bf1740 10 client.16233 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f8811bf1740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f8811bf1740 20 client.16233 populate_metadata read hostname 'smithi007' 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fa7b6f8c00 0x55fa7b6fb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f8811bf1740 4 client.16233 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.399+0000 7f8811bf1740 1 -- 172.21.15.7:0/3470495867 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa7b64fa20 con 0x55fa7b6f8c00 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.400+0000 7f880cb14640 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fa7b6f8c00 0x55fa7b6fb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.400+0000 7f87e7fff640 10 client.16233 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.400+0000 7f880cb14640 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fa7b6f8c00 0x55fa7b6fb010 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b96a0b74dfe314 server_cookie=9d45e4e6ba58cca4 in_seq=0 out_seq=0 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f87e7fff640 1 -- 172.21.15.7:0/3470495867 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa7b64fa20 con 0x55fa7b6f8c00 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f87e7fff640 10 client.16233 handle_command_reply: tid=0 2024-09-10T10:58:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 client.16233 shutdown 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 -- 172.21.15.7:0/3470495867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 msgr2=0x55fa7b6e65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6e65a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8800009c60 tx=0x7f8800005f50 comp rx=0 tx=0).stop 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 -- 172.21.15.7:0/3470495867 shutdown_connections 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fa7b6f8c00 0x55fa7b6fb010 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6e65a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f87e7fff640 0 client.16233 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3470495867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 -- 172.21.15.7:0/3470495867 >> 172.21.15.7:0/3470495867 conn(0x55fa7b6da060 msgr2=0x55fa7b6dfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 -- 172.21.15.7:0/3470495867 shutdown_connections 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 1 -- 172.21.15.7:0/3470495867 wait complete. 2024-09-10T10:58:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.401+0000 7f8811bf1740 20 client.16233 trim_cache size 0 max 16384 2024-09-10T10:58:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8811bf1740 1 Processor -- start 2024-09-10T10:58:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8811bf1740 1 -- start start 2024-09-10T10:58:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6de410 0x55fa7b6e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b700920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8811bf1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b0170 con 0x55fa7b6de410 2024-09-10T10:58:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8811bf1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ada40 con 0x55fa7b6ddaf0 2024-09-10T10:58:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8811bf1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ab890 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8807fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6de410 0x55fa7b6e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8807fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6de410 0x55fa7b6e63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45752/0 (socket says 172.21.15.7:45752) 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f8807fff640 1 -- 172.21.15.7:0/1538078486 learned_addr learned my addr 172.21.15.7:0/1538078486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.416+0000 7f880d315640 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b700920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f880cb14640 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f8807fff640 1 -- 172.21.15.7:0/1538078486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6e8cb0 msgr2=0x55fa7b700920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f8807fff640 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b700920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f8807fff640 1 -- 172.21.15.7:0/1538078486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 msgr2=0x55fa7b6e43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f8807fff640 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6e43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f8807fff640 1 -- 172.21.15.7:0/1538078486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7b6e4ab0 con 0x55fa7b6de410 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f880d315640 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b700920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:58:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f880cb14640 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6e43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f8807fff640 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6de410 0x55fa7b6e63d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009c10 tx=0x7f87f4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8713245f5727768e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f87f27fc640 1 -- 172.21.15.7:0/1538078486 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4009ea0 con 0x55fa7b6de410 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f87f27fc640 1 -- 172.21.15.7:0/1538078486 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f40055a0 con 0x55fa7b6de410 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.417+0000 7f87f27fc640 1 -- 172.21.15.7:0/1538078486 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f400f7e0 con 0x55fa7b6de410 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 -- 172.21.15.7:0/1538078486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6de410 msgr2=0x55fa7b6e63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6de410 0x55fa7b6e63d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009c10 tx=0x7f87f4014040 comp rx=0 tx=0).stop 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 -- 172.21.15.7:0/1538078486 shutdown_connections 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b700920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6de410 0x55fa7b6e63d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 --2- 172.21.15.7:0/1538078486 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6e43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 -- 172.21.15.7:0/1538078486 >> 172.21.15.7:0/1538078486 conn(0x55fa7b6da060 msgr2=0x55fa7b6e7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 -- 172.21.15.7:0/1538078486 shutdown_connections 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 -- 172.21.15.7:0/1538078486 wait complete. 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 -- start start 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 Processor -- start 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88081abf00 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ad370 con 0x55fa7b6ddaf0 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ac5d0 con 0x55fa7b6de410 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.418+0000 7f8811bf1740 10 client.0 _ll_register_callbacks cb 0x7f87f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f880d315640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f880d315640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6ecb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45760/0 (socket says 172.21.15.7:45760) 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f880d315640 1 -- 172.21.15.7:0/3095913302 learned_addr learned my addr 172.21.15.7:0/3095913302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f8807fff640 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f880cb14640 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f880d315640 1 -- 172.21.15.7:0/3095913302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 msgr2=0x55fa7b6ec630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f880d315640 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6ec630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f880d315640 1 -- 172.21.15.7:0/3095913302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 msgr2=0x55fa7b6f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f880d315640 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6f11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f880d315640 1 -- 172.21.15.7:0/3095913302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7b6de240 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f880d315640 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6ecb70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8800004d70 tx=0x7f8800004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab22549a09778ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f87e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f87e7fff640 1 -- 172.21.15.7:0/3095913302 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800066060 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f87e7fff640 1 -- 172.21.15.7:0/3095913302 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8608+0+0 (secure 0 0 0) 0x7f8800051c60 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f8811bf1740 1 -- 172.21.15.7:0/3095913302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f40055a0 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f87e7fff640 1 -- 172.21.15.7:0/3095913302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f880004c040 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f8811bf1740 10 client.16239 fetch_fsmap learned FSMap version 42 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.419+0000 7f8811bf1740 1 -- 172.21.15.7:0/3095913302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fa7b763490 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f87e7fff640 1 -- 172.21.15.7:0/3095913302 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f40055a0 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f87e7fff640 1 -- 172.21.15.7:0/3095913302 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f88000305a0 con 0x55fa7b6e8cb0 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f8811bf1740 10 client.16239 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f8811bf1740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f8811bf1740 20 client.16239 populate_metadata read hostname 'smithi007' 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fa7b763b80 0x55fa7b765fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f8811bf1740 4 client.16239 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:58:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f8811bf1740 1 -- 172.21.15.7:0/3095913302 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa7b649d70 con 0x55fa7b763b80 2024-09-10T10:58:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f880cb14640 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fa7b763b80 0x55fa7b765fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f880cb14640 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fa7b763b80 0x55fa7b765fa0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511d68e9ff316b71 server_cookie=42ce986b07b81b0e in_seq=0 out_seq=0 2024-09-10T10:58:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.420+0000 7f87e7fff640 10 client.16239 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f87e7fff640 1 -- 172.21.15.7:0/3095913302 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa7b649d70 con 0x55fa7b763b80 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f87e7fff640 10 client.16239 handle_command_reply: tid=0 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 client.16239 shutdown 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 -- 172.21.15.7:0/3095913302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 msgr2=0x55fa7b6ecb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6ecb70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8800004d70 tx=0x7f8800004da0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 -- 172.21.15.7:0/3095913302 shutdown_connections 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fa7b763b80 0x55fa7b765fa0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa7b6e8cb0 0x55fa7b6ecb70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f87e7fff640 0 client.16239 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa7b6de410 0x55fa7b6ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 --2- 172.21.15.7:0/3095913302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fa7b6ddaf0 0x55fa7b6f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 -- 172.21.15.7:0/3095913302 >> 172.21.15.7:0/3095913302 conn(0x55fa7b6da060 msgr2=0x55fa7b6e7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 -- 172.21.15.7:0/3095913302 shutdown_connections 2024-09-10T10:58:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 1 -- 172.21.15.7:0/3095913302 wait complete. 2024-09-10T10:58:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.456+0000 7f8811bf1740 20 client.16239 trim_cache size 0 max 16384 2024-09-10T10:58:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:58:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:58:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:58:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:58:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 -- 172.21.15.7:0/477671367 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f87f80a2e20 msgr2=0x7f87f80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f87f80a2e20 0x7f87f80a52e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8800009a60 tx=0x7f88000095c0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 -- 172.21.15.7:0/477671367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88081009d0 msgr2=0x7f8808198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88081009d0 0x7f8808198390 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00e9f0 tx=0x7f87fc00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 -- 172.21.15.7:0/477671367 shutdown_connections 2024-09-10T10:58:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88081069e0 0x7f880819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8808101370 0x7f88081988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f87f80a2e20 0x7f87f80a52e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 --2- 172.21.15.7:0/477671367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88081009d0 0x7f8808198390 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 -- 172.21.15.7:0/477671367 >> 172.21.15.7:0/477671367 conn(0x7f88080fc6b0 msgr2=0x7f88080fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 -- 172.21.15.7:0/477671367 shutdown_connections 2024-09-10T10:58:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:05.457+0000 7f880ed78640 1 -- 172.21.15.7:0/477671367 wait complete. 2024-09-10T10:58:05.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1048755 7.58 MB/sec execute 545 sec latency 2896.601 ms 2024-09-10T10:58:05.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:05 smithi007 ceph-mon[30873]: pgmap v384: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 1.0 MiB/s wr, 61 op/s 2024-09-10T10:58:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:05 smithi133 ceph-mon[35282]: pgmap v384: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 1.0 MiB/s wr, 61 op/s 2024-09-10T10:58:06.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:05 smithi103 ceph-mon[35160]: pgmap v384: 97 pgs: 97 active+clean; 272 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 1.0 MiB/s wr, 61 op/s 2024-09-10T10:58:06.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1049173 7.57 MB/sec execute 546 sec latency 3612.568 ms 2024-09-10T10:58:06.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:06 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:58:06.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:06 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:58:07.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:06 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:58:07.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1051625 7.58 MB/sec execute 547 sec latency 17.878 ms 2024-09-10T10:58:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:07 smithi133 ceph-mon[35282]: pgmap v385: 97 pgs: 97 active+clean; 272 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 661 KiB/s wr, 99 op/s 2024-09-10T10:58:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:07 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:07 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:07 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:07.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:07 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:07.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:07 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:07.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:07 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:07.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:07 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:58:07.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:07 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:58:07.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:07 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:07 smithi103 ceph-mon[35160]: pgmap v385: 97 pgs: 97 active+clean; 272 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 661 KiB/s wr, 99 op/s 2024-09-10T10:58:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:07 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:07 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:07 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:07 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:07 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:07 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:07 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:58:08.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:07 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:58:08.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:07 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:07 smithi007 ceph-mon[30873]: pgmap v385: 97 pgs: 97 active+clean; 272 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 661 KiB/s wr, 99 op/s 2024-09-10T10:58:08.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:07 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:07 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:07 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:07 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:07 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:07 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:07 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:58:08.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:07 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:58:08.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:07 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:58:08.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1052175 7.57 MB/sec execute 548 sec latency 437.464 ms 2024-09-10T10:58:09.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1052175 7.56 MB/sec execute 549 sec latency 1437.619 ms 2024-09-10T10:58:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:09 smithi103 ceph-mon[35160]: pgmap v386: 97 pgs: 97 active+clean; 272 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 661 KiB/s wr, 99 op/s 2024-09-10T10:58:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:09 smithi103 ceph-mon[35160]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T10:58:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:09 smithi007 ceph-mon[30873]: pgmap v386: 97 pgs: 97 active+clean; 272 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 661 KiB/s wr, 99 op/s 2024-09-10T10:58:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:09 smithi007 ceph-mon[30873]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T10:58:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:09 smithi133 ceph-mon[35282]: pgmap v386: 97 pgs: 97 active+clean; 272 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 661 KiB/s wr, 99 op/s 2024-09-10T10:58:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:09 smithi133 ceph-mon[35282]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T10:58:10.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1052175 7.54 MB/sec execute 550 sec latency 2437.739 ms 2024-09-10T10:58:11.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1052231 7.53 MB/sec execute 551 sec latency 3157.968 ms 2024-09-10T10:58:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:11 smithi103 ceph-mon[35160]: pgmap v388: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-10T10:58:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:11 smithi007 ceph-mon[30873]: pgmap v388: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-10T10:58:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:11 smithi133 ceph-mon[35282]: pgmap v388: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-10T10:58:12.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1052231 7.51 MB/sec execute 552 sec latency 1231.899 ms 2024-09-10T10:58:13.543 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1052231 7.50 MB/sec execute 553 sec latency 2231.987 ms 2024-09-10T10:58:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:13 smithi103 ceph-mon[35160]: pgmap v389: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-09-10T10:58:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:13 smithi007 ceph-mon[30873]: pgmap v389: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-09-10T10:58:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:13 smithi133 ceph-mon[35282]: pgmap v389: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-09-10T10:58:14.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1052231 7.49 MB/sec execute 554 sec latency 3232.088 ms 2024-09-10T10:58:15.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1052231 7.47 MB/sec execute 555 sec latency 4232.191 ms 2024-09-10T10:58:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:15 smithi103 ceph-mon[35160]: pgmap v390: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-09-10T10:58:15.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:15 smithi007 ceph-mon[30873]: pgmap v390: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-09-10T10:58:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:15 smithi133 ceph-mon[35282]: pgmap v390: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-09-10T10:58:16.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1052651 7.46 MB/sec execute 556 sec latency 4960.294 ms 2024-09-10T10:58:17.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1055036 7.48 MB/sec execute 557 sec latency 18.564 ms 2024-09-10T10:58:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:17 smithi103 ceph-mon[35160]: pgmap v391: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 87 op/s 2024-09-10T10:58:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:17 smithi007 ceph-mon[30873]: pgmap v391: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 87 op/s 2024-09-10T10:58:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:17 smithi133 ceph-mon[35282]: pgmap v391: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 87 op/s 2024-09-10T10:58:18.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1055778 7.47 MB/sec execute 558 sec latency 271.544 ms 2024-09-10T10:58:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.547+0000 7f0e8a096640 1 -- 172.21.15.7:0/3771688383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8410c110 msgr2=0x7f0e8410e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.547+0000 7f0e8a096640 1 --2- 172.21.15.7:0/3771688383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8410c110 0x7f0e8410e590 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009ca0 tx=0x7f0e7c02a950 comp rx=0 tx=0).stop 2024-09-10T10:58:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.547+0000 7f0e8a096640 1 -- 172.21.15.7:0/3771688383 shutdown_connections 2024-09-10T10:58:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.547+0000 7f0e8a096640 1 --2- 172.21.15.7:0/3771688383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8410c110 0x7f0e8410e590 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.547+0000 7f0e8a096640 1 --2- 172.21.15.7:0/3771688383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e841035e0 0x7f0e8410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.547+0000 7f0e8a096640 1 --2- 172.21.15.7:0/3771688383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e84102cc0 0x7f0e841030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.547+0000 7f0e8a096640 1 -- 172.21.15.7:0/3771688383 >> 172.21.15.7:0/3771688383 conn(0x7f0e840fc6b0 msgr2=0x7f0e840fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.548+0000 7f0e8a096640 1 -- 172.21.15.7:0/3771688383 shutdown_connections 2024-09-10T10:58:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.548+0000 7f0e8a096640 1 -- 172.21.15.7:0/3771688383 wait complete. 2024-09-10T10:58:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.548+0000 7f0e8a096640 1 Processor -- start 2024-09-10T10:58:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.548+0000 7f0e8a096640 1 -- start start 2024-09-10T10:58:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.548+0000 7f0e8a096640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84102cc0 0x7f0e84100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.548+0000 7f0e8a096640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e841035e0 0x7f0e84100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.548+0000 7f0e8a096640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8410c110 0x7f0e84101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.548+0000 7f0e8a096640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84111ef0 con 0x7f0e84102cc0 2024-09-10T10:58:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.548+0000 7f0e8a096640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84111d70 con 0x7f0e841035e0 2024-09-10T10:58:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.548+0000 7f0e8a096640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84112070 con 0x7f0e8410c110 2024-09-10T10:58:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84102cc0 0x7f0e84100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84102cc0 0x7f0e84100910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56250/0 (socket says 172.21.15.7:56250) 2024-09-10T10:58:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e837fe640 1 -- 172.21.15.7:0/3841236258 learned_addr learned my addr 172.21.15.7:0/3841236258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e83fff640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8410c110 0x7f0e84101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e7affd640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e841035e0 0x7f0e84100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e837fe640 1 -- 172.21.15.7:0/3841236258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8410c110 msgr2=0x7f0e84101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e837fe640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8410c110 0x7f0e84101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e837fe640 1 -- 172.21.15.7:0/3841236258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e841035e0 msgr2=0x7f0e84100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e837fe640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e841035e0 0x7f0e84100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e837fe640 1 -- 172.21.15.7:0/3841236258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e841a6840 con 0x7f0e84102cc0 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e837fe640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84102cc0 0x7f0e84100910 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400e9f0 tx=0x7f0e7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e817fa640 1 -- 172.21.15.7:0/3841236258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400cd80 con 0x7f0e84102cc0 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e817fa640 1 -- 172.21.15.7:0/3841236258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e74005500 con 0x7f0e84102cc0 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e8a096640 1 -- 172.21.15.7:0/3841236258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e841a6b30 con 0x7f0e84102cc0 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e817fa640 1 -- 172.21.15.7:0/3841236258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74010690 con 0x7f0e84102cc0 2024-09-10T10:58:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.549+0000 7f0e8a096640 1 -- 172.21.15.7:0/3841236258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e841a7030 con 0x7f0e84102cc0 2024-09-10T10:58:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e817fa640 1 -- 172.21.15.7:0/3841236258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0e740046c0 con 0x7f0e84102cc0 2024-09-10T10:58:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e8ce2f740 1 Processor -- start 2024-09-10T10:58:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e8ce2f740 1 -- start start 2024-09-10T10:58:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a99ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 0x56293a9a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56293a9012f0 con 0x56293a99caf0 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56293a8a9270 con 0x56293a9a7cb0 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56293a8fd2b0 con 0x56293a99d410 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e83fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a99ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 0x56293a9a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e83fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a99ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56252/0 (socket says 172.21.15.7:56252) 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e83fff640 1 -- 172.21.15.7:0/1364637923 learned_addr learned my addr 172.21.15.7:0/1364637923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.552+0000 7f0e7affd640 1 --2- 172.21.15.7:0/1364637923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e817fa640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0e700a2da0 0x7f0e700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e7affd640 1 -- 172.21.15.7:0/1364637923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 msgr2=0x56293a9a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e7affd640 1 --2- 172.21.15.7:0/1364637923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 0x56293a9a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e7affd640 1 -- 172.21.15.7:0/1364637923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 msgr2=0x56293a99ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e7affd640 1 --2- 172.21.15.7:0/1364637923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a99ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e7affd640 1 -- 172.21.15.7:0/1364637923 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56293a8a8340 con 0x56293a9a7cb0 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e83fff640 1 --2- 172.21.15.7:0/1364637923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a99ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e83fff640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0e700a2da0 0x7f0e700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e817fa640 1 -- 172.21.15.7:0/3841236258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f0e740cf900 con 0x7f0e84102cc0 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e7affd640 1 --2- 172.21.15.7:0/1364637923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9aa0a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00bb30 tx=0x7f0e6c0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da4632ae843d6253 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e79ffb640 1 -- 172.21.15.7:0/1364637923 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c004030 con 0x56293a9a7cb0 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e79ffb640 1 -- 172.21.15.7:0/1364637923 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6c0041d0 con 0x56293a9a7cb0 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e79ffb640 1 -- 172.21.15.7:0/1364637923 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c011640 con 0x56293a9a7cb0 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.553+0000 7f0e83fff640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0e700a2da0 0x7f0e700a5260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c030040 tx=0x7f0e7c03c040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1364637923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 msgr2=0x56293a9aa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/1364637923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9aa0a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00bb30 tx=0x7f0e6c0103d0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1364637923 shutdown_connections 2024-09-10T10:58:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/1364637923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9aa0a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/1364637923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 0x56293a9a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/1364637923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a99ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1364637923 >> 172.21.15.7:0/1364637923 conn(0x56293a999060 msgr2=0x56293a999440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1364637923 shutdown_connections 2024-09-10T10:58:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1364637923 wait complete. 2024-09-10T10:58:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 -- start start 2024-09-10T10:58:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.554+0000 7f0e8ce2f740 1 Processor -- start 2024-09-10T10:58:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 0x56293a9a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56293a9aa880 con 0x56293a99caf0 2024-09-10T10:58:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56293a88a290 con 0x56293a9a7cb0 2024-09-10T10:58:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56293a9aaa00 con 0x56293a99d410 2024-09-10T10:58:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e8ce2f740 10 client.0 _ll_register_callbacks cb 0x56293a8bb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:58:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56260/0 (socket says 172.21.15.7:56260) 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e837fe640 1 -- 172.21.15.7:0/1816447288 learned_addr learned my addr 172.21.15.7:0/1816447288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e83fff640 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e7affd640 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 0x56293a9a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e837fe640 1 -- 172.21.15.7:0/1816447288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 msgr2=0x56293a9a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e837fe640 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 0x56293a9a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e837fe640 1 -- 172.21.15.7:0/1816447288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 msgr2=0x56293a9a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e837fe640 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e837fe640 1 -- 172.21.15.7:0/1816447288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56293a614210 con 0x56293a99caf0 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e7affd640 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 0x56293a9a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e837fe640 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a4b50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400c6a0 tx=0x7f0e74004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549cdbe9b9e1ad08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:58:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e62ffd640 1 -- 172.21.15.7:0/1816447288 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e740cd450 con 0x56293a99caf0 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e62ffd640 1 -- 172.21.15.7:0/1816447288 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f0e740a4770 con 0x56293a99caf0 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.555+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1816447288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6c0041d0 con 0x56293a99caf0 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e62ffd640 1 -- 172.21.15.7:0/1816447288 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0e7409c490 con 0x56293a99caf0 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e8ce2f740 10 client.16251 fetch_fsmap learned FSMap version 42 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1816447288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56293a6e0620 con 0x56293a99caf0 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e62ffd640 1 -- 172.21.15.7:0/1816447288 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e6c0041d0 con 0x56293a99caf0 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e62ffd640 1 -- 172.21.15.7:0/1816447288 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f0e7409ac50 con 0x56293a99caf0 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e8ce2f740 10 client.16251 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e8ce2f740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e8ce2f740 20 client.16251 populate_metadata read hostname 'smithi007' 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56293a9b7c20 0x56293a9ba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e8ce2f740 4 client.16251 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:58:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1816447288 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56293a90ea20 con 0x56293a9b7c20 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.556+0000 7f0e7affd640 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56293a9b7c20 0x56293a9ba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e62ffd640 10 client.16251 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e7affd640 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56293a9b7c20 0x56293a9ba010 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2cff146f895c7b8 server_cookie=c222a7b6d9da35e0 in_seq=0 out_seq=0 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e62ffd640 1 -- 172.21.15.7:0/1816447288 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56293a90ea20 con 0x56293a9b7c20 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e62ffd640 10 client.16251 handle_command_reply: tid=0 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e8ce2f740 1 client.16251 shutdown 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1816447288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 msgr2=0x56293a9a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a4b50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400c6a0 tx=0x7f0e74004b60 comp rx=0 tx=0).stop 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1816447288 shutdown_connections 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56293a9b7c20 0x56293a9ba010 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99d410 0x56293a9a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e62ffd640 0 client.16251 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/1816447288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a4b50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.557+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1816447288 >> 172.21.15.7:0/1816447288 conn(0x56293a999060 msgr2=0x56293a99edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.558+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1816447288 shutdown_connections 2024-09-10T10:58:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.558+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/1816447288 wait complete. 2024-09-10T10:58:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.558+0000 7f0e8ce2f740 20 client.16251 trim_cache size 0 max 16384 2024-09-10T10:58:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e8ce2f740 1 Processor -- start 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e8ce2f740 1 -- start start 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a99d410 0x56293a9bf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a9a7cb0 0x56293a9bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841b2330 con 0x56293a99caf0 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841afc00 con 0x56293a99d410 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841ada70 con 0x56293a9a7cb0 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56270/0 (socket says 172.21.15.7:56270) 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e837fe640 1 -- 172.21.15.7:0/3695947794 learned_addr learned my addr 172.21.15.7:0/3695947794 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e837fe640 1 -- 172.21.15.7:0/3695947794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a9a7cb0 msgr2=0x56293a9bfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.572+0000 7f0e837fe640 1 --2- 172.21.15.7:0/3695947794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a9a7cb0 0x56293a9bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e837fe640 1 -- 172.21.15.7:0/3695947794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a99d410 msgr2=0x56293a9bf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e837fe640 1 --2- 172.21.15.7:0/3695947794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a99d410 0x56293a9bf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e837fe640 1 -- 172.21.15.7:0/3695947794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56293a9942d0 con 0x56293a99caf0 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e837fe640 1 --2- 172.21.15.7:0/3695947794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a5400 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0e740a44d0 tx=0x7f0e740cd550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e1a0721c547a2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e797fa640 1 -- 172.21.15.7:0/3695947794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e740a4a00 con 0x56293a99caf0 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e797fa640 1 -- 172.21.15.7:0/3695947794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e740a0140 con 0x56293a99caf0 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e797fa640 1 -- 172.21.15.7:0/3695947794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e740a12d0 con 0x56293a99caf0 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/3695947794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 msgr2=0x56293a9a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/3695947794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a5400 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0e740a44d0 tx=0x7f0e740cd550 comp rx=0 tx=0).stop 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/3695947794 shutdown_connections 2024-09-10T10:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/3695947794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a9a7cb0 0x56293a9bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/3695947794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a99d410 0x56293a9bf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/3695947794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99caf0 0x56293a9a5400 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/3695947794 >> 172.21.15.7:0/3695947794 conn(0x56293a999060 msgr2=0x56293a9a6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.573+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/3695947794 shutdown_connections 2024-09-10T10:58:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/3695947794 wait complete. 2024-09-10T10:58:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e8ce2f740 1 -- start start 2024-09-10T10:58:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e8ce2f740 1 Processor -- start 2024-09-10T10:58:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99caf0 0x56293a9b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99d410 0x56293a9a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e8ce2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841ae0c0 con 0x56293a99d410 2024-09-10T10:58:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841af530 con 0x56293a9a7cb0 2024-09-10T10:58:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e8ce2f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841ae790 con 0x56293a99caf0 2024-09-10T10:58:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e8ce2f740 10 client.0 _ll_register_callbacks cb 0x7f0e7400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:58:18.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e7affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99d410 0x56293a9a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e7affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99d410 0x56293a9a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56272/0 (socket says 172.21.15.7:56272) 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e7affd640 1 -- 172.21.15.7:0/263963938 learned_addr learned my addr 172.21.15.7:0/263963938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e83fff640 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.574+0000 7f0e837fe640 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99caf0 0x56293a9b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e7affd640 1 -- 172.21.15.7:0/263963938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99caf0 msgr2=0x56293a9b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e7affd640 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99caf0 0x56293a9b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e7affd640 1 -- 172.21.15.7:0/263963938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 msgr2=0x56293a9a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e7affd640 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e7affd640 1 -- 172.21.15.7:0/263963938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56293a9a2540 con 0x56293a99d410 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e83fff640 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e7affd640 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99d410 0x56293a9a17d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00ba80 tx=0x7f0e6c0041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5569568597baf30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e62ffd640 1 -- 172.21.15.7:0/263963938 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c003ac0 con 0x56293a99d410 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e62ffd640 1 -- 172.21.15.7:0/263963938 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f0e6c010cc0 con 0x56293a99d410 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/263963938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e740a0140 con 0x56293a99d410 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e62ffd640 1 -- 172.21.15.7:0/263963938 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0e6c036050 con 0x56293a99d410 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e8ce2f740 10 client.16257 fetch_fsmap learned FSMap version 42 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/263963938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56293aa228f0 con 0x56293a99d410 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.575+0000 7f0e62ffd640 1 -- 172.21.15.7:0/263963938 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e740a0140 con 0x56293a99d410 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.576+0000 7f0e62ffd640 1 -- 172.21.15.7:0/263963938 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f0e6c02f060 con 0x56293a99d410 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.576+0000 7f0e8ce2f740 10 client.16257 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.576+0000 7f0e8ce2f740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.576+0000 7f0e8ce2f740 20 client.16257 populate_metadata read hostname 'smithi007' 2024-09-10T10:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.576+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56293aa22f60 0x56293aa25380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.576+0000 7f0e8ce2f740 4 client.16257 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.576+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/263963938 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56293a908d70 con 0x56293aa22f60 2024-09-10T10:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.576+0000 7f0e837fe640 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56293aa22f60 0x56293aa25380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.576+0000 7f0e837fe640 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56293aa22f60 0x56293aa25380 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65e35695c9073878 server_cookie=a1cb396bc3234eb9 in_seq=0 out_seq=0 2024-09-10T10:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.576+0000 7f0e62ffd640 10 client.16257 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e62ffd640 1 -- 172.21.15.7:0/263963938 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56293a908d70 con 0x56293aa22f60 2024-09-10T10:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e62ffd640 10 client.16257 handle_command_reply: tid=0 2024-09-10T10:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e8ce2f740 1 client.16257 shutdown 2024-09-10T10:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/263963938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99d410 msgr2=0x56293a9a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99d410 0x56293a9a17d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00ba80 tx=0x7f0e6c0041b0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/263963938 shutdown_connections 2024-09-10T10:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x56293aa22f60 0x56293aa25380 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e62ffd640 0 client.16257 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56293a9a7cb0 0x56293a9a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56293a99d410 0x56293a9a17d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e8ce2f740 1 --2- 172.21.15.7:0/263963938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56293a99caf0 0x56293a9b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.627+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/263963938 >> 172.21.15.7:0/263963938 conn(0x56293a999060 msgr2=0x56293a9a6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/263963938 shutdown_connections 2024-09-10T10:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8ce2f740 1 -- 172.21.15.7:0/263963938 wait complete. 2024-09-10T10:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8ce2f740 20 client.16257 trim_cache size 0 max 16384 2024-09-10T10:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:58:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:58:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:58:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 -- 172.21.15.7:0/3841236258 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0e700a2da0 msgr2=0x7f0e700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0e700a2da0 0x7f0e700a5260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c030040 tx=0x7f0e7c03c040 comp rx=0 tx=0).stop 2024-09-10T10:58:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 -- 172.21.15.7:0/3841236258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84102cc0 msgr2=0x7f0e84100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84102cc0 0x7f0e84100910 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400e9f0 tx=0x7f0e7400edf0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 -- 172.21.15.7:0/3841236258 shutdown_connections 2024-09-10T10:58:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8410c110 0x7f0e84101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e841035e0 0x7f0e84100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0e700a2da0 0x7f0e700a5260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 --2- 172.21.15.7:0/3841236258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e84102cc0 0x7f0e84100910 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 -- 172.21.15.7:0/3841236258 >> 172.21.15.7:0/3841236258 conn(0x7f0e840fc6b0 msgr2=0x7f0e840fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.628+0000 7f0e8a096640 1 -- 172.21.15.7:0/3841236258 shutdown_connections 2024-09-10T10:58:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:18.629+0000 7f0e8a096640 1 -- 172.21.15.7:0/3841236258 wait complete. 2024-09-10T10:58:19.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1055778 7.45 MB/sec execute 559 sec latency 1271.630 ms 2024-09-10T10:58:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:19 smithi103 ceph-mon[35160]: pgmap v392: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 87 op/s 2024-09-10T10:58:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:19 smithi007 ceph-mon[30873]: pgmap v392: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 87 op/s 2024-09-10T10:58:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:19 smithi133 ceph-mon[35282]: pgmap v392: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.1 MiB/s wr, 87 op/s 2024-09-10T10:58:20.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1055778 7.44 MB/sec execute 560 sec latency 2271.764 ms 2024-09-10T10:58:21.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1056210 7.43 MB/sec execute 561 sec latency 2987.803 ms 2024-09-10T10:58:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:21 smithi103 ceph-mon[35160]: pgmap v393: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-09-10T10:58:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:21 smithi007 ceph-mon[30873]: pgmap v393: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-09-10T10:58:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:21 smithi133 ceph-mon[35282]: pgmap v393: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-09-10T10:58:22.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1058324 7.44 MB/sec execute 562 sec latency 74.158 ms 2024-09-10T10:58:23.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1059321 7.44 MB/sec execute 563 sec latency 189.880 ms 2024-09-10T10:58:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:23 smithi103 ceph-mon[35160]: pgmap v394: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 727 KiB/s wr, 73 op/s 2024-09-10T10:58:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:23 smithi007 ceph-mon[30873]: pgmap v394: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 727 KiB/s wr, 73 op/s 2024-09-10T10:58:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:23 smithi133 ceph-mon[35282]: pgmap v394: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 727 KiB/s wr, 73 op/s 2024-09-10T10:58:24.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1059321 7.42 MB/sec execute 564 sec latency 1189.944 ms 2024-09-10T10:58:25.544 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1059321 7.41 MB/sec execute 565 sec latency 2190.069 ms 2024-09-10T10:58:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:25 smithi103 ceph-mon[35160]: pgmap v395: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 727 KiB/s wr, 73 op/s 2024-09-10T10:58:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:25 smithi007 ceph-mon[30873]: pgmap v395: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 727 KiB/s wr, 73 op/s 2024-09-10T10:58:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:25 smithi133 ceph-mon[35282]: pgmap v395: 97 pgs: 97 active+clean; 276 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 727 KiB/s wr, 73 op/s 2024-09-10T10:58:26.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1059389 7.40 MB/sec execute 566 sec latency 2907.752 ms 2024-09-10T10:58:27.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1059389 7.38 MB/sec execute 567 sec latency 1232.366 ms 2024-09-10T10:58:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:27 smithi103 ceph-mon[35160]: pgmap v396: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-09-10T10:58:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:27 smithi007 ceph-mon[30873]: pgmap v396: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-09-10T10:58:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:27 smithi133 ceph-mon[35282]: pgmap v396: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-09-10T10:58:28.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1059389 7.37 MB/sec execute 568 sec latency 2232.472 ms 2024-09-10T10:58:29.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1059389 7.36 MB/sec execute 569 sec latency 3232.533 ms 2024-09-10T10:58:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:29 smithi103 ceph-mon[35160]: pgmap v397: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1019 KiB/s wr, 103 op/s 2024-09-10T10:58:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:29 smithi007 ceph-mon[30873]: pgmap v397: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1019 KiB/s wr, 103 op/s 2024-09-10T10:58:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:29 smithi133 ceph-mon[35282]: pgmap v397: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1019 KiB/s wr, 103 op/s 2024-09-10T10:58:30.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1059389 7.34 MB/sec execute 570 sec latency 4232.655 ms 2024-09-10T10:58:31.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1059857 7.33 MB/sec execute 571 sec latency 4949.569 ms 2024-09-10T10:58:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.720+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2882063557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79c41069c0 msgr2=0x7f79c4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.720+0000 7f79cc7d6640 1 --2- 172.21.15.7:0/2882063557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79c41069c0 0x7f79c4106da0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009930 tx=0x7f79bc02a420 comp rx=0 tx=0).stop 2024-09-10T10:58:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.720+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2882063557 shutdown_connections 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.720+0000 7f79cc7d6640 1 --2- 172.21.15.7:0/2882063557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79c4101350 0x7f79c410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.720+0000 7f79cc7d6640 1 --2- 172.21.15.7:0/2882063557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c41009b0 0x7f79c4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.720+0000 7f79cc7d6640 1 --2- 172.21.15.7:0/2882063557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79c41069c0 0x7f79c4106da0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.720+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2882063557 >> 172.21.15.7:0/2882063557 conn(0x7f79c4077990 msgr2=0x7f79c40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.720+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2882063557 shutdown_connections 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.720+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2882063557 wait complete. 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79cc7d6640 1 Processor -- start 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79cc7d6640 1 -- start start 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79cc7d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79c41009b0 0x7f79c40fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79cc7d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c4101350 0x7f79c40fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79cc7d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79c41069c0 0x7f79c40fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79cc7d6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4110e50 con 0x7f79c41009b0 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79cc7d6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4110cd0 con 0x7f79c41069c0 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79cc7d6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4110fd0 con 0x7f79c4101350 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79ca572640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79c41009b0 0x7f79c40fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79ca572640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79c41009b0 0x7f79c40fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43268/0 (socket says 172.21.15.7:43268) 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79ca572640 1 -- 172.21.15.7:0/2185054499 learned_addr learned my addr 172.21.15.7:0/2185054499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79cad73640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79c41069c0 0x7f79c40fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79c9d71640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c4101350 0x7f79c40fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79ca572640 1 -- 172.21.15.7:0/2185054499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c4101350 msgr2=0x7f79c40fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79ca572640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c4101350 0x7f79c40fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79ca572640 1 -- 172.21.15.7:0/2185054499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79c41069c0 msgr2=0x7f79c40fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79ca572640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79c41069c0 0x7f79c40fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.721+0000 7f79ca572640 1 -- 172.21.15.7:0/2185054499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c41a6820 con 0x7f79c41009b0 2024-09-10T10:58:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.722+0000 7f79ca572640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79c41009b0 0x7f79c40fcde0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f79bc02a940 tx=0x7f79bc003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.722+0000 7f79c37fe640 1 -- 172.21.15.7:0/2185054499 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc03f070 con 0x7f79c41009b0 2024-09-10T10:58:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.722+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2185054499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c41a6ab0 con 0x7f79c41009b0 2024-09-10T10:58:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.722+0000 7f79c37fe640 1 -- 172.21.15.7:0/2185054499 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79bc031c90 con 0x7f79c41009b0 2024-09-10T10:58:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.722+0000 7f79c37fe640 1 -- 172.21.15.7:0/2185054499 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc035030 con 0x7f79c41009b0 2024-09-10T10:58:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.722+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2185054499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c41a6ff0 con 0x7f79c41009b0 2024-09-10T10:58:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.725+0000 7f79cf650740 1 Processor -- start 2024-09-10T10:58:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.725+0000 7f79c37fe640 1 -- 172.21.15.7:0/2185054499 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f79bc042070 con 0x7f79c41009b0 2024-09-10T10:58:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.725+0000 7f79cf650740 1 -- start start 2024-09-10T10:58:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.725+0000 7f79cf650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 0x5629841dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.725+0000 7f79cf650740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 0x5629841e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.725+0000 7f79cf650740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x5629841eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.725+0000 7f79cf650740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5629841422f0 con 0x5629841e8cb0 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.725+0000 7f79cf650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5629840ea270 con 0x5629841ddaf0 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.725+0000 7f79cf650740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56298413e2b0 con 0x5629841de410 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79ca572640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 0x5629841e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79cad73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 0x5629841dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79c9d71640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x5629841eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79cad73640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 0x5629841dded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:46486/0 (socket says 172.21.15.7:46486) 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79ca572640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 0x5629841e8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:57174/0 (socket says 172.21.15.7:57174) 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79cad73640 1 -- 172.21.15.7:0/3474545196 learned_addr learned my addr 172.21.15.7:0/3474545196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79c9d71640 1 --2- 172.21.15.7:0/3474545196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x5629841eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43276/0 (socket says 172.21.15.7:43276) 2024-09-10T10:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79cad73640 1 -- 172.21.15.7:0/3474545196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 msgr2=0x5629841e8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79cad73640 1 --2- 172.21.15.7:0/3474545196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 0x5629841e8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79cad73640 1 -- 172.21.15.7:0/3474545196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 msgr2=0x5629841eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79cad73640 1 --2- 172.21.15.7:0/3474545196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x5629841eb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79c37fe640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f79ac0a2da0 0x7f79ac0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79cad73640 1 -- 172.21.15.7:0/3474545196 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629840e9340 con 0x5629841ddaf0 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79c9d71640 1 --2- 172.21.15.7:0/3474545196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x5629841eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79ca572640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f79ac0a2da0 0x7f79ac0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79c37fe640 1 -- 172.21.15.7:0/2185054499 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f79bc0efb30 con 0x7f79c41009b0 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79cad73640 1 --2- 172.21.15.7:0/3474545196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 0x5629841dded0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009f50 tx=0x7f79b40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f419fce1dc04580 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79c0ff9640 1 -- 172.21.15.7:0/3474545196 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4004d20 con 0x5629841ddaf0 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79c0ff9640 1 -- 172.21.15.7:0/3474545196 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b4004050 con 0x5629841ddaf0 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.726+0000 7f79c0ff9640 1 -- 172.21.15.7:0/3474545196 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4005280 con 0x5629841ddaf0 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.727+0000 7f79cf650740 1 -- 172.21.15.7:0/3474545196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 msgr2=0x5629841dded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.727+0000 7f79ca572640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f79ac0a2da0 0x7f79ac0a5260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0ed300 tx=0x7f79bc0f3680 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.727+0000 7f79cf650740 1 --2- 172.21.15.7:0/3474545196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 0x5629841dded0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009f50 tx=0x7f79b40118d0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.727+0000 7f79cf650740 1 -- 172.21.15.7:0/3474545196 shutdown_connections 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.727+0000 7f79cf650740 1 --2- 172.21.15.7:0/3474545196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x5629841eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.727+0000 7f79cf650740 1 --2- 172.21.15.7:0/3474545196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 0x5629841e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.727+0000 7f79cf650740 1 --2- 172.21.15.7:0/3474545196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 0x5629841dded0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.727+0000 7f79cf650740 1 -- 172.21.15.7:0/3474545196 >> 172.21.15.7:0/3474545196 conn(0x5629841da060 msgr2=0x5629841da440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.727+0000 7f79cf650740 1 -- 172.21.15.7:0/3474545196 shutdown_connections 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.727+0000 7f79cf650740 1 -- 172.21.15.7:0/3474545196 wait complete. 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cf650740 1 -- start start 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cf650740 1 Processor -- start 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cf650740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cf650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 0x5629841e60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cf650740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 0x5629841e6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cf650740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5629841eb880 con 0x5629841ddaf0 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cf650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5629840cb290 con 0x5629841de410 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cf650740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5629841eba00 con 0x5629841e8cb0 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cf650740 10 client.0 _ll_register_callbacks cb 0x5629840fc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cad73640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cad73640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43286/0 (socket says 172.21.15.7:43286) 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cad73640 1 -- 172.21.15.7:0/33388264 learned_addr learned my addr 172.21.15.7:0/33388264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79c9d71640 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 0x5629841e60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.728+0000 7f79cad73640 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 0x5629841e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79c9d71640 1 -- 172.21.15.7:0/33388264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 msgr2=0x5629841e6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79c9d71640 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 0x5629841e6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79c9d71640 1 -- 172.21.15.7:0/33388264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 msgr2=0x5629841e5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79c9d71640 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79c9d71640 1 -- 172.21.15.7:0/33388264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562983e55210 con 0x5629841de410 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79cad73640 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 0x5629841e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79c9d71640 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 0x5629841e60f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f79c40fe870 tx=0x7f79b8012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a784386916291a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79a9ffb640 1 -- 172.21.15.7:0/33388264 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8011c70 con 0x5629841de410 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79a9ffb640 1 -- 172.21.15.7:0/33388264 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f79b8011e10 con 0x5629841de410 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79cf650740 1 -- 172.21.15.7:0/33388264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b4004050 con 0x5629841de410 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79a9ffb640 1 -- 172.21.15.7:0/33388264 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f79b8004030 con 0x5629841de410 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79cf650740 10 client.25883 fetch_fsmap learned FSMap version 42 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79cf650740 1 -- 172.21.15.7:0/33388264 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562983f21620 con 0x5629841de410 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.729+0000 7f79a9ffb640 1 -- 172.21.15.7:0/33388264 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b4004050 con 0x5629841de410 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.730+0000 7f79a9ffb640 1 -- 172.21.15.7:0/33388264 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f79b8023070 con 0x5629841de410 2024-09-10T10:58:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.730+0000 7f79cf650740 10 client.25883 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.730+0000 7f79cf650740 10 client.25883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.730+0000 7f79cf650740 20 client.25883 populate_metadata read hostname 'smithi007' 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.730+0000 7f79cf650740 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5629841f8cb0 0x5629841fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.730+0000 7f79cf650740 4 client.25883 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.730+0000 7f79cf650740 1 -- 172.21.15.7:0/33388264 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56298414fa20 con 0x5629841f8cb0 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.730+0000 7f79cad73640 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5629841f8cb0 0x5629841fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.730+0000 7f79cad73640 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5629841f8cb0 0x5629841fb0a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e7ef96a6a59a0c server_cookie=c8cfaf9952e76bb2 in_seq=0 out_seq=0 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.730+0000 7f79a9ffb640 10 client.25883 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79a9ffb640 1 -- 172.21.15.7:0/33388264 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56298414fa20 con 0x5629841f8cb0 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79a9ffb640 10 client.25883 handle_command_reply: tid=0 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 client.25883 shutdown 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 -- 172.21.15.7:0/33388264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 msgr2=0x5629841e60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 0x5629841e60f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f79c40fe870 tx=0x7f79b8012b60 comp rx=0 tx=0).stop 2024-09-10T10:58:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 -- 172.21.15.7:0/33388264 shutdown_connections 2024-09-10T10:58:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5629841f8cb0 0x5629841fb0a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 0x5629841e6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 0x5629841e60f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 --2- 172.21.15.7:0/33388264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79a9ffb640 0 client.25883 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 -- 172.21.15.7:0/33388264 >> 172.21.15.7:0/33388264 conn(0x5629841da060 msgr2=0x5629841dfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 -- 172.21.15.7:0/33388264 shutdown_connections 2024-09-10T10:58:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 1 -- 172.21.15.7:0/33388264 wait complete. 2024-09-10T10:58:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.731+0000 7f79cf650740 20 client.25883 trim_cache size 0 max 16384 2024-09-10T10:58:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cf650740 1 Processor -- start 2024-09-10T10:58:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cf650740 1 -- start start 2024-09-10T10:58:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cf650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 0x5629841e0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cf650740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 0x5629841e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cf650740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x562984200990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cf650740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41b22f0 con 0x5629841e8cb0 2024-09-10T10:58:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cf650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41afbc0 con 0x5629841ddaf0 2024-09-10T10:58:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cf650740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41ada30 con 0x5629841de410 2024-09-10T10:58:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cad73640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x562984200990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79c9d71640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 0x5629841e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cad73640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x562984200990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43290/0 (socket says 172.21.15.7:43290) 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.746+0000 7f79cad73640 1 -- 172.21.15.7:0/3100152272 learned_addr learned my addr 172.21.15.7:0/3100152272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79cad73640 1 --2- 172.21.15.7:0/3100152272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 0x5629841e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79c9d71640 1 -- 172.21.15.7:0/3100152272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 msgr2=0x5629841e0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79c9d71640 1 --2- 172.21.15.7:0/3100152272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 0x5629841e0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79c9d71640 1 -- 172.21.15.7:0/3100152272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 msgr2=0x562984200990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79c9d71640 1 --2- 172.21.15.7:0/3100152272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x562984200990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79c9d71640 1 -- 172.21.15.7:0/3100152272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629841df000 con 0x5629841de410 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79cad73640 1 --2- 172.21.15.7:0/3100152272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x562984200990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79c9d71640 1 --2- 172.21.15.7:0/3100152272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 0x5629841e6460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79b8000c00 tx=0x7f79b8011890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8fccc9235c59953 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79abfff640 1 -- 172.21.15.7:0/3100152272 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8029040 con 0x5629841de410 2024-09-10T10:58:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79abfff640 1 -- 172.21.15.7:0/3100152272 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b801c710 con 0x5629841de410 2024-09-10T10:58:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.747+0000 7f79abfff640 1 -- 172.21.15.7:0/3100152272 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b801cce0 con 0x5629841de410 2024-09-10T10:58:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 -- 172.21.15.7:0/3100152272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 msgr2=0x5629841e6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 --2- 172.21.15.7:0/3100152272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 0x5629841e6460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f79b8000c00 tx=0x7f79b8011890 comp rx=0 tx=0).stop 2024-09-10T10:58:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 -- 172.21.15.7:0/3100152272 shutdown_connections 2024-09-10T10:58:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 --2- 172.21.15.7:0/3100152272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841e8cb0 0x562984200990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 --2- 172.21.15.7:0/3100152272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841de410 0x5629841e6460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 --2- 172.21.15.7:0/3100152272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841ddaf0 0x5629841e0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 -- 172.21.15.7:0/3100152272 >> 172.21.15.7:0/3100152272 conn(0x5629841da060 msgr2=0x5629841e7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 -- 172.21.15.7:0/3100152272 shutdown_connections 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 -- 172.21.15.7:0/3100152272 wait complete. 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 -- start start 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.748+0000 7f79cf650740 1 Processor -- start 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cf650740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cf650740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 0x5629841e3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cf650740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 0x5629841e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cf650740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41ae080 con 0x5629841ddaf0 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cf650740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41af4f0 con 0x5629841de410 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cf650740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41ae750 con 0x5629841e8cb0 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cf650740 10 client.0 _ll_register_callbacks cb 0x7f79b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cad73640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cad73640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43298/0 (socket says 172.21.15.7:43298) 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cad73640 1 -- 172.21.15.7:0/3875332570 learned_addr learned my addr 172.21.15.7:0/3875332570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79c9d71640 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 0x5629841e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cad73640 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 0x5629841e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cad73640 1 -- 172.21.15.7:0/3875332570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 msgr2=0x5629841e3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cad73640 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 0x5629841e3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cad73640 1 -- 172.21.15.7:0/3875332570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 msgr2=0x5629841e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cad73640 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 0x5629841e3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cad73640 1 -- 172.21.15.7:0/3875332570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562984262960 con 0x5629841ddaf0 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79c9d71640 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 0x5629841e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cad73640 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e2af0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009270 tx=0x7f79b40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1303f66e8695c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:58:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79a9ffb640 1 -- 172.21.15.7:0/3875332570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4004340 con 0x5629841ddaf0 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79a9ffb640 1 -- 172.21.15.7:0/3875332570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f79b40044e0 con 0x5629841ddaf0 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.749+0000 7f79cf650740 1 -- 172.21.15.7:0/3875332570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b801c710 con 0x5629841ddaf0 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79a9ffb640 1 -- 172.21.15.7:0/3875332570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f79b4003a30 con 0x5629841ddaf0 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79cf650740 10 client.16275 fetch_fsmap learned FSMap version 42 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79cf650740 1 -- 172.21.15.7:0/3875332570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562984263b70 con 0x5629841ddaf0 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79a9ffb640 1 -- 172.21.15.7:0/3875332570 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b801c710 con 0x5629841ddaf0 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79a9ffb640 1 -- 172.21.15.7:0/3875332570 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f79b4035030 con 0x5629841ddaf0 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79cf650740 10 client.16275 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79cf650740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79cf650740 20 client.16275 populate_metadata read hostname 'smithi007' 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79cf650740 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5629842640d0 0x5629842664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79cf650740 4 client.16275 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79cf650740 1 -- 172.21.15.7:0/3875332570 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562984149d70 con 0x5629842640d0 2024-09-10T10:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.750+0000 7f79c9d71640 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5629842640d0 0x5629842664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.751+0000 7f79a9ffb640 10 client.16275 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.751+0000 7f79c9d71640 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5629842640d0 0x5629842664f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8f797d40a6c4f7 server_cookie=a3b7f0b8f35cf59a in_seq=0 out_seq=0 2024-09-10T10:58:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.787+0000 7f79a9ffb640 1 -- 172.21.15.7:0/3875332570 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562984149d70 con 0x5629842640d0 2024-09-10T10:58:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.787+0000 7f79a9ffb640 10 client.16275 handle_command_reply: tid=0 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.787+0000 7f79cf650740 1 client.16275 shutdown 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.787+0000 7f79cf650740 1 -- 172.21.15.7:0/3875332570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 msgr2=0x5629841e2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.787+0000 7f79cf650740 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e2af0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009270 tx=0x7f79b40118d0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.787+0000 7f79cf650740 1 -- 172.21.15.7:0/3875332570 shutdown_connections 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.787+0000 7f79cf650740 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5629842640d0 0x5629842664f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.787+0000 7f79cf650740 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5629841e8cb0 0x5629841e3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79a9ffb640 0 client.16275 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cf650740 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5629841de410 0x5629841e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cf650740 1 --2- 172.21.15.7:0/3875332570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629841ddaf0 0x5629841e2af0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cf650740 1 -- 172.21.15.7:0/3875332570 >> 172.21.15.7:0/3875332570 conn(0x5629841da060 msgr2=0x5629841d89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cf650740 1 -- 172.21.15.7:0/3875332570 shutdown_connections 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cf650740 1 -- 172.21.15.7:0/3875332570 wait complete. 2024-09-10T10:58:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cf650740 20 client.16275 trim_cache size 0 max 16384 2024-09-10T10:58:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:58:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:58:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:58:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:58:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2185054499 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f79ac0a2da0 msgr2=0x7f79ac0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cc7d6640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f79ac0a2da0 0x7f79ac0a5260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0ed300 tx=0x7f79bc0f3680 comp rx=0 tx=0).stop 2024-09-10T10:58:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2185054499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79c41009b0 msgr2=0x7f79c40fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cc7d6640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79c41009b0 0x7f79c40fcde0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f79bc02a940 tx=0x7f79bc003b20 comp rx=0 tx=0).stop 2024-09-10T10:58:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2185054499 shutdown_connections 2024-09-10T10:58:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cc7d6640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79c41069c0 0x7f79c40fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cc7d6640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79c4101350 0x7f79c40fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cc7d6640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f79ac0a2da0 0x7f79ac0a5260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cc7d6640 1 --2- 172.21.15.7:0/2185054499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79c41009b0 0x7f79c40fcde0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.788+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2185054499 >> 172.21.15.7:0/2185054499 conn(0x7f79c4077990 msgr2=0x7f79c40699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.789+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2185054499 shutdown_connections 2024-09-10T10:58:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:31.789+0000 7f79cc7d6640 1 -- 172.21.15.7:0/2185054499 wait complete. 2024-09-10T10:58:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:31 smithi103 ceph-mon[35160]: pgmap v398: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-09-10T10:58:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:31 smithi007 ceph-mon[30873]: pgmap v398: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-09-10T10:58:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:31 smithi133 ceph-mon[35282]: pgmap v398: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-09-10T10:58:32.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1062370 7.35 MB/sec execute 572 sec latency 27.513 ms 2024-09-10T10:58:33.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1063760 7.35 MB/sec execute 573 sec latency 15.079 ms 2024-09-10T10:58:33.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:33 smithi103 ceph-mon[35160]: pgmap v399: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 617 KiB/s wr, 56 op/s 2024-09-10T10:58:33.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:33 smithi007 ceph-mon[30873]: pgmap v399: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 617 KiB/s wr, 56 op/s 2024-09-10T10:58:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:33 smithi133 ceph-mon[35282]: pgmap v399: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 617 KiB/s wr, 56 op/s 2024-09-10T10:58:34.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1065969 7.36 MB/sec execute 574 sec latency 52.556 ms 2024-09-10T10:58:35.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1067392 7.36 MB/sec execute 575 sec latency 17.318 ms 2024-09-10T10:58:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:35 smithi103 ceph-mon[35160]: pgmap v400: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 617 KiB/s wr, 56 op/s 2024-09-10T10:58:35.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:35 smithi007 ceph-mon[30873]: pgmap v400: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 617 KiB/s wr, 56 op/s 2024-09-10T10:58:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:35 smithi133 ceph-mon[35282]: pgmap v400: 97 pgs: 97 active+clean; 279 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 617 KiB/s wr, 56 op/s 2024-09-10T10:58:36.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1069519 7.37 MB/sec execute 576 sec latency 19.985 ms 2024-09-10T10:58:37.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1070860 7.36 MB/sec execute 577 sec latency 18.816 ms 2024-09-10T10:58:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:37 smithi103 ceph-mon[35160]: pgmap v401: 97 pgs: 97 active+clean; 281 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-09-10T10:58:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:37 smithi007 ceph-mon[30873]: pgmap v401: 97 pgs: 97 active+clean; 281 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-09-10T10:58:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:37 smithi133 ceph-mon[35282]: pgmap v401: 97 pgs: 97 active+clean; 281 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-09-10T10:58:38.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1072995 7.37 MB/sec execute 578 sec latency 21.752 ms 2024-09-10T10:58:39.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1074274 7.37 MB/sec execute 579 sec latency 12.716 ms 2024-09-10T10:58:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:39 smithi103 ceph-mon[35160]: pgmap v402: 97 pgs: 97 active+clean; 281 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 879 KiB/s wr, 95 op/s 2024-09-10T10:58:39.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:39 smithi007 ceph-mon[30873]: pgmap v402: 97 pgs: 97 active+clean; 281 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 879 KiB/s wr, 95 op/s 2024-09-10T10:58:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:39 smithi133 ceph-mon[35282]: pgmap v402: 97 pgs: 97 active+clean; 281 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 879 KiB/s wr, 95 op/s 2024-09-10T10:58:40.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1076542 7.38 MB/sec execute 580 sec latency 27.635 ms 2024-09-10T10:58:41.545 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1077833 7.38 MB/sec execute 581 sec latency 99.225 ms 2024-09-10T10:58:41.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:41 smithi103 ceph-mon[35160]: pgmap v403: 97 pgs: 97 active+clean; 289 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-09-10T10:58:41.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:41 smithi007 ceph-mon[30873]: pgmap v403: 97 pgs: 97 active+clean; 289 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-09-10T10:58:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:41 smithi133 ceph-mon[35282]: pgmap v403: 97 pgs: 97 active+clean; 289 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-09-10T10:58:42.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1080159 7.39 MB/sec execute 582 sec latency 19.030 ms 2024-09-10T10:58:43.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1081647 7.39 MB/sec execute 583 sec latency 16.892 ms 2024-09-10T10:58:43.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:43 smithi103 ceph-mon[35160]: pgmap v404: 97 pgs: 97 active+clean; 289 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-09-10T10:58:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:43 smithi007 ceph-mon[30873]: pgmap v404: 97 pgs: 97 active+clean; 289 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-09-10T10:58:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:43 smithi133 ceph-mon[35282]: pgmap v404: 97 pgs: 97 active+clean; 289 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-09-10T10:58:44.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1083719 7.40 MB/sec execute 584 sec latency 113.169 ms 2024-09-10T10:58:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.880+0000 7f8e63952640 1 -- 172.21.15.7:0/641884889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5c1069c0 msgr2=0x7f8e5c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.880+0000 7f8e63952640 1 --2- 172.21.15.7:0/641884889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5c1069c0 0x7f8e5c106da0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009930 tx=0x7f8e5802a420 comp rx=0 tx=0).stop 2024-09-10T10:58:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.880+0000 7f8e63952640 1 -- 172.21.15.7:0/641884889 shutdown_connections 2024-09-10T10:58:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.880+0000 7f8e63952640 1 --2- 172.21.15.7:0/641884889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c101350 0x7f8e5c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.880+0000 7f8e63952640 1 --2- 172.21.15.7:0/641884889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e5c1009b0 0x7f8e5c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.880+0000 7f8e63952640 1 --2- 172.21.15.7:0/641884889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5c1069c0 0x7f8e5c106da0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.880+0000 7f8e63952640 1 -- 172.21.15.7:0/641884889 >> 172.21.15.7:0/641884889 conn(0x7f8e5c077990 msgr2=0x7f8e5c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.880+0000 7f8e63952640 1 -- 172.21.15.7:0/641884889 shutdown_connections 2024-09-10T10:58:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.880+0000 7f8e63952640 1 -- 172.21.15.7:0/641884889 wait complete. 2024-09-10T10:58:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e63952640 1 Processor -- start 2024-09-10T10:58:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e63952640 1 -- start start 2024-09-10T10:58:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e63952640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e5c1009b0 0x7f8e5c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e63952640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c101350 0x7f8e5c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e63952640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5c1069c0 0x7f8e5c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e63952640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c110e00 con 0x7f8e5c1069c0 2024-09-10T10:58:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e63952640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c110c80 con 0x7f8e5c1009b0 2024-09-10T10:58:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e63952640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c110f80 con 0x7f8e5c101350 2024-09-10T10:58:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e61eef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5c1069c0 0x7f8e5c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e61eef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5c1069c0 0x7f8e5c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35480/0 (socket says 172.21.15.7:35480) 2024-09-10T10:58:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e61eef640 1 -- 172.21.15.7:0/3407930993 learned_addr learned my addr 172.21.15.7:0/3407930993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:44.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e616ee640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e5c1009b0 0x7f8e5c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.881+0000 7f8e60eed640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c101350 0x7f8e5c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e61eef640 1 -- 172.21.15.7:0/3407930993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c101350 msgr2=0x7f8e5c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e61eef640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c101350 0x7f8e5c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e61eef640 1 -- 172.21.15.7:0/3407930993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e5c1009b0 msgr2=0x7f8e5c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e61eef640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e5c1009b0 0x7f8e5c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e61eef640 1 -- 172.21.15.7:0/3407930993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5c19d370 con 0x7f8e5c1069c0 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e61eef640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5c1069c0 0x7f8e5c19cc40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400eb10 tx=0x7f8e5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e527fc640 1 -- 172.21.15.7:0/3407930993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54004b20 con 0x7f8e5c1069c0 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e527fc640 1 -- 172.21.15.7:0/3407930993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e54004cc0 con 0x7f8e5c1069c0 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e527fc640 1 -- 172.21.15.7:0/3407930993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54010760 con 0x7f8e5c1069c0 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e63952640 1 -- 172.21.15.7:0/3407930993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5c19d660 con 0x7f8e5c1069c0 2024-09-10T10:58:44.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.882+0000 7f8e63952640 1 -- 172.21.15.7:0/3407930993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5c1a4ef0 con 0x7f8e5c1069c0 2024-09-10T10:58:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.884+0000 7f8e527fc640 1 -- 172.21.15.7:0/3407930993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8e54005020 con 0x7f8e5c1069c0 2024-09-10T10:58:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e667ca740 1 Processor -- start 2024-09-10T10:58:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e667ca740 1 -- start start 2024-09-10T10:58:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 0x55a7310ba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310bacb0 0x55a7310bd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e667ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a7310142f0 con 0x55a7310afaf0 2024-09-10T10:58:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e667ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a730fbc270 con 0x55a7310b0410 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e667ca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a7310102b0 con 0x55a7310bacb0 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e60eed640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e616ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 0x55a7310ba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e60eed640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35484/0 (socket says 172.21.15.7:35484) 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e60eed640 1 -- 172.21.15.7:0/3288154334 learned_addr learned my addr 172.21.15.7:0/3288154334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.885+0000 7f8e60eed640 1 --2- 172.21.15.7:0/3288154334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310bacb0 0x55a7310bd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e616ee640 1 -- 172.21.15.7:0/3288154334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310bacb0 msgr2=0x55a7310bd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e616ee640 1 --2- 172.21.15.7:0/3288154334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310bacb0 0x55a7310bd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e616ee640 1 -- 172.21.15.7:0/3288154334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 msgr2=0x55a7310afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e616ee640 1 --2- 172.21.15.7:0/3288154334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e616ee640 1 -- 172.21.15.7:0/3288154334 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a730fbb340 con 0x55a7310b0410 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e527fc640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8e480a2da0 0x7f8e480a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e60eed640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8e480a2da0 0x7f8e480a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e527fc640 1 -- 172.21.15.7:0/3407930993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f8e540cfab0 con 0x7f8e5c1069c0 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e616ee640 1 --2- 172.21.15.7:0/3288154334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 0x55a7310ba770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005940 tx=0x7f8e58005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5c3202be417378 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e477fe640 1 -- 172.21.15.7:0/3288154334 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58040070 con 0x55a7310b0410 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e477fe640 1 -- 172.21.15.7:0/3288154334 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e58031870 con 0x55a7310b0410 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.886+0000 7f8e477fe640 1 -- 172.21.15.7:0/3288154334 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58035040 con 0x55a7310b0410 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e667ca740 1 -- 172.21.15.7:0/3288154334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 msgr2=0x55a7310ba770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e60eed640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8e480a2da0 0x7f8e480a5260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55a7310ad3e0 tx=0x7f8e4c005f90 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e667ca740 1 --2- 172.21.15.7:0/3288154334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 0x55a7310ba770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005940 tx=0x7f8e58005bd0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e667ca740 1 -- 172.21.15.7:0/3288154334 shutdown_connections 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e667ca740 1 --2- 172.21.15.7:0/3288154334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310bacb0 0x55a7310bd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e667ca740 1 --2- 172.21.15.7:0/3288154334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 0x55a7310ba770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e667ca740 1 --2- 172.21.15.7:0/3288154334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.895 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e667ca740 1 -- 172.21.15.7:0/3288154334 >> 172.21.15.7:0/3288154334 conn(0x55a7310ac060 msgr2=0x55a7310ac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e667ca740 1 -- 172.21.15.7:0/3288154334 shutdown_connections 2024-09-10T10:58:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e667ca740 1 -- 172.21.15.7:0/3288154334 wait complete. 2024-09-10T10:58:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.887+0000 7f8e667ca740 1 -- start start 2024-09-10T10:58:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e667ca740 1 Processor -- start 2024-09-10T10:58:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 0x55a7310b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310b0410 0x55a7310b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e667ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a7310bd880 con 0x55a7310bacb0 2024-09-10T10:58:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e667ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a730f9d290 con 0x55a7310afaf0 2024-09-10T10:58:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e667ca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a7310bda00 con 0x55a7310b0410 2024-09-10T10:58:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e667ca740 10 client.0 _ll_register_callbacks cb 0x55a730fce140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:58:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e61eef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e616ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 0x55a7310b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e61eef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310b85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35486/0 (socket says 172.21.15.7:35486) 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e61eef640 1 -- 172.21.15.7:0/1411769403 learned_addr learned my addr 172.21.15.7:0/1411769403 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e60eed640 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310b0410 0x55a7310b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e61eef640 1 -- 172.21.15.7:0/1411769403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310b0410 msgr2=0x55a7310b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e61eef640 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310b0410 0x55a7310b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e61eef640 1 -- 172.21.15.7:0/1411769403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 msgr2=0x55a7310b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e61eef640 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 0x55a7310b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.888+0000 7f8e61eef640 1 -- 172.21.15.7:0/1411769403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a730d27210 con 0x55a7310bacb0 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e61eef640 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310b85e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8e540a46e0 tx=0x7f8e5409bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97fb1fd7700f73e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e44ff9640 1 -- 172.21.15.7:0/1411769403 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e540bade0 con 0x55a7310bacb0 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e44ff9640 1 -- 172.21.15.7:0/1411769403 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f8e54005310 con 0x55a7310bacb0 2024-09-10T10:58:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e667ca740 1 -- 172.21.15.7:0/1411769403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e58031870 con 0x55a7310bacb0 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e44ff9640 1 -- 172.21.15.7:0/1411769403 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8e5409bfa0 con 0x55a7310bacb0 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e667ca740 10 client.16287 fetch_fsmap learned FSMap version 42 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e667ca740 1 -- 172.21.15.7:0/1411769403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a730df3620 con 0x55a7310bacb0 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e44ff9640 1 -- 172.21.15.7:0/1411769403 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e58031870 con 0x55a7310bacb0 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e44ff9640 1 -- 172.21.15.7:0/1411769403 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f8e5409b3f0 con 0x55a7310bacb0 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e667ca740 10 client.16287 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e667ca740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e667ca740 20 client.16287 populate_metadata read hostname 'smithi007' 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e667ca740 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a7310cad10 0x55a7310cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e667ca740 4 client.16287 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.889+0000 7f8e667ca740 1 -- 172.21.15.7:0/1411769403 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a731021a20 con 0x55a7310cad10 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.890+0000 7f8e616ee640 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a7310cad10 0x55a7310cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.890+0000 7f8e616ee640 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a7310cad10 0x55a7310cd180 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=640589ecbf2f92b4 server_cookie=cf79eecbde74d047 in_seq=0 out_seq=0 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.890+0000 7f8e44ff9640 10 client.16287 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e44ff9640 1 -- 172.21.15.7:0/1411769403 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a731021a20 con 0x55a7310cad10 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e44ff9640 10 client.16287 handle_command_reply: tid=0 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 client.16287 shutdown 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 -- 172.21.15.7:0/1411769403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 msgr2=0x55a7310b85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310b85e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8e540a46e0 tx=0x7f8e5409bba0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 -- 172.21.15.7:0/1411769403 shutdown_connections 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a7310cad10 0x55a7310cd180 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310b85e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310b0410 0x55a7310b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e44ff9640 0 client.16287 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 --2- 172.21.15.7:0/1411769403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 0x55a7310b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 -- 172.21.15.7:0/1411769403 >> 172.21.15.7:0/1411769403 conn(0x55a7310ac060 msgr2=0x55a7310b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 -- 172.21.15.7:0/1411769403 shutdown_connections 2024-09-10T10:58:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 1 -- 172.21.15.7:0/1411769403 wait complete. 2024-09-10T10:58:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.891+0000 7f8e667ca740 20 client.16287 trim_cache size 0 max 16384 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.907+0000 7f8e667ca740 1 Processor -- start 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.907+0000 7f8e667ca740 1 -- start start 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.907+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 0x55a7310b8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.907+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310b0410 0x55a7310d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e667ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1b00c0 con 0x55a7310bacb0 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e667ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1ad990 con 0x55a7310afaf0 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e667ca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1ab800 con 0x55a7310b0410 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e61eef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e616ee640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 0x55a7310b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e61eef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310d2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35502/0 (socket says 172.21.15.7:35502) 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e60eed640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310b0410 0x55a7310d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e61eef640 1 -- 172.21.15.7:0/2330605552 learned_addr learned my addr 172.21.15.7:0/2330605552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e616ee640 1 -- 172.21.15.7:0/2330605552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310b0410 msgr2=0x55a7310d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e616ee640 1 --2- 172.21.15.7:0/2330605552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310b0410 0x55a7310d2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e616ee640 1 -- 172.21.15.7:0/2330605552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 msgr2=0x55a7310d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e616ee640 1 --2- 172.21.15.7:0/2330605552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310d2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e616ee640 1 -- 172.21.15.7:0/2330605552 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7310af920 con 0x55a7310afaf0 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e61eef640 1 --2- 172.21.15.7:0/2330605552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310d2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e616ee640 1 --2- 172.21.15.7:0/2330605552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 0x55a7310b8410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e5802a940 tx=0x7f8e580058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2009f485db01a340 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e46ffd640 1 -- 172.21.15.7:0/2330605552 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58040050 con 0x55a7310afaf0 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e46ffd640 1 -- 172.21.15.7:0/2330605552 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e58031870 con 0x55a7310afaf0 2024-09-10T10:58:44.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.908+0000 7f8e46ffd640 1 -- 172.21.15.7:0/2330605552 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58035040 con 0x55a7310afaf0 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.909+0000 7f8e667ca740 1 -- 172.21.15.7:0/2330605552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 msgr2=0x55a7310b8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.909+0000 7f8e667ca740 1 --2- 172.21.15.7:0/2330605552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 0x55a7310b8410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e5802a940 tx=0x7f8e580058d0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.909+0000 7f8e667ca740 1 -- 172.21.15.7:0/2330605552 shutdown_connections 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.909+0000 7f8e667ca740 1 --2- 172.21.15.7:0/2330605552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310bacb0 0x55a7310d2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.909+0000 7f8e667ca740 1 --2- 172.21.15.7:0/2330605552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310b0410 0x55a7310d2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.909+0000 7f8e667ca740 1 --2- 172.21.15.7:0/2330605552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310afaf0 0x55a7310b8410 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.909+0000 7f8e667ca740 1 -- 172.21.15.7:0/2330605552 >> 172.21.15.7:0/2330605552 conn(0x55a7310ac060 msgr2=0x55a7310b9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.909+0000 7f8e667ca740 1 -- 172.21.15.7:0/2330605552 shutdown_connections 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.909+0000 7f8e667ca740 1 -- 172.21.15.7:0/2330605552 wait complete. 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e667ca740 1 -- start start 2024-09-10T10:58:44.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e667ca740 1 Processor -- start 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 0x55a7310be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e667ca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310bacb0 0x55a7310beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e667ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1abe50 con 0x55a7310afaf0 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e667ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1ad2c0 con 0x55a7310b0410 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e667ca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1ac520 con 0x55a7310bacb0 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e667ca740 10 client.0 _ll_register_callbacks cb 0x7f8e5409bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e616ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e60eed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 0x55a7310be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e616ee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35512/0 (socket says 172.21.15.7:35512) 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e60eed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 0x55a7310be570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:36564/0 (socket says 172.21.15.7:36564) 2024-09-10T10:58:44.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e616ee640 1 -- 172.21.15.7:0/3113891571 learned_addr learned my addr 172.21.15.7:0/3113891571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e61eef640 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310bacb0 0x55a7310beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e616ee640 1 -- 172.21.15.7:0/3113891571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310bacb0 msgr2=0x55a7310beab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e616ee640 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310bacb0 0x55a7310beab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e616ee640 1 -- 172.21.15.7:0/3113891571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 msgr2=0x55a7310be570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.910+0000 7f8e616ee640 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 0x55a7310be570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e616ee640 1 -- 172.21.15.7:0/3113891571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7310bf0f0 con 0x55a7310afaf0 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e616ee640 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310c3250 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005940 tx=0x7f8e580035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546cf59c292f304a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e44ff9640 1 -- 172.21.15.7:0/3113891571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58040070 con 0x55a7310afaf0 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e44ff9640 1 -- 172.21.15.7:0/3113891571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f8e580054a0 con 0x55a7310afaf0 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e667ca740 1 -- 172.21.15.7:0/3113891571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e58031870 con 0x55a7310afaf0 2024-09-10T10:58:44.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e44ff9640 1 -- 172.21.15.7:0/3113891571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8e58035040 con 0x55a7310afaf0 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e667ca740 10 client.16293 fetch_fsmap learned FSMap version 42 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e667ca740 1 -- 172.21.15.7:0/3113891571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a7311358c0 con 0x55a7310afaf0 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e44ff9640 1 -- 172.21.15.7:0/3113891571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e58031870 con 0x55a7310afaf0 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e44ff9640 1 -- 172.21.15.7:0/3113891571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f8e58059570 con 0x55a7310afaf0 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.911+0000 7f8e667ca740 10 client.16293 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.912+0000 7f8e667ca740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.912+0000 7f8e667ca740 20 client.16293 populate_metadata read hostname 'smithi007' 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.912+0000 7f8e667ca740 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a731135ba0 0x55a731137f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.912+0000 7f8e667ca740 4 client.16293 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.912+0000 7f8e667ca740 1 -- 172.21.15.7:0/3113891571 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a73101bd70 con 0x55a731135ba0 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.912+0000 7f8e60eed640 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a731135ba0 0x55a731137f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.912+0000 7f8e60eed640 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a731135ba0 0x55a731137f90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf7250c8d66506ab server_cookie=6e2d0a1ee5348338 in_seq=0 out_seq=0 2024-09-10T10:58:44.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.912+0000 7f8e44ff9640 10 client.16293 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e44ff9640 1 -- 172.21.15.7:0/3113891571 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a73101bd70 con 0x55a731135ba0 2024-09-10T10:58:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e44ff9640 10 client.16293 handle_command_reply: tid=0 2024-09-10T10:58:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e667ca740 1 client.16293 shutdown 2024-09-10T10:58:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e667ca740 1 -- 172.21.15.7:0/3113891571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 msgr2=0x55a7310c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e667ca740 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310c3250 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005940 tx=0x7f8e580035f0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e667ca740 1 -- 172.21.15.7:0/3113891571 shutdown_connections 2024-09-10T10:58:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e667ca740 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a731135ba0 0x55a731137f90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e667ca740 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7310bacb0 0x55a7310beab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e667ca740 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7310b0410 0x55a7310be570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e667ca740 1 --2- 172.21.15.7:0/3113891571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7310afaf0 0x55a7310c3250 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e667ca740 1 -- 172.21.15.7:0/3113891571 >> 172.21.15.7:0/3113891571 conn(0x55a7310ac060 msgr2=0x55a7310b9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.975+0000 7f8e44ff9640 0 client.16293 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e667ca740 1 -- 172.21.15.7:0/3113891571 shutdown_connections 2024-09-10T10:58:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e667ca740 1 -- 172.21.15.7:0/3113891571 wait complete. 2024-09-10T10:58:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e667ca740 20 client.16293 trim_cache size 0 max 16384 2024-09-10T10:58:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:58:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:58:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:58:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:58:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e63952640 1 -- 172.21.15.7:0/3407930993 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8e480a2da0 msgr2=0x7f8e480a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e63952640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8e480a2da0 0x7f8e480a5260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55a7310ad3e0 tx=0x7f8e4c005f90 comp rx=0 tx=0).stop 2024-09-10T10:58:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e63952640 1 -- 172.21.15.7:0/3407930993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5c1069c0 msgr2=0x7f8e5c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e63952640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5c1069c0 0x7f8e5c19cc40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400eb10 tx=0x7f8e5400c2d0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e63952640 1 -- 172.21.15.7:0/3407930993 shutdown_connections 2024-09-10T10:58:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e63952640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f8e480a2da0 0x7f8e480a5260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e63952640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5c1069c0 0x7f8e5c19cc40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e63952640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c101350 0x7f8e5c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e63952640 1 --2- 172.21.15.7:0/3407930993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e5c1009b0 0x7f8e5c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.976+0000 7f8e63952640 1 -- 172.21.15.7:0/3407930993 >> 172.21.15.7:0/3407930993 conn(0x7f8e5c077990 msgr2=0x7f8e5c069900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.977+0000 7f8e63952640 1 -- 172.21.15.7:0/3407930993 shutdown_connections 2024-09-10T10:58:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:44.977+0000 7f8e63952640 1 -- 172.21.15.7:0/3407930993 wait complete. 2024-09-10T10:58:45.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1085112 7.39 MB/sec execute 585 sec latency 56.278 ms 2024-09-10T10:58:45.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:45 smithi103 ceph-mon[35160]: pgmap v405: 97 pgs: 97 active+clean; 289 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-09-10T10:58:45.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:45 smithi007 ceph-mon[30873]: pgmap v405: 97 pgs: 97 active+clean; 289 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-09-10T10:58:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:45 smithi133 ceph-mon[35282]: pgmap v405: 97 pgs: 97 active+clean; 289 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-09-10T10:58:46.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1087249 7.40 MB/sec execute 586 sec latency 24.170 ms 2024-09-10T10:58:47.548 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1088620 7.40 MB/sec execute 587 sec latency 27.675 ms 2024-09-10T10:58:47.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:47 smithi103 ceph-mon[35160]: pgmap v406: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 274 op/s 2024-09-10T10:58:47.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:47 smithi007 ceph-mon[30873]: pgmap v406: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 274 op/s 2024-09-10T10:58:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:47 smithi133 ceph-mon[35282]: pgmap v406: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 274 op/s 2024-09-10T10:58:48.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1090731 7.41 MB/sec execute 588 sec latency 115.976 ms 2024-09-10T10:58:49.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1092016 7.41 MB/sec execute 589 sec latency 11.681 ms 2024-09-10T10:58:49.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:49 smithi103 ceph-mon[35160]: pgmap v407: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-10T10:58:49.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:49 smithi007 ceph-mon[30873]: pgmap v407: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-10T10:58:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:49 smithi133 ceph-mon[35282]: pgmap v407: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-10T10:58:50.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1094275 7.42 MB/sec execute 590 sec latency 26.173 ms 2024-09-10T10:58:51.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1095692 7.42 MB/sec execute 591 sec latency 14.249 ms 2024-09-10T10:58:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:51 smithi103 ceph-mon[35160]: pgmap v408: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-09-10T10:58:51.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:51 smithi007 ceph-mon[30873]: pgmap v408: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-09-10T10:58:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:51 smithi133 ceph-mon[35282]: pgmap v408: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-09-10T10:58:52.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1097878 7.43 MB/sec execute 592 sec latency 29.687 ms 2024-09-10T10:58:53.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1099259 7.42 MB/sec execute 593 sec latency 15.099 ms 2024-09-10T10:58:53.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:53 smithi103 ceph-mon[35160]: pgmap v409: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-10T10:58:53.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:53 smithi007 ceph-mon[30873]: pgmap v409: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-10T10:58:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:53 smithi133 ceph-mon[35282]: pgmap v409: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-10T10:58:54.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1101384 7.43 MB/sec execute 594 sec latency 80.942 ms 2024-09-10T10:58:55.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1102803 7.43 MB/sec execute 595 sec latency 14.829 ms 2024-09-10T10:58:55.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:55 smithi103 ceph-mon[35160]: pgmap v410: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-10T10:58:55.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:55 smithi007 ceph-mon[30873]: pgmap v410: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-10T10:58:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:55 smithi133 ceph-mon[35282]: pgmap v410: 97 pgs: 97 active+clean; 297 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 216 op/s 2024-09-10T10:58:56.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1104953 7.44 MB/sec execute 596 sec latency 22.826 ms 2024-09-10T10:58:57.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1106353 7.44 MB/sec execute 597 sec latency 15.642 ms 2024-09-10T10:58:57.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:57 smithi103 ceph-mon[35160]: pgmap v411: 97 pgs: 97 active+clean; 305 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-09-10T10:58:57.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:57 smithi007 ceph-mon[30873]: pgmap v411: 97 pgs: 97 active+clean; 305 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-09-10T10:58:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:57 smithi133 ceph-mon[35282]: pgmap v411: 97 pgs: 97 active+clean; 305 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 315 op/s 2024-09-10T10:58:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 -- 172.21.15.7:0/2325088856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4720102480 msgr2=0x7f472010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:58.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 --2- 172.21.15.7:0/2325088856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4720102480 0x7f472010e9b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4718009cd0 tx=0x7f471802a8e0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 -- 172.21.15.7:0/2325088856 shutdown_connections 2024-09-10T10:58:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 --2- 172.21.15.7:0/2325088856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4720102480 0x7f472010e9b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 --2- 172.21.15.7:0/2325088856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4720101ae0 0x7f4720101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 --2- 172.21.15.7:0/2325088856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4720107b80 0x7f4720107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 -- 172.21.15.7:0/2325088856 >> 172.21.15.7:0/2325088856 conn(0x7f47200fd7e0 msgr2=0x7f47200ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 -- 172.21.15.7:0/2325088856 shutdown_connections 2024-09-10T10:58:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 -- 172.21.15.7:0/2325088856 wait complete. 2024-09-10T10:58:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 Processor -- start 2024-09-10T10:58:58.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.075+0000 7f47285c4640 1 -- start start 2024-09-10T10:58:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f47285c4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4720101ae0 0x7f47201983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f47285c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4720102480 0x7f4720198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f47285c4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4720107b80 0x7f472019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f47285c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4720111f30 con 0x7f4720102480 2024-09-10T10:58:58.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f47285c4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4720111db0 con 0x7f4720101ae0 2024-09-10T10:58:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f47285c4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f47201120b0 con 0x7f4720107b80 2024-09-10T10:58:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4725b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4720102480 0x7f4720198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4725b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4720102480 0x7f4720198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39802/0 (socket says 172.21.15.7:39802) 2024-09-10T10:58:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4725b5f640 1 -- 172.21.15.7:0/1270208038 learned_addr learned my addr 172.21.15.7:0/1270208038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4726360640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4720101ae0 0x7f47201983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4726b61640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4720107b80 0x7f472019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4725b5f640 1 -- 172.21.15.7:0/1270208038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4720107b80 msgr2=0x7f472019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4725b5f640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4720107b80 0x7f472019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4725b5f640 1 -- 172.21.15.7:0/1270208038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4720101ae0 msgr2=0x7f47201983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:58.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4725b5f640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4720101ae0 0x7f47201983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4725b5f640 1 -- 172.21.15.7:0/1270208038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472019d3c0 con 0x7f4720102480 2024-09-10T10:58:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4726360640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4720101ae0 0x7f47201983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:58:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f4725b5f640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4720102480 0x7f4720198910 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f471000e9f0 tx=0x7f471000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.076+0000 7f47177fe640 1 -- 172.21.15.7:0/1270208038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471000cd80 con 0x7f4720102480 2024-09-10T10:58:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.077+0000 7f47177fe640 1 -- 172.21.15.7:0/1270208038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4710010910 con 0x7f4720102480 2024-09-10T10:58:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.077+0000 7f47285c4640 1 -- 172.21.15.7:0/1270208038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472019d6b0 con 0x7f4720102480 2024-09-10T10:58:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.077+0000 7f47177fe640 1 -- 172.21.15.7:0/1270208038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710018990 con 0x7f4720102480 2024-09-10T10:58:58.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.077+0000 7f47285c4640 1 -- 172.21.15.7:0/1270208038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47201a4f40 con 0x7f4720102480 2024-09-10T10:58:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f47177fe640 1 -- 172.21.15.7:0/1270208038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4710010ac0 con 0x7f4720102480 2024-09-10T10:58:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f472b43a740 1 Processor -- start 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f472b43a740 1 -- start start 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f472b43a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a349ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f472b43a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a34a410 0x55b51a354770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f472b43a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a3570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f472b43a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b51a2ae2f0 con 0x55b51a349af0 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f472b43a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b51a256270 con 0x55b51a34a410 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f472b43a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b51a2aa2b0 con 0x55b51a354cb0 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f4726b61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a349ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f4725b5f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a3570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f4726b61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a349ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39812/0 (socket says 172.21.15.7:39812) 2024-09-10T10:58:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f4726b61640 1 -- 172.21.15.7:0/692002558 learned_addr learned my addr 172.21.15.7:0/692002558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.080+0000 7f4726360640 1 --2- 172.21.15.7:0/692002558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a34a410 0x55b51a354770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f47177fe640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f470c0a2e20 0x7f470c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f4726b61640 1 -- 172.21.15.7:0/692002558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 msgr2=0x55b51a3570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f4726b61640 1 --2- 172.21.15.7:0/692002558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a3570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f4726b61640 1 -- 172.21.15.7:0/692002558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a34a410 msgr2=0x55b51a354770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f4726b61640 1 --2- 172.21.15.7:0/692002558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a34a410 0x55b51a354770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f4726b61640 1 -- 172.21.15.7:0/692002558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b51a255340 con 0x55b51a349af0 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f47177fe640 1 -- 172.21.15.7:0/1270208038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f47100cf6f0 con 0x7f4720102480 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f4726b61640 1 --2- 172.21.15.7:0/692002558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a349ed0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4718009cd0 tx=0x7f4718009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573aa84c3b15f0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f4726b61640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f470c0a2e20 0x7f470c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f4714bfe640 1 -- 172.21.15.7:0/692002558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718048070 con 0x55b51a349af0 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f4714bfe640 1 -- 172.21.15.7:0/692002558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4718036070 con 0x55b51a349af0 2024-09-10T10:58:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.081+0000 7f4714bfe640 1 -- 172.21.15.7:0/692002558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47180305a0 con 0x55b51a349af0 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.082+0000 7f472b43a740 1 -- 172.21.15.7:0/692002558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 msgr2=0x55b51a349ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.082+0000 7f4726b61640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f470c0a2e20 0x7f470c0a52e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4718045000 tx=0x7f4718030bf0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.082+0000 7f472b43a740 1 --2- 172.21.15.7:0/692002558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a349ed0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4718009cd0 tx=0x7f4718009a20 comp rx=0 tx=0).stop 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.082+0000 7f472b43a740 1 -- 172.21.15.7:0/692002558 shutdown_connections 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.082+0000 7f472b43a740 1 --2- 172.21.15.7:0/692002558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a3570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.082+0000 7f472b43a740 1 --2- 172.21.15.7:0/692002558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a34a410 0x55b51a354770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.082+0000 7f472b43a740 1 --2- 172.21.15.7:0/692002558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a349ed0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.082+0000 7f472b43a740 1 -- 172.21.15.7:0/692002558 >> 172.21.15.7:0/692002558 conn(0x55b51a346060 msgr2=0x55b51a346440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.082+0000 7f472b43a740 1 -- 172.21.15.7:0/692002558 shutdown_connections 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.082+0000 7f472b43a740 1 -- 172.21.15.7:0/692002558 wait complete. 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f472b43a740 1 -- start start 2024-09-10T10:58:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f472b43a740 1 Processor -- start 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f472b43a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a349af0 0x55b51a351ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f472b43a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a3520e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f472b43a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a352620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f472b43a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b51a357880 con 0x55b51a34a410 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f472b43a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b51a237290 con 0x55b51a349af0 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f472b43a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b51a357a00 con 0x55b51a354cb0 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f472b43a740 10 client.0 _ll_register_callbacks cb 0x55b51a268140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f4725b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a3520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f4725b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a3520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39824/0 (socket says 172.21.15.7:39824) 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f4725b5f640 1 -- 172.21.15.7:0/2665722886 learned_addr learned my addr 172.21.15.7:0/2665722886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f4726360640 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a349af0 0x55b51a351ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f4726b61640 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a352620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f4725b5f640 1 -- 172.21.15.7:0/2665722886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 msgr2=0x55b51a352620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f4725b5f640 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a352620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f4725b5f640 1 -- 172.21.15.7:0/2665722886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a349af0 msgr2=0x55b51a351ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f4725b5f640 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a349af0 0x55b51a351ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.083+0000 7f4725b5f640 1 -- 172.21.15.7:0/2665722886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b519fc1210 con 0x55b51a34a410 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f4725b5f640 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a3520e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f471000e9c0 tx=0x7f4710097ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c4470df11eaf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f4709ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f4709ffb640 1 -- 172.21.15.7:0/2665722886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471009bc90 con 0x55b51a34a410 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f4709ffb640 1 -- 172.21.15.7:0/2665722886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f47100cfba0 con 0x55b51a34a410 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f472b43a740 1 -- 172.21.15.7:0/2665722886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4718036070 con 0x55b51a34a410 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f4709ffb640 1 -- 172.21.15.7:0/2665722886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f471009ff90 con 0x55b51a34a410 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f472b43a740 10 client.16305 fetch_fsmap learned FSMap version 42 2024-09-10T10:58:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f472b43a740 1 -- 172.21.15.7:0/2665722886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b51a08d620 con 0x55b51a34a410 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f4709ffb640 1 -- 172.21.15.7:0/2665722886 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4718036070 con 0x55b51a34a410 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f4709ffb640 1 -- 172.21.15.7:0/2665722886 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f47100d1ce0 con 0x55b51a34a410 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f472b43a740 10 client.16305 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f472b43a740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f472b43a740 20 client.16305 populate_metadata read hostname 'smithi007' 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f472b43a740 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b51a364c00 0x55b51a366ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f472b43a740 4 client.16305 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.084+0000 7f472b43a740 1 -- 172.21.15.7:0/2665722886 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b51a2bba20 con 0x55b51a364c00 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.085+0000 7f4726360640 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b51a364c00 0x55b51a366ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.085+0000 7f4726360640 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b51a364c00 0x55b51a366ff0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b04dafeb377cdcf6 server_cookie=82e792aa5700fd4 in_seq=0 out_seq=0 2024-09-10T10:58:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.085+0000 7f4709ffb640 10 client.16305 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f4709ffb640 1 -- 172.21.15.7:0/2665722886 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b51a2bba20 con 0x55b51a364c00 2024-09-10T10:58:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f4709ffb640 10 client.16305 handle_command_reply: tid=0 2024-09-10T10:58:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f472b43a740 1 client.16305 shutdown 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f472b43a740 1 -- 172.21.15.7:0/2665722886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 msgr2=0x55b51a3520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f472b43a740 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a3520e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f471000e9c0 tx=0x7f4710097ff0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f472b43a740 1 -- 172.21.15.7:0/2665722886 shutdown_connections 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f472b43a740 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b51a364c00 0x55b51a366ff0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f472b43a740 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a352620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f472b43a740 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a3520e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f4709ffb640 0 client.16305 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.086+0000 7f472b43a740 1 --2- 172.21.15.7:0/2665722886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a349af0 0x55b51a351ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.087+0000 7f472b43a740 1 -- 172.21.15.7:0/2665722886 >> 172.21.15.7:0/2665722886 conn(0x55b51a346060 msgr2=0x55b51a3451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.087+0000 7f472b43a740 1 -- 172.21.15.7:0/2665722886 shutdown_connections 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.087+0000 7f472b43a740 1 -- 172.21.15.7:0/2665722886 wait complete. 2024-09-10T10:58:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.087+0000 7f472b43a740 20 client.16305 trim_cache size 0 max 16384 2024-09-10T10:58:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.101+0000 7f472b43a740 1 Processor -- start 2024-09-10T10:58:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.101+0000 7f472b43a740 1 -- start start 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.101+0000 7f472b43a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a349af0 0x55b51a352450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.101+0000 7f472b43a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a36c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.101+0000 7f472b43a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a36ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.101+0000 7f472b43a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47201b0020 con 0x55b51a34a410 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.101+0000 7f472b43a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47201ad8f0 con 0x55b51a349af0 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.101+0000 7f472b43a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f47201ab760 con 0x55b51a354cb0 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4725b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a36c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4725b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a36c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39832/0 (socket says 172.21.15.7:39832) 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4725b5f640 1 -- 172.21.15.7:0/3966499547 learned_addr learned my addr 172.21.15.7:0/3966499547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4726360640 1 --2- 172.21.15.7:0/3966499547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a349af0 0x55b51a352450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4726b61640 1 --2- 172.21.15.7:0/3966499547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a36ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4725b5f640 1 -- 172.21.15.7:0/3966499547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 msgr2=0x55b51a36ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4725b5f640 1 --2- 172.21.15.7:0/3966499547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a36ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4725b5f640 1 -- 172.21.15.7:0/3966499547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a349af0 msgr2=0x55b51a352450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4725b5f640 1 --2- 172.21.15.7:0/3966499547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a349af0 0x55b51a352450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4725b5f640 1 -- 172.21.15.7:0/3966499547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b51a367650 con 0x55b51a34a410 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4726b61640 1 --2- 172.21.15.7:0/3966499547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a36ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f4725b5f640 1 --2- 172.21.15.7:0/3966499547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a36c8e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f471009bc20 tx=0x7f4710097c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7084f1968ea3ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f470bfff640 1 -- 172.21.15.7:0/3966499547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471009b7d0 con 0x55b51a34a410 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f470bfff640 1 -- 172.21.15.7:0/3966499547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f471009bc90 con 0x55b51a34a410 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.102+0000 7f470bfff640 1 -- 172.21.15.7:0/3966499547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710018b30 con 0x55b51a34a410 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 -- 172.21.15.7:0/3966499547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 msgr2=0x55b51a36c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 --2- 172.21.15.7:0/3966499547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a36c8e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f471009bc20 tx=0x7f4710097c10 comp rx=0 tx=0).stop 2024-09-10T10:58:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 -- 172.21.15.7:0/3966499547 shutdown_connections 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 --2- 172.21.15.7:0/3966499547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a36ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 --2- 172.21.15.7:0/3966499547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a34a410 0x55b51a36c8e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 --2- 172.21.15.7:0/3966499547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a349af0 0x55b51a352450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 -- 172.21.15.7:0/3966499547 >> 172.21.15.7:0/3966499547 conn(0x55b51a346060 msgr2=0x55b51a353c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 -- 172.21.15.7:0/3966499547 shutdown_connections 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 -- 172.21.15.7:0/3966499547 wait complete. 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 -- start start 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 Processor -- start 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a35d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.103+0000 7f472b43a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a34a410 0x55b51a358870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f472b43a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a359000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f472b43a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47201abdb0 con 0x55b51a349af0 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f472b43a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47201ad220 con 0x55b51a34a410 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f472b43a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f47201ac480 con 0x55b51a354cb0 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f472b43a740 10 client.0 _ll_register_callbacks cb 0x7f47100d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4726360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a35d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4726360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a35d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39838/0 (socket says 172.21.15.7:39838) 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4726b61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a359000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4726360640 1 -- 172.21.15.7:0/3621039032 learned_addr learned my addr 172.21.15.7:0/3621039032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4725b5f640 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a34a410 0x55b51a358870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4726360640 1 -- 172.21.15.7:0/3621039032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 msgr2=0x55b51a359000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4726360640 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a359000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4726360640 1 -- 172.21.15.7:0/3621039032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a34a410 msgr2=0x55b51a358870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4726360640 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a34a410 0x55b51a358870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4726360640 1 -- 172.21.15.7:0/3621039032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b51a346cf0 con 0x55b51a349af0 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4726360640 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a35d550 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55b51a367850 tx=0x7f471c012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294213563cff467a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4709ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4709ffb640 1 -- 172.21.15.7:0/3621039032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c005220 con 0x55b51a349af0 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f4709ffb640 1 -- 172.21.15.7:0/3621039032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8564+0+0 (secure 0 0 0) 0x7f471c0053c0 con 0x55b51a349af0 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.104+0000 7f472b43a740 1 -- 172.21.15.7:0/3621039032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f471009bc90 con 0x55b51a349af0 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f4709ffb640 1 -- 172.21.15.7:0/3621039032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f471c0055c0 con 0x55b51a349af0 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f472b43a740 10 client.16311 fetch_fsmap learned FSMap version 42 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f472b43a740 1 -- 172.21.15.7:0/3621039032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b51a349920 con 0x55b51a349af0 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f4709ffb640 1 -- 172.21.15.7:0/3621039032 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f471009bc90 con 0x55b51a349af0 2024-09-10T10:58:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f4709ffb640 1 -- 172.21.15.7:0/3621039032 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 42) ==== 8346+0+0 (secure 0 0 0) 0x7f471c01fb40 con 0x55b51a349af0 2024-09-10T10:58:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f472b43a740 10 client.16311 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:58:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f472b43a740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:58:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f472b43a740 20 client.16311 populate_metadata read hostname 'smithi007' 2024-09-10T10:58:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f472b43a740 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b51a3cfe50 0x55b51a3d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:58:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f472b43a740 4 client.16311 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:58:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f472b43a740 1 -- 172.21.15.7:0/3621039032 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b51a2b5d70 con 0x55b51a3cfe50 2024-09-10T10:58:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.105+0000 7f4725b5f640 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b51a3cfe50 0x55b51a3d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:58:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.106+0000 7f4725b5f640 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b51a3cfe50 0x55b51a3d2270 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3f8f1d269d3db2 server_cookie=c1ce22b3815b39ac in_seq=0 out_seq=0 2024-09-10T10:58:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.106+0000 7f4709ffb640 10 client.16311 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.175+0000 7f4709ffb640 1 -- 172.21.15.7:0/3621039032 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b51a2b5d70 con 0x55b51a3cfe50 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.175+0000 7f4709ffb640 10 client.16311 handle_command_reply: tid=0 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.175+0000 7f472b43a740 1 client.16311 shutdown 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 1 -- 172.21.15.7:0/3621039032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 msgr2=0x55b51a35d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a35d550 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55b51a367850 tx=0x7f471c012b00 comp rx=0 tx=0).stop 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 1 -- 172.21.15.7:0/3621039032 shutdown_connections 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55b51a3cfe50 0x55b51a3d2270 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b51a354cb0 0x55b51a359000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b51a34a410 0x55b51a358870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 1 --2- 172.21.15.7:0/3621039032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b51a349af0 0x55b51a35d550 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 1 -- 172.21.15.7:0/3621039032 >> 172.21.15.7:0/3621039032 conn(0x55b51a346060 msgr2=0x55b51a3449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f4709ffb640 0 client.16311 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:58:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 1 -- 172.21.15.7:0/3621039032 shutdown_connections 2024-09-10T10:58:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 1 -- 172.21.15.7:0/3621039032 wait complete. 2024-09-10T10:58:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.176+0000 7f472b43a740 20 client.16311 trim_cache size 0 max 16384 2024-09-10T10:58:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:58:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:58:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:58:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:58:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.177+0000 7f47285c4640 1 -- 172.21.15.7:0/1270208038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f470c0a2e20 msgr2=0x7f470c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.177+0000 7f47285c4640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f470c0a2e20 0x7f470c0a52e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4718045000 tx=0x7f4718030bf0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.177+0000 7f47285c4640 1 -- 172.21.15.7:0/1270208038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4720102480 msgr2=0x7f4720198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:58:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.177+0000 7f47285c4640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4720102480 0x7f4720198910 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f471000e9f0 tx=0x7f471000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.178+0000 7f47285c4640 1 -- 172.21.15.7:0/1270208038 shutdown_connections 2024-09-10T10:58:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.178+0000 7f47285c4640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4720107b80 0x7f472019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.178+0000 7f47285c4640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f470c0a2e20 0x7f470c0a52e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.178+0000 7f47285c4640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4720102480 0x7f4720198910 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.178+0000 7f47285c4640 1 --2- 172.21.15.7:0/1270208038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4720101ae0 0x7f47201983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:58:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.178+0000 7f47285c4640 1 -- 172.21.15.7:0/1270208038 >> 172.21.15.7:0/1270208038 conn(0x7f47200fd7e0 msgr2=0x7f47200feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:58:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.178+0000 7f47285c4640 1 -- 172.21.15.7:0/1270208038 shutdown_connections 2024-09-10T10:58:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:58:58.178+0000 7f47285c4640 1 -- 172.21.15.7:0/1270208038 wait complete. 2024-09-10T10:58:58.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1108400 7.45 MB/sec execute 598 sec latency 52.867 ms 2024-09-10T10:58:59.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 1109496 7.44 MB/sec execute 599 sec latency 12.688 ms 2024-09-10T10:58:59.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:58:59 smithi103 ceph-mon[35160]: pgmap v412: 97 pgs: 97 active+clean; 305 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-09-10T10:58:59.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:58:59 smithi007 ceph-mon[30873]: pgmap v412: 97 pgs: 97 active+clean; 305 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-09-10T10:58:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:58:59 smithi133 ceph-mon[35282]: pgmap v412: 97 pgs: 97 active+clean; 305 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-09-10T10:59:00.546 INFO:tasks.workunit.client.0.smithi007.stdout: 1 cleanup 600 sec 2024-09-10T10:59:00.779 INFO:tasks.workunit.client.0.smithi007.stdout: 0 cleanup 600 sec 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: Operation Count AvgLat MaxLat 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: ---------------------------------------- 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: NTCreateX 142043 2.264 4961.259 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: Close 104363 0.002 0.089 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: Rename 6021 6.484 4958.423 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: Unlink 28681 2.987 14163.078 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: Deltree 4 76.903 120.054 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: Mkdir 2 0.005 0.006 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: Qpathinfo 128770 0.823 134.497 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: Qfileinfo 22597 0.001 0.033 2024-09-10T10:59:00.780 INFO:tasks.workunit.client.0.smithi007.stdout: Qfsinfo 23638 0.132 7.458 2024-09-10T10:59:00.781 INFO:tasks.workunit.client.0.smithi007.stdout: Sfileinfo 11574 0.016 4.502 2024-09-10T10:59:00.781 INFO:tasks.workunit.client.0.smithi007.stdout: Find 49824 0.155 26.406 2024-09-10T10:59:00.781 INFO:tasks.workunit.client.0.smithi007.stdout: WriteX 70947 0.060 74.150 2024-09-10T10:59:00.781 INFO:tasks.workunit.client.0.smithi007.stdout: ReadX 222989 0.005 26.804 2024-09-10T10:59:00.781 INFO:tasks.workunit.client.0.smithi007.stdout: LockX 464 0.454 1.378 2024-09-10T10:59:00.781 INFO:tasks.workunit.client.0.smithi007.stdout: UnlockX 464 0.432 1.721 2024-09-10T10:59:00.781 INFO:tasks.workunit.client.0.smithi007.stdout: Flush 9958 2.859 307.349 2024-09-10T10:59:00.781 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-09-10T10:59:00.781 INFO:tasks.workunit.client.0.smithi007.stdout:Throughput 7.43881 MB/sec 1 clients 1 procs max_latency=14163.096 ms 2024-09-10T10:59:00.782 INFO:tasks.workunit.client.0.smithi007.stderr:+ dbench 10 2024-09-10T10:59:00.782 INFO:tasks.workunit.client.0.smithi007.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-10T10:59:00.782 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-09-10T10:59:00.782 INFO:tasks.workunit.client.0.smithi007.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-10T10:59:00.782 INFO:tasks.workunit.client.0.smithi007.stdout:9 of 10 processes prepared for launch 0 sec 2024-09-10T10:59:00.881 INFO:tasks.workunit.client.0.smithi007.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-10T10:59:00.881 INFO:tasks.workunit.client.0.smithi007.stdout:releasing clients 2024-09-10T10:59:01.881 INFO:tasks.workunit.client.0.smithi007.stdout: 10 442 155.68 MB/sec warmup 1 sec latency 46.335 ms 2024-09-10T10:59:01.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:01 smithi103 ceph-mon[35160]: pgmap v413: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 343 op/s 2024-09-10T10:59:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:01 smithi007 ceph-mon[30873]: pgmap v413: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 343 op/s 2024-09-10T10:59:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:01 smithi133 ceph-mon[35282]: pgmap v413: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 343 op/s 2024-09-10T10:59:02.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 963 108.23 MB/sec warmup 2 sec latency 621.571 ms 2024-09-10T10:59:02.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:02 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:02.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:02 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:02.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:02 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:03.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 1537 83.14 MB/sec warmup 3 sec latency 1621.610 ms 2024-09-10T10:59:03.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:03 smithi007 ceph-mon[30873]: pgmap v414: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-09-10T10:59:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:03 smithi133 ceph-mon[35282]: pgmap v414: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-09-10T10:59:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:03 smithi103 ceph-mon[35160]: pgmap v414: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-09-10T10:59:04.881 INFO:tasks.workunit.client.0.smithi007.stdout: 10 2291 67.84 MB/sec warmup 4 sec latency 2621.646 ms 2024-09-10T10:59:04.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:04 smithi007 ceph-mon[30873]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T10:59:04.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:04 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:04 smithi133 ceph-mon[35282]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T10:59:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:04 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:05.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:04 smithi103 ceph-mon[35160]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T10:59:05.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:04 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:05.881 INFO:tasks.workunit.client.0.smithi007.stdout: 10 3049 66.26 MB/sec warmup 5 sec latency 3621.745 ms 2024-09-10T10:59:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:05 smithi133 ceph-mon[35282]: pgmap v416: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-09-10T10:59:06.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:05 smithi103 ceph-mon[35160]: pgmap v416: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-09-10T10:59:06.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:05 smithi007 ceph-mon[30873]: pgmap v416: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 263 op/s 2024-09-10T10:59:06.881 INFO:tasks.workunit.client.0.smithi007.stdout: 10 3486 61.91 MB/sec warmup 6 sec latency 4621.787 ms 2024-09-10T10:59:07.881 INFO:tasks.workunit.client.0.smithi007.stdout: 10 3698 53.47 MB/sec warmup 7 sec latency 5621.817 ms 2024-09-10T10:59:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:07 smithi133 ceph-mon[35282]: pgmap v417: 97 pgs: 97 active+clean; 293 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-10T10:59:07.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:07 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:59:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:07 smithi103 ceph-mon[35160]: pgmap v417: 97 pgs: 97 active+clean; 293 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-10T10:59:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:07 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:59:08.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:07 smithi007 ceph-mon[30873]: pgmap v417: 97 pgs: 97 active+clean; 293 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-10T10:59:08.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:07 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:59:08.881 INFO:tasks.workunit.client.0.smithi007.stdout: 10 3895 47.07 MB/sec warmup 8 sec latency 6621.864 ms 2024-09-10T10:59:09.881 INFO:tasks.workunit.client.0.smithi007.stdout: 10 4440 45.03 MB/sec warmup 9 sec latency 7621.892 ms 2024-09-10T10:59:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:09 smithi103 ceph-mon[35160]: pgmap v418: 97 pgs: 97 active+clean; 293 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-10T10:59:09.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:09 smithi103 ceph-mon[35160]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T10:59:09.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:09 smithi007 ceph-mon[30873]: pgmap v418: 97 pgs: 97 active+clean; 293 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-10T10:59:09.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:09 smithi007 ceph-mon[30873]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T10:59:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:09 smithi133 ceph-mon[35282]: pgmap v418: 97 pgs: 97 active+clean; 293 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-10T10:59:09.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:09 smithi133 ceph-mon[35282]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T10:59:10.881 INFO:tasks.workunit.client.0.smithi007.stdout: 10 5383 43.72 MB/sec warmup 10 sec latency 8621.921 ms 2024-09-10T10:59:10.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:10 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:59:10.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:10 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:59:10.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:10 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:59:10.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:10 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:59:10.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:10 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:59:10.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:10 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:59:10.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:10 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:59:10.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:10 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:59:10.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:10 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T10:59:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.276+0000 7f1fa285e640 1 -- 172.21.15.7:0/3886198110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c107b80 msgr2=0x7f1f9c107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.276+0000 7f1fa285e640 1 --2- 172.21.15.7:0/3886198110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c107b80 0x7f1f9c107f60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009930 tx=0x7f1f8c02a420 comp rx=0 tx=0).stop 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.276+0000 7f1fa285e640 1 -- 172.21.15.7:0/3886198110 shutdown_connections 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.276+0000 7f1fa285e640 1 --2- 172.21.15.7:0/3886198110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f9c102480 0x7f1f9c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.276+0000 7f1fa285e640 1 --2- 172.21.15.7:0/3886198110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c101ae0 0x7f1f9c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.276+0000 7f1fa285e640 1 --2- 172.21.15.7:0/3886198110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c107b80 0x7f1f9c107f60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.276+0000 7f1fa285e640 1 -- 172.21.15.7:0/3886198110 >> 172.21.15.7:0/3886198110 conn(0x7f1f9c0fd7e0 msgr2=0x7f1f9c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.276+0000 7f1fa285e640 1 -- 172.21.15.7:0/3886198110 shutdown_connections 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.276+0000 7f1fa285e640 1 -- 172.21.15.7:0/3886198110 wait complete. 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1fa285e640 1 Processor -- start 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1fa285e640 1 -- start start 2024-09-10T10:59:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1fa285e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101ae0 0x7f1f9c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1fa285e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c102480 0x7f1f9c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1fa285e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f9c107b80 0x7f1f9c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1fa285e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c111f80 con 0x7f1f9c101ae0 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1fa285e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c111e00 con 0x7f1f9c102480 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1fa285e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c112100 con 0x7f1f9c107b80 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101ae0 0x7f1f9c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1f9b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c102480 0x7f1f9c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1fa0dfb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f9c107b80 0x7f1f9c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1f9b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c102480 0x7f1f9c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:43982/0 (socket says 172.21.15.7:43982) 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101ae0 0x7f1f9c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44644/0 (socket says 172.21.15.7:44644) 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.277+0000 7f1f9b7fe640 1 -- 172.21.15.7:0/1998435454 learned_addr learned my addr 172.21.15.7:0/1998435454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1f9b7fe640 1 -- 172.21.15.7:0/1998435454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f9c107b80 msgr2=0x7f1f9c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f9c107b80 0x7f1f9c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1f9b7fe640 1 -- 172.21.15.7:0/1998435454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101ae0 msgr2=0x7f1f9c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101ae0 0x7f1f9c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1f9b7fe640 1 -- 172.21.15.7:0/1998435454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c19d380 con 0x7f1f9c102480 2024-09-10T10:59:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1fa0dfb640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f9c107b80 0x7f1f9c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101ae0 0x7f1f9c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c102480 0x7f1f9c1988d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000e9f0 tx=0x7f1f9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1f997fa640 1 -- 172.21.15.7:0/1998435454 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9000cd80 con 0x7f1f9c102480 2024-09-10T10:59:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1f997fa640 1 -- 172.21.15.7:0/1998435454 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f90010910 con 0x7f1f9c102480 2024-09-10T10:59:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1fa285e640 1 -- 172.21.15.7:0/1998435454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c19d670 con 0x7f1f9c102480 2024-09-10T10:59:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1f997fa640 1 -- 172.21.15.7:0/1998435454 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90018990 con 0x7f1f9c102480 2024-09-10T10:59:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.278+0000 7f1fa285e640 1 -- 172.21.15.7:0/1998435454 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c1a4f00 con 0x7f1f9c102480 2024-09-10T10:59:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1f997fa640 1 -- 172.21.15.7:0/1998435454 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1f90010ac0 con 0x7f1f9c102480 2024-09-10T10:59:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1fa55f9740 1 Processor -- start 2024-09-10T10:59:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1fa55f9740 1 -- start start 2024-09-10T10:59:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c24af0 0x560fd3c24ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c2f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c2fcb0 0x560fd3c320a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560fd3b892f0 con 0x560fd3c2fcb0 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560fd3b31270 con 0x560fd3c24af0 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560fd3b852b0 con 0x560fd3c25410 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c2f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1fa0dfb640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c24af0 0x560fd3c24ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c2f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:36148/0 (socket says 172.21.15.7:36148) 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1f9bfff640 1 -- 172.21.15.7:0/1515166723 learned_addr learned my addr 172.21.15.7:0/1515166723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.281+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/1515166723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c2fcb0 0x560fd3c320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f997fa640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1f840a2e20 0x7f1f840a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1fa0dfb640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1f840a2e20 0x7f1f840a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f9bfff640 1 -- 172.21.15.7:0/1515166723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c24af0 msgr2=0x560fd3c24ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/1515166723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c24af0 0x560fd3c24ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f9bfff640 1 -- 172.21.15.7:0/1515166723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c2fcb0 msgr2=0x560fd3c320a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/1515166723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c2fcb0 0x560fd3c320a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f997fa640 1 -- 172.21.15.7:0/1998435454 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f1f900cf6f0 con 0x7f1f9c102480 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f9bfff640 1 -- 172.21.15.7:0/1515166723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd3b30340 con 0x560fd3c25410 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/1515166723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c2f770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009930 tx=0x7f1f8c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd5c475c353469ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1fa0dfb640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1f840a2e20 0x7f1f840a52e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1f940094d0 tx=0x7f1f94004ab0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f827fc640 1 -- 172.21.15.7:0/1515166723 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c035040 con 0x560fd3c25410 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f827fc640 1 -- 172.21.15.7:0/1515166723 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8c004760 con 0x560fd3c25410 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.282+0000 7f1f827fc640 1 -- 172.21.15.7:0/1515166723 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0038e0 con 0x560fd3c25410 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.283+0000 7f1fa55f9740 1 -- 172.21.15.7:0/1515166723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 msgr2=0x560fd3c2f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.283+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/1515166723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c2f770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009930 tx=0x7f1f8c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.283+0000 7f1fa55f9740 1 -- 172.21.15.7:0/1515166723 shutdown_connections 2024-09-10T10:59:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.283+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/1515166723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c2fcb0 0x560fd3c320a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.283+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/1515166723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c2f770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.283+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/1515166723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c24af0 0x560fd3c24ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.283+0000 7f1fa55f9740 1 -- 172.21.15.7:0/1515166723 >> 172.21.15.7:0/1515166723 conn(0x560fd3c21060 msgr2=0x560fd3c21440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.283+0000 7f1fa55f9740 1 -- 172.21.15.7:0/1515166723 shutdown_connections 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.283+0000 7f1fa55f9740 1 -- 172.21.15.7:0/1515166723 wait complete. 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.283+0000 7f1fa55f9740 1 -- start start 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1fa55f9740 1 Processor -- start 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c24af0 0x560fd3c2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c25410 0x560fd3c2d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560fd3c32880 con 0x560fd3c25410 2024-09-10T10:59:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560fd3b12290 con 0x560fd3c2fcb0 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560fd3c32a00 con 0x560fd3c24af0 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1fa55f9740 10 client.0 _ll_register_callbacks cb 0x560fd3b43140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c24af0 0x560fd3c2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1f9b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c25410 0x560fd3c2d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c24af0 0x560fd3c2cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:36150/0 (socket says 172.21.15.7:36150) 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1f9bfff640 1 -- 172.21.15.7:0/3484013921 learned_addr learned my addr 172.21.15.7:0/3484013921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1fa0dfb640 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1f9bfff640 1 -- 172.21.15.7:0/3484013921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 msgr2=0x560fd3c2d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c2d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1f9bfff640 1 -- 172.21.15.7:0/3484013921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c25410 msgr2=0x560fd3c2d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c25410 0x560fd3c2d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1f9bfff640 1 -- 172.21.15.7:0/3484013921 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd389c210 con 0x560fd3c24af0 2024-09-10T10:59:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c25410 0x560fd3c2d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.284+0000 7f1fa0dfb640 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c2d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.285+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c24af0 0x560fd3c2cb10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c02a940 tx=0x7f1f8c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba1d573ad0428a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.285+0000 7f1f7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.285+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/3484013921 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c053060 con 0x560fd3c24af0 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.285+0000 7f1fa55f9740 1 -- 172.21.15.7:0/3484013921 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f8c004760 con 0x560fd3c24af0 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.285+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/3484013921 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f1f8c005130 con 0x560fd3c24af0 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.285+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/3484013921 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1f8c035050 con 0x560fd3c24af0 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.285+0000 7f1fa55f9740 10 client.25975 fetch_fsmap learned FSMap version 44 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.285+0000 7f1fa55f9740 1 -- 172.21.15.7:0/3484013921 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560fd3968620 con 0x560fd3c24af0 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.285+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/3484013921 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8c004760 con 0x560fd3c24af0 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.286+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/3484013921 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7f1f8c0315b0 con 0x560fd3c24af0 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.286+0000 7f1fa55f9740 10 client.25975 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T10:59:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.286+0000 7f1fa55f9740 10 client.25975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:59:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.286+0000 7f1fa55f9740 20 client.25975 populate_metadata read hostname 'smithi007' 2024-09-10T10:59:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.286+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560fd3c3fa00 0x560fd3c41df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.286+0000 7f1fa55f9740 4 client.25975 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:59:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.286+0000 7f1fa55f9740 1 -- 172.21.15.7:0/3484013921 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fd3b96a20 con 0x560fd3c3fa00 2024-09-10T10:59:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.286+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560fd3c3fa00 0x560fd3c41df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.286+0000 7f1f7b7fe640 10 client.25975 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.286+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560fd3c3fa00 0x560fd3c41df0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e91ecfc8274177f server_cookie=5b6398d862025c6c in_seq=0 out_seq=0 2024-09-10T10:59:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/3484013921 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fd3b96a20 con 0x560fd3c3fa00 2024-09-10T10:59:11.296 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1f7b7fe640 10 client.25975 handle_command_reply: tid=0 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 client.25975 shutdown 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 -- 172.21.15.7:0/3484013921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c24af0 msgr2=0x560fd3c2cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c24af0 0x560fd3c2cb10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c02a940 tx=0x7f1f8c004b40 comp rx=0 tx=0).stop 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 -- 172.21.15.7:0/3484013921 shutdown_connections 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560fd3c3fa00 0x560fd3c41df0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c2d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1f7b7fe640 0 client.25975 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c25410 0x560fd3c2d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/3484013921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c24af0 0x560fd3c2cb10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 -- 172.21.15.7:0/3484013921 >> 172.21.15.7:0/3484013921 conn(0x560fd3c21060 msgr2=0x560fd3c26d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 -- 172.21.15.7:0/3484013921 shutdown_connections 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 1 -- 172.21.15.7:0/3484013921 wait complete. 2024-09-10T10:59:11.297 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.289+0000 7f1fa55f9740 20 client.25975 trim_cache size 0 max 16384 2024-09-10T10:59:11.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1fa55f9740 1 Processor -- start 2024-09-10T10:59:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1fa55f9740 1 -- start start 2024-09-10T10:59:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c2d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c200b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b00b0 con 0x560fd3c24af0 2024-09-10T10:59:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1ad950 con 0x560fd3c2fcb0 2024-09-10T10:59:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1ab7c0 con 0x560fd3c25410 2024-09-10T10:59:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c2d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c2d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44676/0 (socket says 172.21.15.7:44676) 2024-09-10T10:59:11.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1f9bfff640 1 -- 172.21.15.7:0/615822625 learned_addr learned my addr 172.21.15.7:0/615822625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.306+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/615822625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c200b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.307+0000 7f1fa0dfb640 1 --2- 172.21.15.7:0/615822625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.307+0000 7f1f9bfff640 1 -- 172.21.15.7:0/615822625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 msgr2=0x560fd3c200b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.307+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/615822625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c200b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.307+0000 7f1f9bfff640 1 -- 172.21.15.7:0/615822625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 msgr2=0x560fd3c473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.307+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/615822625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.307+0000 7f1f9bfff640 1 -- 172.21.15.7:0/615822625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd3c20e90 con 0x560fd3c24af0 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.307+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/615822625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c2d3c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009800 tx=0x7f1f8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8992fbe57b659980 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.307+0000 7f1f81ffb640 1 -- 172.21.15.7:0/615822625 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c053070 con 0x560fd3c24af0 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.307+0000 7f1f81ffb640 1 -- 172.21.15.7:0/615822625 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8c005610 con 0x560fd3c24af0 2024-09-10T10:59:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.307+0000 7f1f81ffb640 1 -- 172.21.15.7:0/615822625 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c003c30 con 0x560fd3c24af0 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 -- 172.21.15.7:0/615822625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 msgr2=0x560fd3c2d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/615822625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c2d3c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009800 tx=0x7f1f8c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 -- 172.21.15.7:0/615822625 shutdown_connections 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/615822625 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/615822625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c200b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/615822625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c2d3c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 -- 172.21.15.7:0/615822625 >> 172.21.15.7:0/615822625 conn(0x560fd3c21060 msgr2=0x560fd3c1f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 -- 172.21.15.7:0/615822625 shutdown_connections 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 -- 172.21.15.7:0/615822625 wait complete. 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 -- start start 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.308+0000 7f1fa55f9740 1 Processor -- start 2024-09-10T10:59:11.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1fa55f9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c32c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1abe10 con 0x560fd3c24af0 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1ad280 con 0x560fd3c2fcb0 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1fa55f9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1ac4e0 con 0x560fd3c25410 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1fa55f9740 10 client.0 _ll_register_callbacks cb 0x7f1f9009ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c37ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44680/0 (socket says 172.21.15.7:44680) 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9bfff640 1 -- 172.21.15.7:0/2855053492 learned_addr learned my addr 172.21.15.7:0/2855053492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c381e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:36164/0 (socket says 172.21.15.7:36164) 2024-09-10T10:59:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1fa0dfb640 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c32c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9bfff640 1 -- 172.21.15.7:0/2855053492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 msgr2=0x560fd3c381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9bfff640 1 -- 172.21.15.7:0/2855053492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 msgr2=0x560fd3c32c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c32c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9bfff640 1 -- 172.21.15.7:0/2855053492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd3c33190 con 0x560fd3c24af0 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f9bfff640 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c37ca0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c02a940 tx=0x7f1f8c004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66eebd28ebd05aae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.309+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/2855053492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c053040 con 0x560fd3c24af0 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/2855053492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f1f8c005130 con 0x560fd3c24af0 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1fa55f9740 1 -- 172.21.15.7:0/2855053492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f8c005610 con 0x560fd3c24af0 2024-09-10T10:59:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/2855053492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1f8c06f050 con 0x560fd3c24af0 2024-09-10T10:59:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1fa55f9740 10 client.16329 fetch_fsmap learned FSMap version 44 2024-09-10T10:59:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1fa55f9740 1 -- 172.21.15.7:0/2855053492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560fd3caaaa0 con 0x560fd3c24af0 2024-09-10T10:59:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/2855053492 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8c005610 con 0x560fd3c24af0 2024-09-10T10:59:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/2855053492 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7f1f8c06a030 con 0x560fd3c24af0 2024-09-10T10:59:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1fa55f9740 10 client.16329 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T10:59:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1fa55f9740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:59:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1fa55f9740 20 client.16329 populate_metadata read hostname 'smithi007' 2024-09-10T10:59:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560fd3cab0f0 0x560fd3cad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1fa55f9740 4 client.16329 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:59:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1fa55f9740 1 -- 172.21.15.7:0/2855053492 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fd3b90d70 con 0x560fd3cab0f0 2024-09-10T10:59:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.310+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560fd3cab0f0 0x560fd3cad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.311+0000 7f1f9b7fe640 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560fd3cab0f0 0x560fd3cad510 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d23076558f0a19 server_cookie=5b5eb5a5605596a9 in_seq=0 out_seq=0 2024-09-10T10:59:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.311+0000 7f1f7b7fe640 10 client.16329 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:11.881 INFO:tasks.workunit.client.0.smithi007.stdout: 10 5650 41.68 MB/sec warmup 11 sec latency 9621.969 ms 2024-09-10T10:59:11.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:11 smithi103 ceph-mon[35160]: pgmap v420: 97 pgs: 97 active+clean; 305 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 398 op/s 2024-09-10T10:59:11.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:11 smithi007 ceph-mon[30873]: pgmap v420: 97 pgs: 97 active+clean; 305 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 398 op/s 2024-09-10T10:59:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1f7b7fe640 1 -- 172.21.15.7:0/2855053492 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fd3b90d70 con 0x560fd3cab0f0 2024-09-10T10:59:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1f7b7fe640 10 client.16329 handle_command_reply: tid=0 2024-09-10T10:59:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 client.16329 shutdown 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 -- 172.21.15.7:0/2855053492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 msgr2=0x560fd3c37ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c37ca0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c02a940 tx=0x7f1f8c004820 comp rx=0 tx=0).stop 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 -- 172.21.15.7:0/2855053492 shutdown_connections 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x560fd3cab0f0 0x560fd3cad510 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560fd3c2fcb0 0x560fd3c32c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fd3c25410 0x560fd3c381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 --2- 172.21.15.7:0/2855053492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560fd3c24af0 0x560fd3c37ca0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 -- 172.21.15.7:0/2855053492 >> 172.21.15.7:0/2855053492 conn(0x560fd3c21060 msgr2=0x560fd3c26d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1f7b7fe640 0 client.16329 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 -- 172.21.15.7:0/2855053492 shutdown_connections 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 1 -- 172.21.15.7:0/2855053492 wait complete. 2024-09-10T10:59:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.937+0000 7f1fa55f9740 20 client.16329 trim_cache size 0 max 16384 2024-09-10T10:59:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:59:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:59:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:59:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:59:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 -- 172.21.15.7:0/1998435454 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1f840a2e20 msgr2=0x7f1f840a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1f840a2e20 0x7f1f840a52e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1f940094d0 tx=0x7f1f94004ab0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 -- 172.21.15.7:0/1998435454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c102480 msgr2=0x7f1f9c1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c102480 0x7f1f9c1988d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000e9f0 tx=0x7f1f9000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 -- 172.21.15.7:0/1998435454 shutdown_connections 2024-09-10T10:59:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f9c107b80 0x7f1f9c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f9c102480 0x7f1f9c1988d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f1f840a2e20 0x7f1f840a52e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 --2- 172.21.15.7:0/1998435454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c101ae0 0x7f1f9c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 -- 172.21.15.7:0/1998435454 >> 172.21.15.7:0/1998435454 conn(0x7f1f9c0fd7e0 msgr2=0x7f1f9c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 -- 172.21.15.7:0/1998435454 shutdown_connections 2024-09-10T10:59:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:11.938+0000 7f1fa285e640 1 -- 172.21.15.7:0/1998435454 wait complete. 2024-09-10T10:59:11.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:11 smithi133 ceph-mon[35282]: pgmap v420: 97 pgs: 97 active+clean; 305 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 398 op/s 2024-09-10T10:59:12.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 6220 41.84 MB/sec warmup 12 sec latency 10622.006 ms 2024-09-10T10:59:13.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 6644 41.81 MB/sec warmup 13 sec latency 11622.116 ms 2024-09-10T10:59:13.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:13 smithi103 ceph-mon[35160]: pgmap v421: 97 pgs: 97 active+clean; 305 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 383 op/s 2024-09-10T10:59:13.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:13 smithi007 ceph-mon[30873]: pgmap v421: 97 pgs: 97 active+clean; 305 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 383 op/s 2024-09-10T10:59:13.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:13 smithi133 ceph-mon[35282]: pgmap v421: 97 pgs: 97 active+clean; 305 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 383 op/s 2024-09-10T10:59:14.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 6831 39.02 MB/sec warmup 14 sec latency 12622.160 ms 2024-09-10T10:59:15.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 7078 36.57 MB/sec warmup 15 sec latency 13622.201 ms 2024-09-10T10:59:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:15 smithi103 ceph-mon[35160]: pgmap v422: 97 pgs: 97 active+clean; 305 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2024-09-10T10:59:15.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:15 smithi007 ceph-mon[30873]: pgmap v422: 97 pgs: 97 active+clean; 305 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2024-09-10T10:59:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:15 smithi133 ceph-mon[35282]: pgmap v422: 97 pgs: 97 active+clean; 305 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 318 op/s 2024-09-10T10:59:16.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 7544 35.46 MB/sec warmup 16 sec latency 14105.380 ms 2024-09-10T10:59:17.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 8435 35.66 MB/sec warmup 17 sec latency 128.751 ms 2024-09-10T10:59:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:17 smithi103 ceph-mon[35160]: pgmap v423: 97 pgs: 97 active+clean; 296 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 647 op/s 2024-09-10T10:59:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:17 smithi007 ceph-mon[30873]: pgmap v423: 97 pgs: 97 active+clean; 296 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 647 op/s 2024-09-10T10:59:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:17 smithi133 ceph-mon[35282]: pgmap v423: 97 pgs: 97 active+clean; 296 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 647 op/s 2024-09-10T10:59:18.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 8992 36.38 MB/sec warmup 18 sec latency 86.704 ms 2024-09-10T10:59:19.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 9747 37.70 MB/sec warmup 19 sec latency 95.462 ms 2024-09-10T10:59:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:19 smithi103 ceph-mon[35160]: pgmap v424: 97 pgs: 97 active+clean; 296 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 647 op/s 2024-09-10T10:59:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:19 smithi007 ceph-mon[30873]: pgmap v424: 97 pgs: 97 active+clean; 296 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 647 op/s 2024-09-10T10:59:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:19 smithi133 ceph-mon[35282]: pgmap v424: 97 pgs: 97 active+clean; 296 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 647 op/s 2024-09-10T10:59:20.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 10052 36.43 MB/sec warmup 20 sec latency 101.888 ms 2024-09-10T10:59:21.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 10298 34.93 MB/sec warmup 21 sec latency 80.853 ms 2024-09-10T10:59:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:21 smithi103 ceph-mon[35160]: pgmap v425: 97 pgs: 97 active+clean; 312 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.2 MiB/s wr, 727 op/s 2024-09-10T10:59:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:21 smithi007 ceph-mon[30873]: pgmap v425: 97 pgs: 97 active+clean; 312 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.2 MiB/s wr, 727 op/s 2024-09-10T10:59:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:21 smithi133 ceph-mon[35282]: pgmap v425: 97 pgs: 97 active+clean; 312 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.2 MiB/s wr, 727 op/s 2024-09-10T10:59:22.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 10581 33.67 MB/sec warmup 22 sec latency 189.255 ms 2024-09-10T10:59:23.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 11129 33.66 MB/sec warmup 23 sec latency 129.210 ms 2024-09-10T10:59:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:23 smithi103 ceph-mon[35160]: pgmap v426: 97 pgs: 97 active+clean; 312 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-09-10T10:59:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:23 smithi007 ceph-mon[30873]: pgmap v426: 97 pgs: 97 active+clean; 312 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-09-10T10:59:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:23 smithi133 ceph-mon[35282]: pgmap v426: 97 pgs: 97 active+clean; 312 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-09-10T10:59:24.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12357 34.43 MB/sec warmup 24 sec latency 59.942 ms 2024-09-10T10:59:24.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:24 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:24.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:24 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:24.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:24 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.037+0000 7f7c55311640 1 -- 172.21.15.7:0/1870427118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c501009f0 msgr2=0x7f7c50100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.037+0000 7f7c55311640 1 --2- 172.21.15.7:0/1870427118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c501009f0 0x7f7c50100e50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009930 tx=0x7f7c4402a420 comp rx=0 tx=0).stop 2024-09-10T10:59:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.037+0000 7f7c55311640 1 -- 172.21.15.7:0/1870427118 shutdown_connections 2024-09-10T10:59:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.037+0000 7f7c55311640 1 --2- 172.21.15.7:0/1870427118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c50101390 0x7f7c5010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.037+0000 7f7c55311640 1 --2- 172.21.15.7:0/1870427118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c501009f0 0x7f7c50100e50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.037+0000 7f7c55311640 1 --2- 172.21.15.7:0/1870427118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50106a00 0x7f7c50106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.037+0000 7f7c55311640 1 -- 172.21.15.7:0/1870427118 >> 172.21.15.7:0/1870427118 conn(0x7f7c500fc6b0 msgr2=0x7f7c500fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.037+0000 7f7c55311640 1 -- 172.21.15.7:0/1870427118 shutdown_connections 2024-09-10T10:59:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.037+0000 7f7c55311640 1 -- 172.21.15.7:0/1870427118 wait complete. 2024-09-10T10:59:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.038+0000 7f7c55311640 1 Processor -- start 2024-09-10T10:59:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.038+0000 7f7c55311640 1 -- start start 2024-09-10T10:59:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.038+0000 7f7c55311640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c501009f0 0x7f7c500fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.038+0000 7f7c55311640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c50101390 0x7f7c500fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.038+0000 7f7c55311640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50106a00 0x7f7c500fd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.038+0000 7f7c55311640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50110e40 con 0x7f7c50106a00 2024-09-10T10:59:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.038+0000 7f7c55311640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50110cc0 con 0x7f7c50101390 2024-09-10T10:59:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.038+0000 7f7c55311640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7c50110fc0 con 0x7f7c501009f0 2024-09-10T10:59:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50106a00 0x7f7c500fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50106a00 0x7f7c500fd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39688/0 (socket says 172.21.15.7:39688) 2024-09-10T10:59:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4f7fe640 1 -- 172.21.15.7:0/1299411223 learned_addr learned my addr 172.21.15.7:0/1299411223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4effd640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c501009f0 0x7f7c500fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c50101390 0x7f7c500fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4f7fe640 1 -- 172.21.15.7:0/1299411223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c501009f0 msgr2=0x7f7c500fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4f7fe640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c501009f0 0x7f7c500fcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4f7fe640 1 -- 172.21.15.7:0/1299411223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c50101390 msgr2=0x7f7c500fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4f7fe640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c50101390 0x7f7c500fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4f7fe640 1 -- 172.21.15.7:0/1299411223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c501a6820 con 0x7f7c50106a00 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4effd640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c501009f0 0x7f7c500fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c4f7fe640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50106a00 0x7f7c500fd720 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800eb10 tx=0x7f7c4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c37fff640 1 -- 172.21.15.7:0/1299411223 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48004b20 con 0x7f7c50106a00 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c55311640 1 -- 172.21.15.7:0/1299411223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c501a6b10 con 0x7f7c50106a00 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c37fff640 1 -- 172.21.15.7:0/1299411223 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c48004cc0 con 0x7f7c50106a00 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c37fff640 1 -- 172.21.15.7:0/1299411223 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48010760 con 0x7f7c50106a00 2024-09-10T10:59:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.039+0000 7f7c55311640 1 -- 172.21.15.7:0/1299411223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c501a6fe0 con 0x7f7c50106a00 2024-09-10T10:59:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.042+0000 7f7c37fff640 1 -- 172.21.15.7:0/1299411223 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7c48005020 con 0x7f7c50106a00 2024-09-10T10:59:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.042+0000 7f7c580ac740 1 Processor -- start 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.042+0000 7f7c580ac740 1 -- start start 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.042+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230b9af0 0x55cf230b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.042+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.042+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230c4cb0 0x55cf230c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.042+0000 7f7c580ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cf2301e2f0 con 0x55cf230ba410 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.042+0000 7f7c580ac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf22fc6270 con 0x55cf230b9af0 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.042+0000 7f7c580ac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cf2301a2b0 con 0x55cf230c4cb0 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.042+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c37fff640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7c380a2da0 0x7f7c380a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39692/0 (socket says 172.21.15.7:39692) 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4effd640 1 -- 172.21.15.7:0/1276796912 learned_addr learned my addr 172.21.15.7:0/1276796912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/1276796912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230b9af0 0x55cf230b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/1276796912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230c4cb0 0x55cf230c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4f7fe640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7c380a2da0 0x7f7c380a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4effd640 1 -- 172.21.15.7:0/1276796912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230c4cb0 msgr2=0x55cf230c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c37fff640 1 -- 172.21.15.7:0/1299411223 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f7c480cfa90 con 0x7f7c50106a00 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4effd640 1 --2- 172.21.15.7:0/1276796912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230c4cb0 0x55cf230c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4effd640 1 -- 172.21.15.7:0/1276796912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230b9af0 msgr2=0x55cf230b9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4effd640 1 --2- 172.21.15.7:0/1276796912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230b9af0 0x55cf230b9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4effd640 1 -- 172.21.15.7:0/1276796912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf22fc5340 con 0x55cf230ba410 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/1276796912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230b9af0 0x55cf230b9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4effd640 1 --2- 172.21.15.7:0/1276796912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c4770 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7c400114f0 tx=0x7f7c400119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebadbd0f87fb58dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c4f7fe640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7c380a2da0 0x7f7c380a5260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c48098380 tx=0x7f7c4800efd0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c357fa640 1 -- 172.21.15.7:0/1276796912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4001e070 con 0x55cf230ba410 2024-09-10T10:59:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c357fa640 1 -- 172.21.15.7:0/1276796912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c4000faa0 con 0x55cf230ba410 2024-09-10T10:59:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.043+0000 7f7c357fa640 1 -- 172.21.15.7:0/1276796912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40019b30 con 0x55cf230ba410 2024-09-10T10:59:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 -- 172.21.15.7:0/1276796912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 msgr2=0x55cf230c4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 --2- 172.21.15.7:0/1276796912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c4770 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7c400114f0 tx=0x7f7c400119b0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 -- 172.21.15.7:0/1276796912 shutdown_connections 2024-09-10T10:59:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 --2- 172.21.15.7:0/1276796912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230c4cb0 0x55cf230c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 --2- 172.21.15.7:0/1276796912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c4770 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 --2- 172.21.15.7:0/1276796912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230b9af0 0x55cf230b9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 -- 172.21.15.7:0/1276796912 >> 172.21.15.7:0/1276796912 conn(0x55cf230b6060 msgr2=0x55cf230b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 -- 172.21.15.7:0/1276796912 shutdown_connections 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 -- 172.21.15.7:0/1276796912 wait complete. 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 -- start start 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.044+0000 7f7c580ac740 1 Processor -- start 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230b9af0 0x55cf230c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 0x55cf230c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c580ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cf230c7880 con 0x55cf230ba410 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c580ac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cf22fa7290 con 0x55cf230c4cb0 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c580ac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cf230c7a00 con 0x55cf230b9af0 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c580ac740 10 client.0 _ll_register_callbacks cb 0x55cf22fd8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:59:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39694/0 (socket says 172.21.15.7:39694) 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4effd640 1 -- 172.21.15.7:0/2027926417 learned_addr learned my addr 172.21.15.7:0/2027926417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230b9af0 0x55cf230c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 0x55cf230c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4effd640 1 -- 172.21.15.7:0/2027926417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230b9af0 msgr2=0x55cf230c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4effd640 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230b9af0 0x55cf230c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4effd640 1 -- 172.21.15.7:0/2027926417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 msgr2=0x55cf230c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4effd640 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 0x55cf230c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4effd640 1 -- 172.21.15.7:0/2027926417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf22d31210 con 0x55cf230ba410 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c4effd640 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c2120 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7c4001c040 tx=0x7f7c40009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1123c0e4c00b446e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:59:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c32ff5640 1 -- 172.21.15.7:0/2027926417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4001e040 con 0x55cf230ba410 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c32ff5640 1 -- 172.21.15.7:0/2027926417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f7c4000f680 con 0x55cf230ba410 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.045+0000 7f7c580ac740 1 -- 172.21.15.7:0/2027926417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c4000faa0 con 0x55cf230ba410 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c32ff5640 1 -- 172.21.15.7:0/2027926417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7c4002f050 con 0x55cf230ba410 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c580ac740 10 client.16341 fetch_fsmap learned FSMap version 45 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c580ac740 1 -- 172.21.15.7:0/2027926417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cf22dfd620 con 0x55cf230ba410 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c32ff5640 1 -- 172.21.15.7:0/2027926417 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c4000faa0 con 0x55cf230ba410 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c32ff5640 1 -- 172.21.15.7:0/2027926417 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f7c4002a030 con 0x55cf230ba410 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c580ac740 10 client.16341 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c580ac740 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c580ac740 20 client.16341 populate_metadata read hostname 'smithi007' 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c580ac740 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cf230d4be0 0x55cf230d6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c580ac740 4 client.16341 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:59:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.046+0000 7f7c580ac740 1 -- 172.21.15.7:0/2027926417 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf2302ba20 con 0x55cf230d4be0 2024-09-10T10:59:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.087+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cf230d4be0 0x55cf230d6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.088+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cf230d4be0 0x55cf230d6fd0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27a0dc54c5773fa server_cookie=451b2a016059f4c5 in_seq=0 out_seq=0 2024-09-10T10:59:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.088+0000 7f7c32ff5640 10 client.16341 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c32ff5640 1 -- 172.21.15.7:0/2027926417 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf2302ba20 con 0x55cf230d4be0 2024-09-10T10:59:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c32ff5640 10 client.16341 handle_command_reply: tid=0 2024-09-10T10:59:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 client.16341 shutdown 2024-09-10T10:59:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 -- 172.21.15.7:0/2027926417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 msgr2=0x55cf230c2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c2120 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7c4001c040 tx=0x7f7c40009cc0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 -- 172.21.15.7:0/2027926417 shutdown_connections 2024-09-10T10:59:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cf230d4be0 0x55cf230d6fd0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 0x55cf230c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c32ff5640 0 client.16341 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230c2120 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 --2- 172.21.15.7:0/2027926417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230b9af0 0x55cf230c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 -- 172.21.15.7:0/2027926417 >> 172.21.15.7:0/2027926417 conn(0x55cf230b6060 msgr2=0x55cf230bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 -- 172.21.15.7:0/2027926417 shutdown_connections 2024-09-10T10:59:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 1 -- 172.21.15.7:0/2027926417 wait complete. 2024-09-10T10:59:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.092+0000 7f7c580ac740 20 client.16341 trim_cache size 0 max 16384 2024-09-10T10:59:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c580ac740 1 Processor -- start 2024-09-10T10:59:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c580ac740 1 -- start start 2024-09-10T10:59:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230b9af0 0x55cf230c0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230ba410 0x55cf230c2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230c4cb0 0x55cf230dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c580ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501b2340 con 0x55cf230b9af0 2024-09-10T10:59:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c580ac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501afbb0 con 0x55cf230ba410 2024-09-10T10:59:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c580ac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501ada20 con 0x55cf230c4cb0 2024-09-10T10:59:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c4e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230b9af0 0x55cf230c0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c4e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230b9af0 0x55cf230c0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39702/0 (socket says 172.21.15.7:39702) 2024-09-10T10:59:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c4e7fc640 1 -- 172.21.15.7:0/2419419076 learned_addr learned my addr 172.21.15.7:0/2419419076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c4effd640 1 --2- 172.21.15.7:0/2419419076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230ba410 0x55cf230c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.113+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/2419419076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230c4cb0 0x55cf230dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.114+0000 7f7c4e7fc640 1 -- 172.21.15.7:0/2419419076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230c4cb0 msgr2=0x55cf230dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.114+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/2419419076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230c4cb0 0x55cf230dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.114+0000 7f7c4e7fc640 1 -- 172.21.15.7:0/2419419076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230ba410 msgr2=0x55cf230c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.114+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/2419419076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230ba410 0x55cf230c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.114+0000 7f7c4e7fc640 1 -- 172.21.15.7:0/2419419076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf230ba240 con 0x55cf230b9af0 2024-09-10T10:59:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.114+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/2419419076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230b9af0 0x55cf230c0490 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009850 tx=0x7f7c44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3474d61cf395002 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.114+0000 7f7c34ff9640 1 -- 172.21.15.7:0/2419419076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44035040 con 0x55cf230b9af0 2024-09-10T10:59:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.114+0000 7f7c34ff9640 1 -- 172.21.15.7:0/2419419076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c44004c80 con 0x55cf230b9af0 2024-09-10T10:59:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.114+0000 7f7c34ff9640 1 -- 172.21.15.7:0/2419419076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44005440 con 0x55cf230b9af0 2024-09-10T10:59:25.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.115+0000 7f7c580ac740 1 -- 172.21.15.7:0/2419419076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230b9af0 msgr2=0x55cf230c0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.115+0000 7f7c580ac740 1 --2- 172.21.15.7:0/2419419076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230b9af0 0x55cf230c0490 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009850 tx=0x7f7c44000bc0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.115+0000 7f7c580ac740 1 -- 172.21.15.7:0/2419419076 shutdown_connections 2024-09-10T10:59:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.115+0000 7f7c580ac740 1 --2- 172.21.15.7:0/2419419076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230c4cb0 0x55cf230dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.115+0000 7f7c580ac740 1 --2- 172.21.15.7:0/2419419076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230ba410 0x55cf230c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.115+0000 7f7c580ac740 1 --2- 172.21.15.7:0/2419419076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230b9af0 0x55cf230c0490 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.115+0000 7f7c580ac740 1 -- 172.21.15.7:0/2419419076 >> 172.21.15.7:0/2419419076 conn(0x55cf230b6060 msgr2=0x55cf230c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.115+0000 7f7c580ac740 1 -- 172.21.15.7:0/2419419076 shutdown_connections 2024-09-10T10:59:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.115+0000 7f7c580ac740 1 -- 172.21.15.7:0/2419419076 wait complete. 2024-09-10T10:59:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c580ac740 1 -- start start 2024-09-10T10:59:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c580ac740 1 Processor -- start 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230b9af0 0x55cf230cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c580ac740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 0x55cf230be440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c580ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501ae070 con 0x55cf230ba410 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c580ac740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501af4e0 con 0x55cf230c4cb0 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c580ac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501ae740 con 0x55cf230b9af0 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c580ac740 10 client.0 _ll_register_callbacks cb 0x7f7c4001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c4e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 0x55cf230be440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c4effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39704/0 (socket says 172.21.15.7:39704) 2024-09-10T10:59:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.116+0000 7f7c4effd640 1 -- 172.21.15.7:0/288044221 learned_addr learned my addr 172.21.15.7:0/288044221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230b9af0 0x55cf230cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c4effd640 1 -- 172.21.15.7:0/288044221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230b9af0 msgr2=0x55cf230cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c4effd640 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230b9af0 0x55cf230cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c4effd640 1 -- 172.21.15.7:0/288044221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 msgr2=0x55cf230be440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c4effd640 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 0x55cf230be440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c4effd640 1 -- 172.21.15.7:0/288044221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf230bec40 con 0x55cf230ba410 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 0x55cf230be440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c4effd640 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230cd3c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7c40011eb0 tx=0x7f7c40003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8e82aeac11c605 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c32ff5640 1 -- 172.21.15.7:0/288044221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4001e070 con 0x55cf230ba410 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c32ff5640 1 -- 172.21.15.7:0/288044221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f7c40003d60 con 0x55cf230ba410 2024-09-10T10:59:25.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c580ac740 1 -- 172.21.15.7:0/288044221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c44004c80 con 0x55cf230ba410 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c32ff5640 1 -- 172.21.15.7:0/288044221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7c40041050 con 0x55cf230ba410 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c580ac740 10 client.16347 fetch_fsmap learned FSMap version 45 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c580ac740 1 -- 172.21.15.7:0/288044221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cf2313f240 con 0x55cf230ba410 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.117+0000 7f7c32ff5640 1 -- 172.21.15.7:0/288044221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c44004c80 con 0x55cf230ba410 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.118+0000 7f7c32ff5640 1 -- 172.21.15.7:0/288044221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f7c4002a040 con 0x55cf230ba410 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.118+0000 7f7c580ac740 10 client.16347 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.118+0000 7f7c580ac740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.118+0000 7f7c580ac740 20 client.16347 populate_metadata read hostname 'smithi007' 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.118+0000 7f7c580ac740 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cf2313f890 0x55cf23141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.118+0000 7f7c580ac740 4 client.16347 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.118+0000 7f7c580ac740 1 -- 172.21.15.7:0/288044221 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf23025d70 con 0x55cf2313f890 2024-09-10T10:59:25.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.118+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cf2313f890 0x55cf23141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.118+0000 7f7c32ff5640 10 client.16347 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:25.125 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.118+0000 7f7c4e7fc640 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cf2313f890 0x55cf23141cb0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfca75860ad14a1 server_cookie=709d2a31fa078436 in_seq=0 out_seq=0 2024-09-10T10:59:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.502+0000 7f7c32ff5640 1 -- 172.21.15.7:0/288044221 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf23025d70 con 0x55cf2313f890 2024-09-10T10:59:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c32ff5640 10 client.16347 handle_command_reply: tid=0 2024-09-10T10:59:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 client.16347 shutdown 2024-09-10T10:59:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 -- 172.21.15.7:0/288044221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 msgr2=0x55cf230cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230cd3c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7c40011eb0 tx=0x7f7c40003970 comp rx=0 tx=0).stop 2024-09-10T10:59:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 -- 172.21.15.7:0/288044221 shutdown_connections 2024-09-10T10:59:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cf2313f890 0x55cf23141cb0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cf230c4cb0 0x55cf230be440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf230ba410 0x55cf230cd3c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 --2- 172.21.15.7:0/288044221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf230b9af0 0x55cf230cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 -- 172.21.15.7:0/288044221 >> 172.21.15.7:0/288044221 conn(0x55cf230b6060 msgr2=0x55cf230bb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c32ff5640 0 client.16347 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 -- 172.21.15.7:0/288044221 shutdown_connections 2024-09-10T10:59:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 1 -- 172.21.15.7:0/288044221 wait complete. 2024-09-10T10:59:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.503+0000 7f7c580ac740 20 client.16347 trim_cache size 0 max 16384 2024-09-10T10:59:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:59:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:59:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:59:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:59:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 -- 172.21.15.7:0/1299411223 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7c380a2da0 msgr2=0x7f7c380a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7c380a2da0 0x7f7c380a5260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c48098380 tx=0x7f7c4800efd0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 -- 172.21.15.7:0/1299411223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50106a00 msgr2=0x7f7c500fd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50106a00 0x7f7c500fd720 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800eb10 tx=0x7f7c4800c2d0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 -- 172.21.15.7:0/1299411223 shutdown_connections 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f7c380a2da0 0x7f7c380a5260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c50106a00 0x7f7c500fd720 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c50101390 0x7f7c500fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 --2- 172.21.15.7:0/1299411223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c501009f0 0x7f7c500fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 -- 172.21.15.7:0/1299411223 >> 172.21.15.7:0/1299411223 conn(0x7f7c500fc6b0 msgr2=0x7f7c5006a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 -- 172.21.15.7:0/1299411223 shutdown_connections 2024-09-10T10:59:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:25.504+0000 7f7c55311640 1 -- 172.21.15.7:0/1299411223 wait complete. 2024-09-10T10:59:25.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 12934 35.10 MB/sec warmup 25 sec latency 244.206 ms 2024-09-10T10:59:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:25 smithi103 ceph-mon[35160]: pgmap v427: 97 pgs: 97 active+clean; 312 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-09-10T10:59:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:25 smithi007 ceph-mon[30873]: pgmap v427: 97 pgs: 97 active+clean; 312 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-09-10T10:59:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:25 smithi133 ceph-mon[35282]: pgmap v427: 97 pgs: 97 active+clean; 312 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-09-10T10:59:26.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 13695 35.55 MB/sec warmup 26 sec latency 88.976 ms 2024-09-10T10:59:27.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 14324 35.13 MB/sec warmup 27 sec latency 90.606 ms 2024-09-10T10:59:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:27 smithi103 ceph-mon[35160]: pgmap v428: 97 pgs: 97 active+clean; 322 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 705 op/s 2024-09-10T10:59:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:27 smithi007 ceph-mon[30873]: pgmap v428: 97 pgs: 97 active+clean; 322 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 705 op/s 2024-09-10T10:59:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:27 smithi133 ceph-mon[35282]: pgmap v428: 97 pgs: 97 active+clean; 322 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.1 MiB/s wr, 705 op/s 2024-09-10T10:59:28.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 14916 34.97 MB/sec warmup 28 sec latency 119.340 ms 2024-09-10T10:59:29.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 16154 35.49 MB/sec warmup 29 sec latency 131.113 ms 2024-09-10T10:59:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:29 smithi103 ceph-mon[35160]: pgmap v429: 97 pgs: 97 active+clean; 322 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 296 op/s 2024-09-10T10:59:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:29 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:29 smithi007 ceph-mon[30873]: pgmap v429: 97 pgs: 97 active+clean; 322 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 296 op/s 2024-09-10T10:59:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:29 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:29 smithi133 ceph-mon[35282]: pgmap v429: 97 pgs: 97 active+clean; 322 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 296 op/s 2024-09-10T10:59:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:29 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T10:59:30.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 17647 37.82 MB/sec warmup 30 sec latency 37.667 ms 2024-09-10T10:59:31.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 18338 37.52 MB/sec warmup 31 sec latency 63.067 ms 2024-09-10T10:59:31.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:31 smithi103 ceph-mon[35160]: pgmap v430: 97 pgs: 97 active+clean; 328 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 712 op/s 2024-09-10T10:59:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:31 smithi007 ceph-mon[30873]: pgmap v430: 97 pgs: 97 active+clean; 328 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 712 op/s 2024-09-10T10:59:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:31 smithi133 ceph-mon[35282]: pgmap v430: 97 pgs: 97 active+clean; 328 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.3 MiB/s wr, 712 op/s 2024-09-10T10:59:32.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 19280 37.85 MB/sec warmup 32 sec latency 62.482 ms 2024-09-10T10:59:33.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 20253 37.92 MB/sec warmup 33 sec latency 316.085 ms 2024-09-10T10:59:33.943 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:33 smithi007 ceph-mon[30873]: pgmap v431: 97 pgs: 97 active+clean; 328 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-10T10:59:33.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:33 smithi133 ceph-mon[35282]: pgmap v431: 97 pgs: 97 active+clean; 328 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-10T10:59:34.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:33 smithi103 ceph-mon[35160]: pgmap v431: 97 pgs: 97 active+clean; 328 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-10T10:59:34.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 21341 39.28 MB/sec warmup 34 sec latency 35.022 ms 2024-09-10T10:59:35.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 21975 38.71 MB/sec warmup 35 sec latency 87.847 ms 2024-09-10T10:59:35.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:35 smithi007 ceph-mon[30873]: pgmap v432: 97 pgs: 97 active+clean; 328 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-10T10:59:35.973 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:35 smithi103 ceph-mon[35160]: pgmap v432: 97 pgs: 97 active+clean; 328 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-10T10:59:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:35 smithi133 ceph-mon[35282]: pgmap v432: 97 pgs: 97 active+clean; 328 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-10T10:59:36.882 INFO:tasks.workunit.client.0.smithi007.stdout: 10 23363 39.48 MB/sec warmup 36 sec latency 51.800 ms 2024-09-10T10:59:37.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 24561 40.63 MB/sec warmup 37 sec latency 65.483 ms 2024-09-10T10:59:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:37 smithi007 ceph-mon[30873]: pgmap v433: 97 pgs: 97 active+clean; 534 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 23 MiB/s wr, 847 op/s 2024-09-10T10:59:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:37 smithi133 ceph-mon[35282]: pgmap v433: 97 pgs: 97 active+clean; 534 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 23 MiB/s wr, 847 op/s 2024-09-10T10:59:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:37 smithi103 ceph-mon[35160]: pgmap v433: 97 pgs: 97 active+clean; 534 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 23 MiB/s wr, 847 op/s 2024-09-10T10:59:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.655+0000 7f6d78ff8640 1 -- 172.21.15.7:0/3311052769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d741069e0 msgr2=0x7f6d74106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.655+0000 7f6d78ff8640 1 --2- 172.21.15.7:0/3311052769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d741069e0 0x7f6d74106dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009930 tx=0x7f6d6402a440 comp rx=0 tx=0).stop 2024-09-10T10:59:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.655+0000 7f6d78ff8640 1 -- 172.21.15.7:0/3311052769 shutdown_connections 2024-09-10T10:59:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.655+0000 7f6d78ff8640 1 --2- 172.21.15.7:0/3311052769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d74101370 0x7f6d7410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.655+0000 7f6d78ff8640 1 --2- 172.21.15.7:0/3311052769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d741009d0 0x7f6d74100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.655+0000 7f6d78ff8640 1 --2- 172.21.15.7:0/3311052769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d741069e0 0x7f6d74106dc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.655+0000 7f6d78ff8640 1 -- 172.21.15.7:0/3311052769 >> 172.21.15.7:0/3311052769 conn(0x7f6d740fc6f0 msgr2=0x7f6d740feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.660+0000 7f6d78ff8640 1 -- 172.21.15.7:0/3311052769 shutdown_connections 2024-09-10T10:59:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.660+0000 7f6d78ff8640 1 -- 172.21.15.7:0/3311052769 wait complete. 2024-09-10T10:59:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.661+0000 7f6d78ff8640 1 Processor -- start 2024-09-10T10:59:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.661+0000 7f6d78ff8640 1 -- start start 2024-09-10T10:59:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.661+0000 7f6d78ff8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d741009d0 0x7f6d74116330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.661+0000 7f6d78ff8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d74101370 0x7f6d741103a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.661+0000 7f6d78ff8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d741069e0 0x7f6d741108e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.661+0000 7f6d78ff8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7410eea0 con 0x7f6d74101370 2024-09-10T10:59:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.661+0000 7f6d78ff8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7410ed20 con 0x7f6d741069e0 2024-09-10T10:59:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.661+0000 7f6d78ff8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7410f020 con 0x7f6d741009d0 2024-09-10T10:59:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.662+0000 7f6d71d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d74101370 0x7f6d741103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.662+0000 7f6d71d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d74101370 0x7f6d741103a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46202/0 (socket says 172.21.15.7:46202) 2024-09-10T10:59:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.662+0000 7f6d71d9b640 1 -- 172.21.15.7:0/2733395453 learned_addr learned my addr 172.21.15.7:0/2733395453 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.662+0000 7f6d72d9d640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d741069e0 0x7f6d741108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.662+0000 7f6d7259c640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d741009d0 0x7f6d74116330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.663+0000 7f6d72d9d640 1 -- 172.21.15.7:0/2733395453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d741009d0 msgr2=0x7f6d74116330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.663+0000 7f6d72d9d640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d741009d0 0x7f6d74116330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.663+0000 7f6d72d9d640 1 -- 172.21.15.7:0/2733395453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d74101370 msgr2=0x7f6d741103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.663+0000 7f6d72d9d640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d74101370 0x7f6d741103a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.663+0000 7f6d72d9d640 1 -- 172.21.15.7:0/2733395453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d74111010 con 0x7f6d741069e0 2024-09-10T10:59:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.666+0000 7f6d72d9d640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d741069e0 0x7f6d741108e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009fd0 tx=0x7f6d6c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.666+0000 7f6d637fe640 1 -- 172.21.15.7:0/2733395453 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00ce20 con 0x7f6d741069e0 2024-09-10T10:59:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.666+0000 7f6d637fe640 1 -- 172.21.15.7:0/2733395453 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c005020 con 0x7f6d741069e0 2024-09-10T10:59:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.666+0000 7f6d637fe640 1 -- 172.21.15.7:0/2733395453 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c021730 con 0x7f6d741069e0 2024-09-10T10:59:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.666+0000 7f6d78ff8640 1 -- 172.21.15.7:0/2733395453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7406bc10 con 0x7f6d741069e0 2024-09-10T10:59:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.667+0000 7f6d78ff8640 1 -- 172.21.15.7:0/2733395453 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7406c110 con 0x7f6d741069e0 2024-09-10T10:59:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.671+0000 7f6d7be68740 1 Processor -- start 2024-09-10T10:59:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.674+0000 7f6d637fe640 1 -- 172.21.15.7:0/2733395453 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6d6c004a40 con 0x7f6d741069e0 2024-09-10T10:59:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.674+0000 7f6d7be68740 1 -- start start 2024-09-10T10:59:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.675+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d3754af0 0x5570d3754ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.675+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d3755410 0x5570d375f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.675+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d375fcb0 0x5570d37620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.675+0000 7f6d7be68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5570d36b92f0 con 0x5570d3755410 2024-09-10T10:59:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.675+0000 7f6d7be68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570d3661270 con 0x5570d375fcb0 2024-09-10T10:59:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.675+0000 7f6d7be68740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5570d36b52b0 con 0x5570d3754af0 2024-09-10T10:59:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.676+0000 7f6d637fe640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6d5c0a2fd0 0x7f6d5c0a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.676+0000 7f6d637fe640 1 -- 172.21.15.7:0/2733395453 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f6d6c0e09a0 con 0x7f6d741069e0 2024-09-10T10:59:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.677+0000 7f6d7259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d3755410 0x5570d375f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.677+0000 7f6d7259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d3755410 0x5570d375f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46208/0 (socket says 172.21.15.7:46208) 2024-09-10T10:59:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.677+0000 7f6d7259c640 1 -- 172.21.15.7:0/1985687684 learned_addr learned my addr 172.21.15.7:0/1985687684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.677+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/1985687684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d375fcb0 0x5570d37620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.677+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/1985687684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d3754af0 0x5570d3754ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.677+0000 7f6d7259c640 1 -- 172.21.15.7:0/1985687684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d3754af0 msgr2=0x5570d3754ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.677+0000 7f6d7259c640 1 --2- 172.21.15.7:0/1985687684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d3754af0 0x5570d3754ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.677+0000 7f6d7259c640 1 -- 172.21.15.7:0/1985687684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d375fcb0 msgr2=0x5570d37620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.677+0000 7f6d7259c640 1 --2- 172.21.15.7:0/1985687684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d375fcb0 0x5570d37620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.677+0000 7f6d7259c640 1 -- 172.21.15.7:0/1985687684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d3660340 con 0x5570d3755410 2024-09-10T10:59:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.678+0000 7f6d7259c640 1 --2- 172.21.15.7:0/1985687684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d3755410 0x5570d375f770 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6d64000c00 tx=0x7f6d640314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42efd90715649c48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.678+0000 7f6d72d9d640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6d5c0a2fd0 0x7f6d5c0a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.679+0000 7f6d60ff9640 1 -- 172.21.15.7:0/1985687684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64035040 con 0x5570d3755410 2024-09-10T10:59:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.679+0000 7f6d60ff9640 1 -- 172.21.15.7:0/1985687684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d640043c0 con 0x5570d3755410 2024-09-10T10:59:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.680+0000 7f6d7be68740 1 -- 172.21.15.7:0/1985687684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d3755410 msgr2=0x5570d375f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.680+0000 7f6d7be68740 1 --2- 172.21.15.7:0/1985687684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d3755410 0x5570d375f770 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6d64000c00 tx=0x7f6d640314d0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.680+0000 7f6d7be68740 1 -- 172.21.15.7:0/1985687684 shutdown_connections 2024-09-10T10:59:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.680+0000 7f6d7be68740 1 --2- 172.21.15.7:0/1985687684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d375fcb0 0x5570d37620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.680+0000 7f6d7be68740 1 --2- 172.21.15.7:0/1985687684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d3755410 0x5570d375f770 secure :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6d64000c00 tx=0x7f6d640314d0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.680+0000 7f6d7be68740 1 --2- 172.21.15.7:0/1985687684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d3754af0 0x5570d3754ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.680+0000 7f6d7be68740 1 -- 172.21.15.7:0/1985687684 >> 172.21.15.7:0/1985687684 conn(0x5570d3751060 msgr2=0x5570d3751440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.681+0000 7f6d72d9d640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6d5c0a2fd0 0x7f6d5c0a5490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0b5570 tx=0x7f6d6c005f70 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.681+0000 7f6d7be68740 1 -- 172.21.15.7:0/1985687684 shutdown_connections 2024-09-10T10:59:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.681+0000 7f6d7be68740 1 -- 172.21.15.7:0/1985687684 wait complete. 2024-09-10T10:59:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.681+0000 7f6d7be68740 1 -- start start 2024-09-10T10:59:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d7be68740 1 Processor -- start 2024-09-10T10:59:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 0x5570d375cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d375d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375d810 0x5570d376be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d7be68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5570d37628d0 con 0x5570d375fcb0 2024-09-10T10:59:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d7be68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570d3642290 con 0x5570d3754af0 2024-09-10T10:59:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d7be68740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5570d3762a50 con 0x5570d375d810 2024-09-10T10:59:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d7be68740 10 client.0 _ll_register_callbacks cb 0x5570d3673140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:59:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 0x5570d375cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d7259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d375d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 0x5570d375cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:53022/0 (socket says 172.21.15.7:53022) 2024-09-10T10:59:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d71d9b640 1 -- 172.21.15.7:0/4205400204 learned_addr learned my addr 172.21.15.7:0/4205400204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.682+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375d810 0x5570d376be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d71d9b640 1 -- 172.21.15.7:0/4205400204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375d810 msgr2=0x5570d376be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375d810 0x5570d376be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d71d9b640 1 -- 172.21.15.7:0/4205400204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 msgr2=0x5570d375d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d375d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d71d9b640 1 -- 172.21.15.7:0/4205400204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d33cc210 con 0x5570d3754af0 2024-09-10T10:59:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d7259c640 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d375d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 0x5570d375cd90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d68013e80 tx=0x7f6d68009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a1daeec1d40666 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T10:59:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d59ffb640 1 -- 172.21.15.7:0/4205400204 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6801c040 con 0x5570d3754af0 2024-09-10T10:59:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d59ffb640 1 -- 172.21.15.7:0/4205400204 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f6d680132d0 con 0x5570d3754af0 2024-09-10T10:59:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d7be68740 1 -- 172.21.15.7:0/4205400204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d640043c0 con 0x5570d3754af0 2024-09-10T10:59:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d59ffb640 1 -- 172.21.15.7:0/4205400204 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6d68004030 con 0x5570d3754af0 2024-09-10T10:59:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d7be68740 10 client.25970 fetch_fsmap learned FSMap version 46 2024-09-10T10:59:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.683+0000 7f6d7be68740 1 -- 172.21.15.7:0/4205400204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5570d3498620 con 0x5570d3754af0 2024-09-10T10:59:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.684+0000 7f6d59ffb640 1 -- 172.21.15.7:0/4205400204 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d640043c0 con 0x5570d3754af0 2024-09-10T10:59:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.684+0000 7f6d59ffb640 1 -- 172.21.15.7:0/4205400204 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f6d68011440 con 0x5570d3754af0 2024-09-10T10:59:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.684+0000 7f6d7be68740 10 client.25970 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T10:59:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.684+0000 7f6d7be68740 10 client.25970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:59:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.684+0000 7f6d7be68740 20 client.25970 populate_metadata read hostname 'smithi007' 2024-09-10T10:59:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.684+0000 7f6d7be68740 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5570d3770360 0x5570d37727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.684+0000 7f6d7be68740 4 client.25970 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:59:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.684+0000 7f6d7be68740 1 -- 172.21.15.7:0/4205400204 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570d36c6a20 con 0x5570d3770360 2024-09-10T10:59:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.684+0000 7f6d7259c640 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5570d3770360 0x5570d37727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.686+0000 7f6d7259c640 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5570d3770360 0x5570d37727e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b11462849e56f09 server_cookie=b06b2321d1f656a in_seq=0 out_seq=0 2024-09-10T10:59:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.686+0000 7f6d59ffb640 10 client.25970 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d59ffb640 1 -- 172.21.15.7:0/4205400204 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570d36c6a20 con 0x5570d3770360 2024-09-10T10:59:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d59ffb640 10 client.25970 handle_command_reply: tid=0 2024-09-10T10:59:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 client.25970 shutdown 2024-09-10T10:59:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 -- 172.21.15.7:0/4205400204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 msgr2=0x5570d375cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 0x5570d375cd90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6d68013e80 tx=0x7f6d68009be0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 -- 172.21.15.7:0/4205400204 shutdown_connections 2024-09-10T10:59:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5570d3770360 0x5570d37727e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375d810 0x5570d376be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d375d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 --2- 172.21.15.7:0/4205400204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 0x5570d375cd90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d59ffb640 0 client.25970 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 -- 172.21.15.7:0/4205400204 >> 172.21.15.7:0/4205400204 conn(0x5570d3751060 msgr2=0x5570d3756dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 -- 172.21.15.7:0/4205400204 shutdown_connections 2024-09-10T10:59:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 1 -- 172.21.15.7:0/4205400204 wait complete. 2024-09-10T10:59:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.688+0000 7f6d7be68740 20 client.25970 trim_cache size 0 max 16384 2024-09-10T10:59:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d7be68740 1 Processor -- start 2024-09-10T10:59:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d7be68740 1 -- start start 2024-09-10T10:59:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d3754af0 0x5570d375b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d375d810 0x5570d376bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d3778100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d7be68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54011150 con 0x5570d375fcb0 2024-09-10T10:59:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d7be68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5400e990 con 0x5570d375d810 2024-09-10T10:59:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d7be68740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5400c800 con 0x5570d3754af0 2024-09-10T10:59:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d71d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d3754af0 0x5570d375b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d71d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d3778100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d71d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d3778100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46236/0 (socket says 172.21.15.7:46236) 2024-09-10T10:59:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d71d9b640 1 -- 172.21.15.7:0/2271255767 learned_addr learned my addr 172.21.15.7:0/2271255767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.709+0000 7f6d7259c640 1 --2- 172.21.15.7:0/2271255767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d375d810 0x5570d376bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.710+0000 7f6d71d9b640 1 -- 172.21.15.7:0/2271255767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d3754af0 msgr2=0x5570d375b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.710+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/2271255767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d3754af0 0x5570d375b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.710+0000 7f6d71d9b640 1 -- 172.21.15.7:0/2271255767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d375d810 msgr2=0x5570d376bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.710+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/2271255767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d375d810 0x5570d376bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.710+0000 7f6d71d9b640 1 -- 172.21.15.7:0/2271255767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d375b2c0 con 0x5570d375fcb0 2024-09-10T10:59:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.710+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/2271255767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d3778100 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6d680092e0 tx=0x7f6d68000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dc40a4eec0e3f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.710+0000 7f6d5bfff640 1 -- 172.21.15.7:0/2271255767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68011640 con 0x5570d375fcb0 2024-09-10T10:59:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.710+0000 7f6d5bfff640 1 -- 172.21.15.7:0/2271255767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d680130d0 con 0x5570d375fcb0 2024-09-10T10:59:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.710+0000 7f6d5bfff640 1 -- 172.21.15.7:0/2271255767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68003850 con 0x5570d375fcb0 2024-09-10T10:59:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.711+0000 7f6d7be68740 1 -- 172.21.15.7:0/2271255767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 msgr2=0x5570d3778100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.711+0000 7f6d7be68740 1 --2- 172.21.15.7:0/2271255767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d3778100 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6d680092e0 tx=0x7f6d68000bc0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.711+0000 7f6d7be68740 1 -- 172.21.15.7:0/2271255767 shutdown_connections 2024-09-10T10:59:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.711+0000 7f6d7be68740 1 --2- 172.21.15.7:0/2271255767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375fcb0 0x5570d3778100 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.711+0000 7f6d7be68740 1 --2- 172.21.15.7:0/2271255767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d375d810 0x5570d376bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.711+0000 7f6d7be68740 1 --2- 172.21.15.7:0/2271255767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d3754af0 0x5570d375b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.711+0000 7f6d7be68740 1 -- 172.21.15.7:0/2271255767 >> 172.21.15.7:0/2271255767 conn(0x5570d3751060 msgr2=0x5570d375ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.711+0000 7f6d7be68740 1 -- 172.21.15.7:0/2271255767 shutdown_connections 2024-09-10T10:59:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.711+0000 7f6d7be68740 1 -- 172.21.15.7:0/2271255767 wait complete. 2024-09-10T10:59:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.711+0000 7f6d7be68740 1 -- start start 2024-09-10T10:59:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7be68740 1 Processor -- start 2024-09-10T10:59:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 0x5570d3768200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375d810 0x5570d3763520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7be68740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375fcb0 0x5570d3763a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7be68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5400ce50 con 0x5570d375d810 2024-09-10T10:59:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7be68740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5400e2c0 con 0x5570d3754af0 2024-09-10T10:59:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7be68740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5400d520 con 0x5570d375fcb0 2024-09-10T10:59:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7be68740 10 client.0 _ll_register_callbacks cb 0x7f6d68009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:59:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375d810 0x5570d3763520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d71d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 0x5570d3768200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375d810 0x5570d3763520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46246/0 (socket says 172.21.15.7:46246) 2024-09-10T10:59:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7259c640 1 -- 172.21.15.7:0/2953891739 learned_addr learned my addr 172.21.15.7:0/2953891739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375fcb0 0x5570d3763a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7259c640 1 -- 172.21.15.7:0/2953891739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375fcb0 msgr2=0x5570d3763a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7259c640 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375fcb0 0x5570d3763a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.712+0000 7f6d7259c640 1 -- 172.21.15.7:0/2953891739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 msgr2=0x5570d3768200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d7259c640 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 0x5570d3768200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d7259c640 1 -- 172.21.15.7:0/2953891739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d3763fa0 con 0x5570d375d810 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375fcb0 0x5570d3763a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d7259c640 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375d810 0x5570d3763520 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005ab0 tx=0x7f6d6402aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f4377f67005ba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d59ffb640 1 -- 172.21.15.7:0/2953891739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64054070 con 0x5570d375d810 2024-09-10T10:59:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d59ffb640 1 -- 172.21.15.7:0/2953891739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f6d64004140 con 0x5570d375d810 2024-09-10T10:59:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d7be68740 1 -- 172.21.15.7:0/2953891739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d680130d0 con 0x5570d375d810 2024-09-10T10:59:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d59ffb640 1 -- 172.21.15.7:0/2953891739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6d64035040 con 0x5570d375d810 2024-09-10T10:59:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d7be68740 10 client.16365 fetch_fsmap learned FSMap version 46 2024-09-10T10:59:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.713+0000 7f6d7be68740 1 -- 172.21.15.7:0/2953891739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5570d37dae60 con 0x5570d375d810 2024-09-10T10:59:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.714+0000 7f6d59ffb640 1 -- 172.21.15.7:0/2953891739 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d64004360 con 0x5570d375d810 2024-09-10T10:59:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.714+0000 7f6d59ffb640 1 -- 172.21.15.7:0/2953891739 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f6d64060070 con 0x5570d375d810 2024-09-10T10:59:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.714+0000 7f6d7be68740 10 client.16365 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T10:59:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.714+0000 7f6d7be68740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:59:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.714+0000 7f6d7be68740 20 client.16365 populate_metadata read hostname 'smithi007' 2024-09-10T10:59:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.714+0000 7f6d7be68740 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5570d37db150 0x5570d37dd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.715+0000 7f6d7be68740 4 client.16365 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:59:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.715+0000 7f6d7be68740 1 -- 172.21.15.7:0/2953891739 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6d680130d0 con 0x5570d37db150 2024-09-10T10:59:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.715+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5570d37db150 0x5570d37dd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.715+0000 7f6d59ffb640 10 client.16365 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:38.715+0000 7f6d71d9b640 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5570d37db150 0x5570d37dd540 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3bcda961875bd82 server_cookie=32d552e95b182b70 in_seq=0 out_seq=0 2024-09-10T10:59:38.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 25157 40.43 MB/sec warmup 38 sec latency 207.693 ms 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.250+0000 7f6d59ffb640 1 -- 172.21.15.7:0/2953891739 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6d680130d0 con 0x5570d37db150 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.250+0000 7f6d59ffb640 10 client.16365 handle_command_reply: tid=0 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.250+0000 7f6d7be68740 1 client.16365 shutdown 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.250+0000 7f6d7be68740 1 -- 172.21.15.7:0/2953891739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375d810 msgr2=0x5570d3763520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.250+0000 7f6d7be68740 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375d810 0x5570d3763520 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005ab0 tx=0x7f6d6402aa80 comp rx=0 tx=0).stop 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.250+0000 7f6d7be68740 1 -- 172.21.15.7:0/2953891739 shutdown_connections 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.250+0000 7f6d7be68740 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5570d37db150 0x5570d37dd540 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.250+0000 7f6d7be68740 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5570d375fcb0 0x5570d3763a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.251+0000 7f6d59ffb640 0 client.16365 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.251+0000 7f6d7be68740 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570d375d810 0x5570d3763520 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.251+0000 7f6d7be68740 1 --2- 172.21.15.7:0/2953891739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570d3754af0 0x5570d3768200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.251+0000 7f6d7be68740 1 -- 172.21.15.7:0/2953891739 >> 172.21.15.7:0/2953891739 conn(0x5570d3751060 msgr2=0x5570d3756550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.251+0000 7f6d7be68740 1 -- 172.21.15.7:0/2953891739 shutdown_connections 2024-09-10T10:59:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.251+0000 7f6d7be68740 1 -- 172.21.15.7:0/2953891739 wait complete. 2024-09-10T10:59:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.251+0000 7f6d7be68740 20 client.16365 trim_cache size 0 max 16384 2024-09-10T10:59:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:59:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:59:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:59:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:59:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 -- 172.21.15.7:0/2733395453 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6d5c0a2fd0 msgr2=0x7f6d5c0a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6d5c0a2fd0 0x7f6d5c0a5490 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0b5570 tx=0x7f6d6c005f70 comp rx=0 tx=0).stop 2024-09-10T10:59:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 -- 172.21.15.7:0/2733395453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d741069e0 msgr2=0x7f6d741108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d741069e0 0x7f6d741108e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009fd0 tx=0x7f6d6c00ef10 comp rx=0 tx=0).stop 2024-09-10T10:59:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 -- 172.21.15.7:0/2733395453 shutdown_connections 2024-09-10T10:59:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d741069e0 0x7f6d741108e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6d5c0a2fd0 0x7f6d5c0a5490 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d74101370 0x7f6d741103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 --2- 172.21.15.7:0/2733395453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d741009d0 0x7f6d74116330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 -- 172.21.15.7:0/2733395453 >> 172.21.15.7:0/2733395453 conn(0x7f6d740fc6f0 msgr2=0x7f6d740fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 -- 172.21.15.7:0/2733395453 shutdown_connections 2024-09-10T10:59:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:39.252+0000 7f6d78ff8640 1 -- 172.21.15.7:0/2733395453 wait complete. 2024-09-10T10:59:39.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 25837 39.98 MB/sec warmup 39 sec latency 334.840 ms 2024-09-10T10:59:39.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:39 smithi007 ceph-mon[30873]: pgmap v434: 97 pgs: 97 active+clean; 534 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 21 MiB/s wr, 701 op/s 2024-09-10T10:59:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:39 smithi133 ceph-mon[35282]: pgmap v434: 97 pgs: 97 active+clean; 534 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 21 MiB/s wr, 701 op/s 2024-09-10T10:59:40.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:39 smithi103 ceph-mon[35160]: pgmap v434: 97 pgs: 97 active+clean; 534 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 21 MiB/s wr, 701 op/s 2024-09-10T10:59:40.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 27026 40.82 MB/sec warmup 40 sec latency 39.939 ms 2024-09-10T10:59:41.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 27963 41.06 MB/sec warmup 41 sec latency 59.921 ms 2024-09-10T10:59:41.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:41 smithi007 ceph-mon[30873]: pgmap v435: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 26 MiB/s wr, 1.16k op/s 2024-09-10T10:59:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:41 smithi133 ceph-mon[35282]: pgmap v435: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 26 MiB/s wr, 1.16k op/s 2024-09-10T10:59:42.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:41 smithi103 ceph-mon[35160]: pgmap v435: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 26 MiB/s wr, 1.16k op/s 2024-09-10T10:59:42.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 28712 41.04 MB/sec warmup 42 sec latency 231.702 ms 2024-09-10T10:59:43.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 29558 41.09 MB/sec warmup 43 sec latency 161.459 ms 2024-09-10T10:59:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:43 smithi133 ceph-mon[35282]: pgmap v436: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 747 op/s 2024-09-10T10:59:44.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:43 smithi103 ceph-mon[35160]: pgmap v436: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 747 op/s 2024-09-10T10:59:44.339 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:43 smithi007 ceph-mon[30873]: pgmap v436: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 747 op/s 2024-09-10T10:59:44.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 30345 41.11 MB/sec warmup 44 sec latency 364.117 ms 2024-09-10T10:59:45.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 31376 41.26 MB/sec warmup 45 sec latency 68.234 ms 2024-09-10T10:59:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:45 smithi133 ceph-mon[35282]: pgmap v437: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 747 op/s 2024-09-10T10:59:46.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:45 smithi103 ceph-mon[35160]: pgmap v437: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 747 op/s 2024-09-10T10:59:46.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:45 smithi007 ceph-mon[30873]: pgmap v437: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 23 MiB/s wr, 747 op/s 2024-09-10T10:59:46.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 32586 41.98 MB/sec warmup 46 sec latency 55.003 ms 2024-09-10T10:59:47.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 33421 42.12 MB/sec warmup 47 sec latency 52.977 ms 2024-09-10T10:59:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:47 smithi133 ceph-mon[35282]: pgmap v438: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 1.05k op/s 2024-09-10T10:59:48.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:47 smithi103 ceph-mon[35160]: pgmap v438: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 1.05k op/s 2024-09-10T10:59:48.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:47 smithi007 ceph-mon[30873]: pgmap v438: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 28 MiB/s wr, 1.05k op/s 2024-09-10T10:59:48.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 33839 41.49 MB/sec warmup 48 sec latency 287.260 ms 2024-09-10T10:59:49.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 34745 41.73 MB/sec warmup 49 sec latency 292.369 ms 2024-09-10T10:59:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:49 smithi133 ceph-mon[35282]: pgmap v439: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 769 op/s 2024-09-10T10:59:50.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:49 smithi103 ceph-mon[35160]: pgmap v439: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 769 op/s 2024-09-10T10:59:50.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:49 smithi007 ceph-mon[30873]: pgmap v439: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 10 MiB/s wr, 769 op/s 2024-09-10T10:59:50.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 35986 42.37 MB/sec warmup 50 sec latency 68.257 ms 2024-09-10T10:59:51.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 36824 42.35 MB/sec warmup 51 sec latency 39.923 ms 2024-09-10T10:59:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:51 smithi133 ceph-mon[35282]: pgmap v440: 97 pgs: 97 active+clean; 602 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-09-10T10:59:52.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:51 smithi103 ceph-mon[35160]: pgmap v440: 97 pgs: 97 active+clean; 602 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-09-10T10:59:52.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:51 smithi007 ceph-mon[30873]: pgmap v440: 97 pgs: 97 active+clean; 602 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-09-10T10:59:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.358+0000 7f52dd123640 1 -- 172.21.15.7:0/4179444207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d810c0d0 msgr2=0x7f52d810e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.358+0000 7f52dd123640 1 --2- 172.21.15.7:0/4179444207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d810c0d0 0x7f52d810e550 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009ca0 tx=0x7f52d002a950 comp rx=0 tx=0).stop 2024-09-10T10:59:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.358+0000 7f52dd123640 1 -- 172.21.15.7:0/4179444207 shutdown_connections 2024-09-10T10:59:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.358+0000 7f52dd123640 1 --2- 172.21.15.7:0/4179444207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d810c0d0 0x7f52d810e550 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.358+0000 7f52dd123640 1 --2- 172.21.15.7:0/4179444207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d81035a0 0x7f52d810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.358+0000 7f52dd123640 1 --2- 172.21.15.7:0/4179444207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d8102c80 0x7f52d8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.358+0000 7f52dd123640 1 -- 172.21.15.7:0/4179444207 >> 172.21.15.7:0/4179444207 conn(0x7f52d8077990 msgr2=0x7f52d80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.358+0000 7f52dd123640 1 -- 172.21.15.7:0/4179444207 shutdown_connections 2024-09-10T10:59:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.358+0000 7f52dd123640 1 -- 172.21.15.7:0/4179444207 wait complete. 2024-09-10T10:59:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.359+0000 7f52dd123640 1 Processor -- start 2024-09-10T10:59:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52dd123640 1 -- start start 2024-09-10T10:59:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52dd123640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d8102c80 0x7f52d81008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52dd123640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81035a0 0x7f52d8100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52dd123640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d810c0d0 0x7f52d8101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52dd123640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8111eb0 con 0x7f52d810c0d0 2024-09-10T10:59:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52dd123640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8111d30 con 0x7f52d8102c80 2024-09-10T10:59:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52dd123640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8112030 con 0x7f52d81035a0 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81035a0 0x7f52d8100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d810c0d0 0x7f52d8101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d8102c80 0x7f52d81008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81035a0 0x7f52d8100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:53416/0 (socket says 172.21.15.7:53416) 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d810c0d0 0x7f52d8101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56928/0 (socket says 172.21.15.7:56928) 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d8102c80 0x7f52d81008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:33724/0 (socket says 172.21.15.7:33724) 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d659c640 1 -- 172.21.15.7:0/2919684806 learned_addr learned my addr 172.21.15.7:0/2919684806 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d659c640 1 -- 172.21.15.7:0/2919684806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d8102c80 msgr2=0x7f52d81008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d659c640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d8102c80 0x7f52d81008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d659c640 1 -- 172.21.15.7:0/2919684806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d810c0d0 msgr2=0x7f52d8101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d659c640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d810c0d0 0x7f52d8101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d659c640 1 -- 172.21.15.7:0/2919684806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d81a6860 con 0x7f52d81035a0 2024-09-10T10:59:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.360+0000 7f52d759e640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d810c0d0 0x7f52d8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.361+0000 7f52d659c640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81035a0 0x7f52d8100e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52c000e9f0 tx=0x7f52c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.361+0000 7f52cffff640 1 -- 172.21.15.7:0/2919684806 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c000ccc0 con 0x7f52d81035a0 2024-09-10T10:59:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.361+0000 7f52dd123640 1 -- 172.21.15.7:0/2919684806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d81a6b50 con 0x7f52d81035a0 2024-09-10T10:59:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.361+0000 7f52cffff640 1 -- 172.21.15.7:0/2919684806 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52c0010910 con 0x7f52d81035a0 2024-09-10T10:59:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.361+0000 7f52cffff640 1 -- 172.21.15.7:0/2919684806 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0018920 con 0x7f52d81035a0 2024-09-10T10:59:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.361+0000 7f52dd123640 1 -- 172.21.15.7:0/2919684806 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d81a70a0 con 0x7f52d81035a0 2024-09-10T10:59:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.363+0000 7f52cffff640 1 -- 172.21.15.7:0/2919684806 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f52c0010430 con 0x7f52d81035a0 2024-09-10T10:59:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.364+0000 7f52dfebc740 1 Processor -- start 2024-09-10T10:59:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.364+0000 7f52dfebc740 1 -- start start 2024-09-10T10:59:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.364+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfdeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.364+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6edfe9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 0x55a6edfec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52dfebc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a6edf432f0 con 0x55a6edfe9cb0 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52dfebc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a6edeeb270 con 0x55a6edfdf410 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6edfe9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6edfe9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:33740/0 (socket says 172.21.15.7:33740) 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfdeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d6d9d640 1 -- 172.21.15.7:0/2735524128 learned_addr learned my addr 172.21.15.7:0/2735524128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52cffff640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f52c40a2da0 0x7f52c40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52dfebc740 1 -- 172.21.15.7:0/2735524128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a6edf3f2b0 con 0x55a6edfdeaf0 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d659c640 1 --2- 172.21.15.7:0/2735524128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 0x55a6edfec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d759e640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f52c40a2da0 0x7f52c40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d6d9d640 1 -- 172.21.15.7:0/2735524128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 msgr2=0x55a6edfdeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/2735524128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfdeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d6d9d640 1 -- 172.21.15.7:0/2735524128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 msgr2=0x55a6edfec0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/2735524128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 0x55a6edfec0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d6d9d640 1 -- 172.21.15.7:0/2735524128 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6edeea340 con 0x55a6edfdf410 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52cffff640 1 -- 172.21.15.7:0/2919684806 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f52c00cf620 con 0x7f52d81035a0 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/2735524128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6edfe9770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009cf0 tx=0x7f52c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94638369d138233 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52d759e640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f52c40a2da0 0x7f52c40a5260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f52d0004af0 tx=0x7f52d002a950 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.365+0000 7f52cd7fa640 1 -- 172.21.15.7:0/2735524128 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8004230 con 0x55a6edfdf410 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.366+0000 7f52cd7fa640 1 -- 172.21.15.7:0/2735524128 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52c80043d0 con 0x55a6edfdf410 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.366+0000 7f52cd7fa640 1 -- 172.21.15.7:0/2735524128 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c80198c0 con 0x55a6edfdf410 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.366+0000 7f52dfebc740 1 -- 172.21.15.7:0/2735524128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 msgr2=0x55a6edfe9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.366+0000 7f52dfebc740 1 --2- 172.21.15.7:0/2735524128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6edfe9770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009cf0 tx=0x7f52c8011810 comp rx=0 tx=0).stop 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.366+0000 7f52dfebc740 1 -- 172.21.15.7:0/2735524128 shutdown_connections 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.366+0000 7f52dfebc740 1 --2- 172.21.15.7:0/2735524128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 0x55a6edfec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.366+0000 7f52dfebc740 1 --2- 172.21.15.7:0/2735524128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6edfe9770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.366+0000 7f52dfebc740 1 --2- 172.21.15.7:0/2735524128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfdeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 -- 172.21.15.7:0/2735524128 >> 172.21.15.7:0/2735524128 conn(0x55a6edfdb060 msgr2=0x55a6edfdb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 -- 172.21.15.7:0/2735524128 shutdown_connections 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 -- 172.21.15.7:0/2735524128 wait complete. 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 -- start start 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 Processor -- start 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfe6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edfe7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfe9cb0 0x55a6edfe75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a6edff5090 con 0x55a6edfdf410 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a6edecc290 con 0x55a6edfe9cb0 2024-09-10T10:59:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a6edff5210 con 0x55a6edfdeaf0 2024-09-10T10:59:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52dfebc740 10 client.0 _ll_register_callbacks cb 0x55a6edefd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:59:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.367+0000 7f52d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edfe7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edfe7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56934/0 (socket says 172.21.15.7:56934) 2024-09-10T10:59:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfe6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52d659c640 1 -- 172.21.15.7:0/1478442802 learned_addr learned my addr 172.21.15.7:0/1478442802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52d759e640 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfe9cb0 0x55a6edfe75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52d6d9d640 1 -- 172.21.15.7:0/1478442802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfe9cb0 msgr2=0x55a6edfe75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfe9cb0 0x55a6edfe75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52d6d9d640 1 -- 172.21.15.7:0/1478442802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 msgr2=0x55a6edfe7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edfe7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52d6d9d640 1 -- 172.21.15.7:0/1478442802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6edc56210 con 0x55a6edfdeaf0 2024-09-10T10:59:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfe6b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f52c8019460 tx=0x7f52c8009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e82b7a575c4905a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T10:59:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52be7fc640 1 -- 172.21.15.7:0/1478442802 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c80037f0 con 0x55a6edfdeaf0 2024-09-10T10:59:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52be7fc640 1 -- 172.21.15.7:0/1478442802 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f52c8003990 con 0x55a6edfdeaf0 2024-09-10T10:59:52.376 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.368+0000 7f52dfebc740 1 -- 172.21.15.7:0/1478442802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52c80043d0 con 0x55a6edfdeaf0 2024-09-10T10:59:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52be7fc640 1 -- 172.21.15.7:0/1478442802 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f52c8032050 con 0x55a6edfdeaf0 2024-09-10T10:59:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52dfebc740 10 client.26026 fetch_fsmap learned FSMap version 46 2024-09-10T10:59:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52dfebc740 1 -- 172.21.15.7:0/1478442802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a6edd22620 con 0x55a6edfdeaf0 2024-09-10T10:59:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52be7fc640 1 -- 172.21.15.7:0/1478442802 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52c80043d0 con 0x55a6edfdeaf0 2024-09-10T10:59:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52be7fc640 1 -- 172.21.15.7:0/1478442802 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f52c80195c0 con 0x55a6edfdeaf0 2024-09-10T10:59:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52dfebc740 10 client.26026 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T10:59:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52dfebc740 10 client.26026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:59:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52dfebc740 20 client.26026 populate_metadata read hostname 'smithi007' 2024-09-10T10:59:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52dfebc740 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a6ee002390 0x55a6ee004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52dfebc740 4 client.26026 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52dfebc740 1 -- 172.21.15.7:0/1478442802 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6edf50a20 con 0x55a6ee002390 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.369+0000 7f52d659c640 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a6ee002390 0x55a6ee004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.370+0000 7f52d659c640 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a6ee002390 0x55a6ee004780 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cfbec19caa3ece6 server_cookie=c6934691c3b236bc in_seq=0 out_seq=0 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.370+0000 7f52be7fc640 10 client.26026 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.374+0000 7f52be7fc640 1 -- 172.21.15.7:0/1478442802 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6edf50a20 con 0x55a6ee002390 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.374+0000 7f52be7fc640 10 client.26026 handle_command_reply: tid=0 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.374+0000 7f52dfebc740 1 client.26026 shutdown 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 1 -- 172.21.15.7:0/1478442802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 msgr2=0x55a6edfe6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfe6b30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f52c8019460 tx=0x7f52c8009f10 comp rx=0 tx=0).stop 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 1 -- 172.21.15.7:0/1478442802 shutdown_connections 2024-09-10T10:59:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a6ee002390 0x55a6ee004780 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfe9cb0 0x55a6edfe75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52be7fc640 0 client.26026 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edfe7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 1 --2- 172.21.15.7:0/1478442802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfe6b30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 1 -- 172.21.15.7:0/1478442802 >> 172.21.15.7:0/1478442802 conn(0x55a6edfdb060 msgr2=0x55a6edfe0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 1 -- 172.21.15.7:0/1478442802 shutdown_connections 2024-09-10T10:59:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 1 -- 172.21.15.7:0/1478442802 wait complete. 2024-09-10T10:59:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.375+0000 7f52dfebc740 20 client.26026 trim_cache size 0 max 16384 2024-09-10T10:59:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.397+0000 7f52dfebc740 1 Processor -- start 2024-09-10T10:59:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.397+0000 7f52dfebc740 1 -- start start 2024-09-10T10:59:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.397+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfe8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.397+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6ee009f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.397+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 0x55a6ee00a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.397+0000 7f52dfebc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81adf30 con 0x55a6edfe9cb0 2024-09-10T10:59:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.397+0000 7f52dfebc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81ab770 con 0x55a6edfdf410 2024-09-10T10:59:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.397+0000 7f52dfebc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81a9620 con 0x55a6edfdeaf0 2024-09-10T10:59:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 0x55a6ee00a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 0x55a6ee00a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56942/0 (socket says 172.21.15.7:56942) 2024-09-10T10:59:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d659c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6ee009f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d759e640 1 -- 172.21.15.7:0/2520422553 learned_addr learned my addr 172.21.15.7:0/2520422553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/2520422553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfe8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d759e640 1 -- 172.21.15.7:0/2520422553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 msgr2=0x55a6edfe8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d759e640 1 --2- 172.21.15.7:0/2520422553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfe8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d759e640 1 -- 172.21.15.7:0/2520422553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 msgr2=0x55a6ee009f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d759e640 1 --2- 172.21.15.7:0/2520422553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6ee009f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d759e640 1 -- 172.21.15.7:0/2520422553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6edfdf240 con 0x55a6edfe9cb0 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/2520422553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfe8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d659c640 1 --2- 172.21.15.7:0/2520422553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6ee009f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52d759e640 1 --2- 172.21.15.7:0/2520422553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 0x55a6ee00a450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009ca0 tx=0x7f52d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad346fc8e65ee9f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52ccff9640 1 -- 172.21.15.7:0/2520422553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0043050 con 0x55a6edfe9cb0 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52ccff9640 1 -- 172.21.15.7:0/2520422553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d0044040 con 0x55a6edfe9cb0 2024-09-10T10:59:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.398+0000 7f52ccff9640 1 -- 172.21.15.7:0/2520422553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0034040 con 0x55a6edfe9cb0 2024-09-10T10:59:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.399+0000 7f52dfebc740 1 -- 172.21.15.7:0/2520422553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 msgr2=0x55a6ee00a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.399+0000 7f52dfebc740 1 --2- 172.21.15.7:0/2520422553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 0x55a6ee00a450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009ca0 tx=0x7f52d0004a40 comp rx=0 tx=0).stop 2024-09-10T10:59:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 -- 172.21.15.7:0/2520422553 shutdown_connections 2024-09-10T10:59:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 --2- 172.21.15.7:0/2520422553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfe9cb0 0x55a6ee00a450 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 --2- 172.21.15.7:0/2520422553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdf410 0x55a6ee009f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 --2- 172.21.15.7:0/2520422553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfdeaf0 0x55a6edfe8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 -- 172.21.15.7:0/2520422553 >> 172.21.15.7:0/2520422553 conn(0x55a6edfdb060 msgr2=0x55a6edfe8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 -- 172.21.15.7:0/2520422553 shutdown_connections 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 -- 172.21.15.7:0/2520422553 wait complete. 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 -- start start 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 Processor -- start 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdeaf0 0x55a6edffa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edffad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfe9cb0 0x55a6edfe3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81a9c70 con 0x55a6edfdf410 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81ab0a0 con 0x55a6edfdeaf0 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.400+0000 7f52dfebc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81aa300 con 0x55a6edfe9cb0 2024-09-10T10:59:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52dfebc740 10 client.0 _ll_register_callbacks cb 0x7f52c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edffad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d759e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfe9cb0 0x55a6edfe3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edffad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56946/0 (socket says 172.21.15.7:56946) 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdeaf0 0x55a6edffa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d659c640 1 -- 172.21.15.7:0/4065430032 learned_addr learned my addr 172.21.15.7:0/4065430032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d659c640 1 -- 172.21.15.7:0/4065430032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfe9cb0 msgr2=0x55a6edfe3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d659c640 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfe9cb0 0x55a6edfe3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d659c640 1 -- 172.21.15.7:0/4065430032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdeaf0 msgr2=0x55a6edffa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d659c640 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdeaf0 0x55a6edffa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d659c640 1 -- 172.21.15.7:0/4065430032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6edfe3f40 con 0x55a6edfdf410 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d759e640 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfe9cb0 0x55a6edfe3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdeaf0 0x55a6edffa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52d659c640 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edffad70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f52c009bb50 tx=0x7f52c000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1f7c05f1919e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:59:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52be7fc640 1 -- 172.21.15.7:0/4065430032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0098230 con 0x55a6edfdf410 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52be7fc640 1 -- 172.21.15.7:0/4065430032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f52c00ccfc0 con 0x55a6edfdf410 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.401+0000 7f52dfebc740 1 -- 172.21.15.7:0/4065430032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52d0044040 con 0x55a6edfdf410 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52be7fc640 1 -- 172.21.15.7:0/4065430032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f52c00983d0 con 0x55a6edfdf410 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52dfebc740 10 client.16383 fetch_fsmap learned FSMap version 46 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52dfebc740 1 -- 172.21.15.7:0/4065430032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55a6edffa160 con 0x55a6edfdf410 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52be7fc640 1 -- 172.21.15.7:0/4065430032 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d0044040 con 0x55a6edfdf410 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52be7fc640 1 -- 172.21.15.7:0/4065430032 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f52c00ba2b0 con 0x55a6edfdf410 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52dfebc740 10 client.16383 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52dfebc740 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T10:59:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52dfebc740 20 client.16383 populate_metadata read hostname 'smithi007' 2024-09-10T10:59:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52dfebc740 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a6ee002390 0x55a6ee06f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:59:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52dfebc740 4 client.16383 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:59:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52dfebc740 1 -- 172.21.15.7:0/4065430032 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6edf4ad70 con 0x55a6ee002390 2024-09-10T10:59:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.402+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a6ee002390 0x55a6ee06f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:59:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.403+0000 7f52d6d9d640 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a6ee002390 0x55a6ee06f430 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b9d2b54e89c6a24 server_cookie=41e4db37a8560ebb in_seq=0 out_seq=0 2024-09-10T10:59:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.403+0000 7f52be7fc640 10 client.16383 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52be7fc640 1 -- 172.21.15.7:0/4065430032 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6edf4ad70 con 0x55a6ee002390 2024-09-10T10:59:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52be7fc640 10 client.16383 handle_command_reply: tid=0 2024-09-10T10:59:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52dfebc740 1 client.16383 shutdown 2024-09-10T10:59:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52dfebc740 1 -- 172.21.15.7:0/4065430032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 msgr2=0x55a6edffad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52dfebc740 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edffad70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f52c009bb50 tx=0x7f52c000cf60 comp rx=0 tx=0).stop 2024-09-10T10:59:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52dfebc740 1 -- 172.21.15.7:0/4065430032 shutdown_connections 2024-09-10T10:59:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52dfebc740 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55a6ee002390 0x55a6ee06f430 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52be7fc640 0 client.16383 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T10:59:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52dfebc740 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6edfe9cb0 0x55a6edfe3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52dfebc740 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6edfdf410 0x55a6edffad70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52dfebc740 1 --2- 172.21.15.7:0/4065430032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6edfdeaf0 0x55a6edffa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.817+0000 7f52dfebc740 1 -- 172.21.15.7:0/4065430032 >> 172.21.15.7:0/4065430032 conn(0x55a6edfdb060 msgr2=0x55a6edfe0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.818+0000 7f52dfebc740 1 -- 172.21.15.7:0/4065430032 shutdown_connections 2024-09-10T10:59:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.818+0000 7f52dfebc740 1 -- 172.21.15.7:0/4065430032 wait complete. 2024-09-10T10:59:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.818+0000 7f52dfebc740 20 client.16383 trim_cache size 0 max 16384 2024-09-10T10:59:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T10:59:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T10:59:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T10:59:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T10:59:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 -- 172.21.15.7:0/2919684806 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f52c40a2da0 msgr2=0x7f52c40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f52c40a2da0 0x7f52c40a5260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f52d0004af0 tx=0x7f52d002a950 comp rx=0 tx=0).stop 2024-09-10T10:59:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 -- 172.21.15.7:0/2919684806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81035a0 msgr2=0x7f52d8100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:59:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81035a0 0x7f52d8100e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52c000e9f0 tx=0x7f52c000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 -- 172.21.15.7:0/2919684806 shutdown_connections 2024-09-10T10:59:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f52c40a2da0 0x7f52c40a5260 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d810c0d0 0x7f52d8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d81035a0 0x7f52d8100e10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 --2- 172.21.15.7:0/2919684806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52d8102c80 0x7f52d81008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:59:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 -- 172.21.15.7:0/2919684806 >> 172.21.15.7:0/2919684806 conn(0x7f52d8077990 msgr2=0x7f52d80fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:59:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 -- 172.21.15.7:0/2919684806 shutdown_connections 2024-09-10T10:59:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T10:59:52.819+0000 7f52dd123640 1 -- 172.21.15.7:0/2919684806 wait complete. 2024-09-10T10:59:52.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 37461 42.15 MB/sec warmup 52 sec latency 222.407 ms 2024-09-10T10:59:53.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 38472 42.37 MB/sec warmup 53 sec latency 147.890 ms 2024-09-10T10:59:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:53 smithi133 ceph-mon[35282]: pgmap v441: 97 pgs: 97 active+clean; 602 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 625 op/s 2024-09-10T10:59:54.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:53 smithi103 ceph-mon[35160]: pgmap v441: 97 pgs: 97 active+clean; 602 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 625 op/s 2024-09-10T10:59:54.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:53 smithi007 ceph-mon[30873]: pgmap v441: 97 pgs: 97 active+clean; 602 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 625 op/s 2024-09-10T10:59:54.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 39268 42.38 MB/sec warmup 54 sec latency 147.836 ms 2024-09-10T10:59:55.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 40181 42.63 MB/sec warmup 55 sec latency 76.284 ms 2024-09-10T10:59:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:55 smithi133 ceph-mon[35282]: pgmap v442: 97 pgs: 97 active+clean; 602 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 625 op/s 2024-09-10T10:59:56.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:55 smithi103 ceph-mon[35160]: pgmap v442: 97 pgs: 97 active+clean; 602 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 625 op/s 2024-09-10T10:59:56.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:55 smithi007 ceph-mon[30873]: pgmap v442: 97 pgs: 97 active+clean; 602 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 625 op/s 2024-09-10T10:59:56.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 41215 42.73 MB/sec warmup 56 sec latency 55.239 ms 2024-09-10T10:59:57.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 42076 42.95 MB/sec warmup 57 sec latency 43.206 ms 2024-09-10T10:59:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:57 smithi133 ceph-mon[35282]: pgmap v443: 97 pgs: 97 active+clean; 612 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-10T10:59:58.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:57 smithi103 ceph-mon[35160]: pgmap v443: 97 pgs: 97 active+clean; 612 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-10T10:59:58.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:57 smithi007 ceph-mon[30873]: pgmap v443: 97 pgs: 97 active+clean; 612 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-10T10:59:58.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 42936 42.78 MB/sec warmup 58 sec latency 64.031 ms 2024-09-10T10:59:59.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 43717 42.78 MB/sec warmup 59 sec latency 268.461 ms 2024-09-10T10:59:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 10:59:59 smithi133 ceph-mon[35282]: pgmap v444: 97 pgs: 97 active+clean; 612 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 761 op/s 2024-09-10T11:00:00.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 10:59:59 smithi103 ceph-mon[35160]: pgmap v444: 97 pgs: 97 active+clean; 612 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 761 op/s 2024-09-10T11:00:00.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 10:59:59 smithi007 ceph-mon[30873]: pgmap v444: 97 pgs: 97 active+clean; 612 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.9 MiB/s wr, 761 op/s 2024-09-10T11:00:00.883 INFO:tasks.workunit.client.0.smithi007.stdout: 10 44624 43.08 MB/sec warmup 60 sec latency 267.231 ms 2024-09-10T11:00:00.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:00 smithi133 ceph-mon[35282]: overall HEALTH_OK 2024-09-10T11:00:01.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:00 smithi103 ceph-mon[35160]: overall HEALTH_OK 2024-09-10T11:00:01.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:00 smithi007 ceph-mon[30873]: overall HEALTH_OK 2024-09-10T11:00:01.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 44710 42.45 MB/sec warmup 61 sec latency 692.438 ms 2024-09-10T11:00:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:01 smithi133 ceph-mon[35282]: pgmap v445: 97 pgs: 97 active+clean; 652 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-10T11:00:02.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:01 smithi103 ceph-mon[35160]: pgmap v445: 97 pgs: 97 active+clean; 652 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-10T11:00:02.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:01 smithi007 ceph-mon[30873]: pgmap v445: 97 pgs: 97 active+clean; 652 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-10T11:00:02.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 44710 41.77 MB/sec warmup 62 sec latency 1518.953 ms 2024-09-10T11:00:03.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 44710 41.11 MB/sec warmup 63 sec latency 2519.073 ms 2024-09-10T11:00:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:03 smithi133 ceph-mon[35282]: pgmap v446: 97 pgs: 97 active+clean; 652 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 772 op/s 2024-09-10T11:00:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:03 smithi103 ceph-mon[35160]: pgmap v446: 97 pgs: 97 active+clean; 652 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 772 op/s 2024-09-10T11:00:04.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:03 smithi007 ceph-mon[30873]: pgmap v446: 97 pgs: 97 active+clean; 652 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 772 op/s 2024-09-10T11:00:04.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 44710 40.46 MB/sec warmup 64 sec latency 3519.125 ms 2024-09-10T11:00:05.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 44710 39.84 MB/sec warmup 65 sec latency 4519.177 ms 2024-09-10T11:00:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.922+0000 7f5b121d6640 1 -- 172.21.15.7:0/863909072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c1009f0 msgr2=0x7f5b0c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.922+0000 7f5b121d6640 1 --2- 172.21.15.7:0/863909072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c1009f0 0x7f5b0c100e50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5b00009930 tx=0x7f5b0002a420 comp rx=0 tx=0).stop 2024-09-10T11:00:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.922+0000 7f5b121d6640 1 -- 172.21.15.7:0/863909072 shutdown_connections 2024-09-10T11:00:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.922+0000 7f5b121d6640 1 --2- 172.21.15.7:0/863909072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c101390 0x7f5b0c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.922+0000 7f5b121d6640 1 --2- 172.21.15.7:0/863909072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c1009f0 0x7f5b0c100e50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.922+0000 7f5b121d6640 1 --2- 172.21.15.7:0/863909072 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0c106a00 0x7f5b0c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.922+0000 7f5b121d6640 1 -- 172.21.15.7:0/863909072 >> 172.21.15.7:0/863909072 conn(0x7f5b0c0fc6b0 msgr2=0x7f5b0c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.922+0000 7f5b121d6640 1 -- 172.21.15.7:0/863909072 shutdown_connections 2024-09-10T11:00:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.922+0000 7f5b121d6640 1 -- 172.21.15.7:0/863909072 wait complete. 2024-09-10T11:00:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b121d6640 1 Processor -- start 2024-09-10T11:00:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b121d6640 1 -- start start 2024-09-10T11:00:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b121d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c1009f0 0x7f5b0c0fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b121d6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0c101390 0x7f5b0c0fd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b121d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c106a00 0x7f5b0c0fd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b121d6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c110e40 con 0x7f5b0c106a00 2024-09-10T11:00:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b121d6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c110cc0 con 0x7f5b0c101390 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b121d6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c110fc0 con 0x7f5b0c1009f0 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c106a00 0x7f5b0c0fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b0b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c1009f0 0x7f5b0c0fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b0b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c1009f0 0x7f5b0c0fcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:35056/0 (socket says 172.21.15.7:35056) 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.923+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c106a00 0x7f5b0c0fd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46636/0 (socket says 172.21.15.7:46636) 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b0b7fe640 1 -- 172.21.15.7:0/142698316 learned_addr learned my addr 172.21.15.7:0/142698316 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b0affd640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0c101390 0x7f5b0c0fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b0b7fe640 1 -- 172.21.15.7:0/142698316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0c101390 msgr2=0x7f5b0c0fd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0c101390 0x7f5b0c0fd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b0b7fe640 1 -- 172.21.15.7:0/142698316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c106a00 msgr2=0x7f5b0c0fd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c106a00 0x7f5b0c0fd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b0b7fe640 1 -- 172.21.15.7:0/142698316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0c1a6930 con 0x7f5b0c1009f0 2024-09-10T11:00:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c106a00 0x7f5b0c0fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:00:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c1009f0 0x7f5b0c0fcc00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00d950 tx=0x7f5afc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b08ff9640 1 -- 172.21.15.7:0/142698316 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc017030 con 0x7f5b0c1009f0 2024-09-10T11:00:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b121d6640 1 -- 172.21.15.7:0/142698316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0c1a6c20 con 0x7f5b0c1009f0 2024-09-10T11:00:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b08ff9640 1 -- 172.21.15.7:0/142698316 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5afc004020 con 0x7f5b0c1009f0 2024-09-10T11:00:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b08ff9640 1 -- 172.21.15.7:0/142698316 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0047a0 con 0x7f5b0c1009f0 2024-09-10T11:00:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.924+0000 7f5b121d6640 1 -- 172.21.15.7:0/142698316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0c1a7140 con 0x7f5b0c1009f0 2024-09-10T11:00:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.927+0000 7f5b08ff9640 1 -- 172.21.15.7:0/142698316 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f5afc0041d0 con 0x7f5b0c1009f0 2024-09-10T11:00:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.927+0000 7f5b1504f740 1 Processor -- start 2024-09-10T11:00:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.927+0000 7f5b1504f740 1 -- start start 2024-09-10T11:00:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.927+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1055faf0 0x563c1055fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.927+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 0x563c1056a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.927+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c1056d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.927+0000 7f5b1504f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563c104c42f0 con 0x563c1056acb0 2024-09-10T11:00:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b1504f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563c1046c270 con 0x563c10560410 2024-09-10T11:00:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b1504f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563c104c02b0 con 0x563c1055faf0 2024-09-10T11:00:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b08ff9640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5af40a2da0 0x7f5af40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c1056d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c1056d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46648/0 (socket says 172.21.15.7:46648) 2024-09-10T11:00:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0affd640 1 -- 172.21.15.7:0/3039179985 learned_addr learned my addr 172.21.15.7:0/3039179985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/3039179985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1055faf0 0x563c1055fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5af40a2da0 0x7f5af40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b08ff9640 1 -- 172.21.15.7:0/142698316 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f5afc0ced60 con 0x7f5b0c1009f0 2024-09-10T11:00:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/3039179985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 0x563c1056a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0affd640 1 -- 172.21.15.7:0/3039179985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1055faf0 msgr2=0x563c1055fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0affd640 1 --2- 172.21.15.7:0/3039179985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1055faf0 0x563c1055fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0affd640 1 -- 172.21.15.7:0/3039179985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 msgr2=0x563c1056a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0affd640 1 --2- 172.21.15.7:0/3039179985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 0x563c1056a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0affd640 1 -- 172.21.15.7:0/3039179985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1046b340 con 0x563c1056acb0 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/3039179985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 0x563c1056a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0affd640 1 --2- 172.21.15.7:0/3039179985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c1056d0a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5b0002a940 tx=0x7f5b00005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84445d59b78a1b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5af40a2da0 0x7f5af40a5260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5b040099e0 tx=0x7f5b04009500 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.928+0000 7f5af1ffb640 1 -- 172.21.15.7:0/3039179985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00040070 con 0x563c1056acb0 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5af1ffb640 1 -- 172.21.15.7:0/3039179985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b000318f0 con 0x563c1056acb0 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5af1ffb640 1 -- 172.21.15.7:0/3039179985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00035040 con 0x563c1056acb0 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5b1504f740 1 -- 172.21.15.7:0/3039179985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 msgr2=0x563c1056d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5b1504f740 1 --2- 172.21.15.7:0/3039179985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c1056d0a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5b0002a940 tx=0x7f5b00005c00 comp rx=0 tx=0).stop 2024-09-10T11:00:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5b1504f740 1 -- 172.21.15.7:0/3039179985 shutdown_connections 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5b1504f740 1 --2- 172.21.15.7:0/3039179985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c1056d0a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5b1504f740 1 --2- 172.21.15.7:0/3039179985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 0x563c1056a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5b1504f740 1 --2- 172.21.15.7:0/3039179985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1055faf0 0x563c1055fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5b1504f740 1 -- 172.21.15.7:0/3039179985 >> 172.21.15.7:0/3039179985 conn(0x563c1055c060 msgr2=0x563c1055c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5b1504f740 1 -- 172.21.15.7:0/3039179985 shutdown_connections 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.929+0000 7f5b1504f740 1 -- 172.21.15.7:0/3039179985 wait complete. 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b1504f740 1 -- start start 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b1504f740 1 Processor -- start 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 0x563c10567b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c10560410 0x563c10568050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10568590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b1504f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563c1056d880 con 0x563c1056acb0 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b1504f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563c1044d290 con 0x563c1055faf0 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b1504f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563c1056da00 con 0x563c10560410 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b1504f740 10 client.0 _ll_register_callbacks cb 0x563c1047e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10568590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10568590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46658/0 (socket says 172.21.15.7:46658) 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.930+0000 7f5b0bfff640 1 -- 172.21.15.7:0/3590078101 learned_addr learned my addr 172.21.15.7:0/3590078101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 0x563c10567b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b0affd640 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c10560410 0x563c10568050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b0bfff640 1 -- 172.21.15.7:0/3590078101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c10560410 msgr2=0x563c10568050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c10560410 0x563c10568050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b0bfff640 1 -- 172.21.15.7:0/3590078101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 msgr2=0x563c10567b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 0x563c10567b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b0bfff640 1 -- 172.21.15.7:0/3590078101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c101d7210 con 0x563c1056acb0 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10568590 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5b04003ed0 tx=0x7f5b04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d50b46125b658a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5aeaffd640 1 -- 172.21.15.7:0/3590078101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0401f690 con 0x563c1056acb0 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5aeaffd640 1 -- 172.21.15.7:0/3590078101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f5b0401f830 con 0x563c1056acb0 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b1504f740 1 -- 172.21.15.7:0/3590078101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b000318f0 con 0x563c1056acb0 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5aeaffd640 1 -- 172.21.15.7:0/3590078101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5b0401fa30 con 0x563c1056acb0 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b1504f740 10 client.16395 fetch_fsmap learned FSMap version 46 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.931+0000 7f5b1504f740 1 -- 172.21.15.7:0/3590078101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563c102a3620 con 0x563c1056acb0 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.932+0000 7f5aeaffd640 1 -- 172.21.15.7:0/3590078101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b000318f0 con 0x563c1056acb0 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.932+0000 7f5aeaffd640 1 -- 172.21.15.7:0/3590078101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f5b04031070 con 0x563c1056acb0 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.932+0000 7f5b1504f740 10 client.16395 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.932+0000 7f5b1504f740 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.932+0000 7f5b1504f740 20 client.16395 populate_metadata read hostname 'smithi007' 2024-09-10T11:00:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.932+0000 7f5b1504f740 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563c1057aae0 0x563c1057cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.932+0000 7f5b1504f740 4 client.16395 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.932+0000 7f5b1504f740 1 -- 172.21.15.7:0/3590078101 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c104d1a20 con 0x563c1057aae0 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.932+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563c1057aae0 0x563c1057cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.933+0000 7f5aeaffd640 10 client.16395 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.933+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563c1057aae0 0x563c1057cf70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cf2753d90150dae server_cookie=162d285504b17ff8 in_seq=0 out_seq=0 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.933+0000 7f5aeaffd640 1 -- 172.21.15.7:0/3590078101 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c104d1a20 con 0x563c1057aae0 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.933+0000 7f5aeaffd640 10 client.16395 handle_command_reply: tid=0 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.933+0000 7f5b1504f740 1 client.16395 shutdown 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 1 -- 172.21.15.7:0/3590078101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 msgr2=0x563c10568590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10568590 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5b04003ed0 tx=0x7f5b04004a40 comp rx=0 tx=0).stop 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 1 -- 172.21.15.7:0/3590078101 shutdown_connections 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563c1057aae0 0x563c1057cf70 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10568590 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c10560410 0x563c10568050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5aeaffd640 0 client.16395 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 1 --2- 172.21.15.7:0/3590078101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 0x563c10567b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 1 -- 172.21.15.7:0/3590078101 >> 172.21.15.7:0/3590078101 conn(0x563c1055c060 msgr2=0x563c10561d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 1 -- 172.21.15.7:0/3590078101 shutdown_connections 2024-09-10T11:00:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 1 -- 172.21.15.7:0/3590078101 wait complete. 2024-09-10T11:00:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.934+0000 7f5b1504f740 20 client.16395 trim_cache size 0 max 16384 2024-09-10T11:00:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.948+0000 7f5b1504f740 1 Processor -- start 2024-09-10T11:00:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.948+0000 7f5b1504f740 1 -- start start 2024-09-10T11:00:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 0x563c105683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c10560410 0x563c10582840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1056acb0 0x563c10582d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b1504f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1b23a0 con 0x563c10560410 2024-09-10T11:00:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b1504f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1afbe0 con 0x563c1055faf0 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b1504f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1ada50 con 0x563c1056acb0 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c10560410 0x563c10582840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c10560410 0x563c10582840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46666/0 (socket says 172.21.15.7:46666) 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0affd640 1 -- 172.21.15.7:0/1980411110 learned_addr learned my addr 172.21.15.7:0/1980411110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/1980411110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 0x563c105683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0affd640 1 -- 172.21.15.7:0/1980411110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1056acb0 msgr2=0x563c10582d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0affd640 1 --2- 172.21.15.7:0/1980411110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1056acb0 0x563c10582d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0affd640 1 -- 172.21.15.7:0/1980411110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 msgr2=0x563c105683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0affd640 1 --2- 172.21.15.7:0/1980411110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 0x563c105683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0affd640 1 -- 172.21.15.7:0/1980411110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1055be90 con 0x563c10560410 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/1980411110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 0x563c105683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5b0affd640 1 --2- 172.21.15.7:0/1980411110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c10560410 0x563c10582840 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x563c10562690 tx=0x7f5b0002aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860e2c5780cbecd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:05.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5af17fa640 1 -- 172.21.15.7:0/1980411110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0004b020 con 0x563c10560410 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.949+0000 7f5af17fa640 1 -- 172.21.15.7:0/1980411110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b00005520 con 0x563c10560410 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5af17fa640 1 -- 172.21.15.7:0/1980411110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00040070 con 0x563c10560410 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5b1504f740 1 -- 172.21.15.7:0/1980411110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c10560410 msgr2=0x563c10582840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5b1504f740 1 --2- 172.21.15.7:0/1980411110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c10560410 0x563c10582840 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x563c10562690 tx=0x7f5b0002aae0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5b1504f740 1 -- 172.21.15.7:0/1980411110 shutdown_connections 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5b1504f740 1 --2- 172.21.15.7:0/1980411110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1056acb0 0x563c10582d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5b1504f740 1 --2- 172.21.15.7:0/1980411110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c10560410 0x563c10582840 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5b1504f740 1 --2- 172.21.15.7:0/1980411110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c1055faf0 0x563c105683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5b1504f740 1 -- 172.21.15.7:0/1980411110 >> 172.21.15.7:0/1980411110 conn(0x563c1055c060 msgr2=0x563c10569b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5b1504f740 1 -- 172.21.15.7:0/1980411110 shutdown_connections 2024-09-10T11:00:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5b1504f740 1 -- 172.21.15.7:0/1980411110 wait complete. 2024-09-10T11:00:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.950+0000 7f5b1504f740 1 -- start start 2024-09-10T11:00:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b1504f740 1 Processor -- start 2024-09-10T11:00:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1055faf0 0x563c105734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 0x563c10564910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b1504f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10564f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b1504f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1ae0a0 con 0x563c1056acb0 2024-09-10T11:00:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b1504f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1af510 con 0x563c10560410 2024-09-10T11:00:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b1504f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0c1ae770 con 0x563c1055faf0 2024-09-10T11:00:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b1504f740 10 client.0 _ll_register_callbacks cb 0x7f5b04003d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:00:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10564f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10564f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46682/0 (socket says 172.21.15.7:46682) 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b0bfff640 1 -- 172.21.15.7:0/2827251251 learned_addr learned my addr 172.21.15.7:0/2827251251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1055faf0 0x563c105734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b0affd640 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 0x563c10564910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b0bfff640 1 -- 172.21.15.7:0/2827251251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1055faf0 msgr2=0x563c105734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1055faf0 0x563c105734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b0bfff640 1 -- 172.21.15.7:0/2827251251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 msgr2=0x563c10564910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 0x563c10564910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.951+0000 7f5b0bfff640 1 -- 172.21.15.7:0/2827251251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c105572d0 con 0x563c1056acb0 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5b0bfff640 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10564f70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5b04003d00 tx=0x7f5b040204d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a02fd8a0ca9e23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:00:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5aeaffd640 1 -- 172.21.15.7:0/2827251251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0401fbd0 con 0x563c1056acb0 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5aeaffd640 1 -- 172.21.15.7:0/2827251251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8564+0+0 (secure 0 0 0) 0x7f5b0401fd70 con 0x563c1056acb0 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5b1504f740 1 -- 172.21.15.7:0/2827251251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b00005520 con 0x563c1056acb0 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5aeaffd640 1 -- 172.21.15.7:0/2827251251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5b04044050 con 0x563c1056acb0 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5b1504f740 10 client.16401 fetch_fsmap learned FSMap version 46 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5b1504f740 1 -- 172.21.15.7:0/2827251251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563c1055ccf0 con 0x563c1056acb0 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5aeaffd640 1 -- 172.21.15.7:0/2827251251 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b00005520 con 0x563c1056acb0 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5aeaffd640 1 -- 172.21.15.7:0/2827251251 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f5b04031050 con 0x563c1056acb0 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5b1504f740 10 client.16401 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5b1504f740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5b1504f740 20 client.16401 populate_metadata read hostname 'smithi007' 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5b1504f740 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563c105e5c80 0x563c105e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5b1504f740 4 client.16401 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.952+0000 7f5b1504f740 1 -- 172.21.15.7:0/2827251251 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c104cbd70 con 0x563c105e5c80 2024-09-10T11:00:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.953+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563c105e5c80 0x563c105e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.953+0000 7f5aeaffd640 10 client.16401 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:05.953+0000 7f5b0b7fe640 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563c105e5c80 0x563c105e80a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349143a9079dc312 server_cookie=d940d334f8d49ba8 in_seq=0 out_seq=0 2024-09-10T11:00:05.974 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:05 smithi103 ceph-mon[35160]: pgmap v447: 97 pgs: 97 active+clean; 652 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 772 op/s 2024-09-10T11:00:05.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:05 smithi133 ceph-mon[35282]: pgmap v447: 97 pgs: 97 active+clean; 652 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 772 op/s 2024-09-10T11:00:05.987 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:05 smithi007 ceph-mon[30873]: pgmap v447: 97 pgs: 97 active+clean; 652 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 772 op/s 2024-09-10T11:00:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5aeaffd640 1 -- 172.21.15.7:0/2827251251 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c104cbd70 con 0x563c105e5c80 2024-09-10T11:00:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5aeaffd640 10 client.16401 handle_command_reply: tid=0 2024-09-10T11:00:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5b1504f740 1 client.16401 shutdown 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5b1504f740 1 -- 172.21.15.7:0/2827251251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 msgr2=0x563c10564f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5b1504f740 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10564f70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5b04003d00 tx=0x7f5b040204d0 comp rx=0 tx=0).stop 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5b1504f740 1 -- 172.21.15.7:0/2827251251 shutdown_connections 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5b1504f740 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x563c105e5c80 0x563c105e80a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5aeaffd640 0 client.16401 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5b1504f740 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c1056acb0 0x563c10564f70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5b1504f740 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563c10560410 0x563c10564910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5b1504f740 1 --2- 172.21.15.7:0/2827251251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c1055faf0 0x563c105734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.234+0000 7f5b1504f740 1 -- 172.21.15.7:0/2827251251 >> 172.21.15.7:0/2827251251 conn(0x563c1055c060 msgr2=0x563c1055a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b1504f740 1 -- 172.21.15.7:0/2827251251 shutdown_connections 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b1504f740 1 -- 172.21.15.7:0/2827251251 wait complete. 2024-09-10T11:00:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b1504f740 20 client.16401 trim_cache size 0 max 16384 2024-09-10T11:00:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T11:00:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T11:00:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T11:00:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T11:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b121d6640 1 -- 172.21.15.7:0/142698316 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5af40a2da0 msgr2=0x7f5af40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b121d6640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5af40a2da0 0x7f5af40a5260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5b040099e0 tx=0x7f5b04009500 comp rx=0 tx=0).stop 2024-09-10T11:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b121d6640 1 -- 172.21.15.7:0/142698316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c1009f0 msgr2=0x7f5b0c0fcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b121d6640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c1009f0 0x7f5b0c0fcc00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00d950 tx=0x7f5afc00de10 comp rx=0 tx=0).stop 2024-09-10T11:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b121d6640 1 -- 172.21.15.7:0/142698316 shutdown_connections 2024-09-10T11:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b121d6640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f5af40a2da0 0x7f5af40a5260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b121d6640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b0c106a00 0x7f5b0c0fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b121d6640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b0c101390 0x7f5b0c0fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b121d6640 1 --2- 172.21.15.7:0/142698316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c1009f0 0x7f5b0c0fcc00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.235+0000 7f5b121d6640 1 -- 172.21.15.7:0/142698316 >> 172.21.15.7:0/142698316 conn(0x7f5b0c0fc6b0 msgr2=0x7f5b0c06a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.236+0000 7f5b121d6640 1 -- 172.21.15.7:0/142698316 shutdown_connections 2024-09-10T11:00:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:06.236+0000 7f5b121d6640 1 -- 172.21.15.7:0/142698316 wait complete. 2024-09-10T11:00:06.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 45121 39.60 MB/sec warmup 66 sec latency 4912.470 ms 2024-09-10T11:00:07.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 45599 39.27 MB/sec warmup 67 sec latency 116.553 ms 2024-09-10T11:00:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:07 smithi103 ceph-mon[35160]: pgmap v448: 97 pgs: 97 active+clean; 702 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 973 op/s 2024-09-10T11:00:08.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:07 smithi007 ceph-mon[30873]: pgmap v448: 97 pgs: 97 active+clean; 702 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 973 op/s 2024-09-10T11:00:08.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:07 smithi133 ceph-mon[35282]: pgmap v448: 97 pgs: 97 active+clean; 702 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 973 op/s 2024-09-10T11:00:08.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 46195 39.09 MB/sec warmup 68 sec latency 152.656 ms 2024-09-10T11:00:09.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 46528 38.79 MB/sec warmup 69 sec latency 288.843 ms 2024-09-10T11:00:10.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:09 smithi103 ceph-mon[35160]: pgmap v449: 97 pgs: 97 active+clean; 702 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 531 op/s 2024-09-10T11:00:10.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:09 smithi103 ceph-mon[35160]: osdmap e102: 12 total, 12 up, 12 in 2024-09-10T11:00:10.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:09 smithi007 ceph-mon[30873]: pgmap v449: 97 pgs: 97 active+clean; 702 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 531 op/s 2024-09-10T11:00:10.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:09 smithi007 ceph-mon[30873]: osdmap e102: 12 total, 12 up, 12 in 2024-09-10T11:00:10.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:09 smithi133 ceph-mon[35282]: pgmap v449: 97 pgs: 97 active+clean; 702 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 12 MiB/s wr, 531 op/s 2024-09-10T11:00:10.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:09 smithi133 ceph-mon[35282]: osdmap e102: 12 total, 12 up, 12 in 2024-09-10T11:00:10.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 47340 38.78 MB/sec warmup 70 sec latency 93.091 ms 2024-09-10T11:00:11.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:10 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:00:11.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:10 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:00:11.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:10 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:00:11.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 48346 39.26 MB/sec warmup 71 sec latency 55.855 ms 2024-09-10T11:00:12.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:11 smithi103 ceph-mon[35160]: pgmap v451: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 702 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-09-10T11:00:12.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:11 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:00:12.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:11 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:00:12.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:11 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T11:00:12.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:11 smithi007 ceph-mon[30873]: pgmap v451: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 702 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-09-10T11:00:12.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:11 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:00:12.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:11 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:00:12.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:11 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T11:00:12.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:11 smithi133 ceph-mon[35282]: pgmap v451: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 702 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-09-10T11:00:12.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:11 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:00:12.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:11 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:00:12.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:11 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T11:00:12.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 48962 39.12 MB/sec warmup 72 sec latency 43.197 ms 2024-09-10T11:00:13.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 49750 39.07 MB/sec warmup 73 sec latency 73.829 ms 2024-09-10T11:00:14.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:13 smithi103 ceph-mon[35160]: pgmap v452: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 702 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-09-10T11:00:14.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:13 smithi007 ceph-mon[30873]: pgmap v452: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 702 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-09-10T11:00:14.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:13 smithi133 ceph-mon[35282]: pgmap v452: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 702 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-09-10T11:00:14.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 50536 39.06 MB/sec warmup 74 sec latency 249.783 ms 2024-09-10T11:00:15.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 51496 39.30 MB/sec warmup 75 sec latency 69.336 ms 2024-09-10T11:00:16.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:15 smithi103 ceph-mon[35160]: pgmap v453: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 702 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-09-10T11:00:16.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:15 smithi007 ceph-mon[30873]: pgmap v453: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 702 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-09-10T11:00:16.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:15 smithi133 ceph-mon[35282]: pgmap v453: 97 pgs: 33 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 49 active+clean; 702 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-09-10T11:00:16.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 52401 39.50 MB/sec warmup 76 sec latency 44.735 ms 2024-09-10T11:00:17.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 53201 39.46 MB/sec warmup 77 sec latency 58.441 ms 2024-09-10T11:00:18.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:17 smithi103 ceph-mon[35160]: pgmap v454: 97 pgs: 97 active+clean; 731 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-09-10T11:00:18.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:17 smithi007 ceph-mon[30873]: pgmap v454: 97 pgs: 97 active+clean; 731 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-09-10T11:00:18.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:17 smithi133 ceph-mon[35282]: pgmap v454: 97 pgs: 97 active+clean; 731 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-09-10T11:00:18.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 54117 39.57 MB/sec warmup 78 sec latency 70.181 ms 2024-09-10T11:00:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.353+0000 7fd60d34b640 1 -- 172.21.15.7:0/4193050225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd608101370 msgr2=0x7fd60810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.353+0000 7fd60d34b640 1 --2- 172.21.15.7:0/4193050225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd608101370 0x7fd60810d8a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd600009cd0 tx=0x7fd60002a8e0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.354+0000 7fd60d34b640 1 -- 172.21.15.7:0/4193050225 shutdown_connections 2024-09-10T11:00:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.354+0000 7fd60d34b640 1 --2- 172.21.15.7:0/4193050225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd608101370 0x7fd60810d8a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.354+0000 7fd60d34b640 1 --2- 172.21.15.7:0/4193050225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6081009d0 0x7fd608100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.354+0000 7fd60d34b640 1 --2- 172.21.15.7:0/4193050225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6081069e0 0x7fd608106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.354+0000 7fd60d34b640 1 -- 172.21.15.7:0/4193050225 >> 172.21.15.7:0/4193050225 conn(0x7fd6080fc6b0 msgr2=0x7fd6080fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.354+0000 7fd60d34b640 1 -- 172.21.15.7:0/4193050225 shutdown_connections 2024-09-10T11:00:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.354+0000 7fd60d34b640 1 -- 172.21.15.7:0/4193050225 wait complete. 2024-09-10T11:00:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.354+0000 7fd60d34b640 1 Processor -- start 2024-09-10T11:00:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd60d34b640 1 -- start start 2024-09-10T11:00:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd60d34b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6081009d0 0x7fd6081025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd60d34b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd608101370 0x7fd608102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd60d34b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6081069e0 0x7fd608103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd60d34b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd608110e70 con 0x7fd6081009d0 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd60d34b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd608110cf0 con 0x7fd6081069e0 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd60d34b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd608110ff0 con 0x7fd608101370 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd606ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6081009d0 0x7fd6081025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd606ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6081009d0 0x7fd6081025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59554/0 (socket says 172.21.15.7:59554) 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd606ffd640 1 -- 172.21.15.7:0/4143404397 learned_addr learned my addr 172.21.15.7:0/4143404397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd6077fe640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6081069e0 0x7fd608103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd6067fc640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd608101370 0x7fd608102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd606ffd640 1 -- 172.21.15.7:0/4143404397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd608101370 msgr2=0x7fd608102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd606ffd640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd608101370 0x7fd608102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd606ffd640 1 -- 172.21.15.7:0/4143404397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6081069e0 msgr2=0x7fd608103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd606ffd640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6081069e0 0x7fd608103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd606ffd640 1 -- 172.21.15.7:0/4143404397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6081a6810 con 0x7fd6081009d0 2024-09-10T11:00:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.355+0000 7fd6077fe640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6081069e0 0x7fd608103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:00:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.356+0000 7fd606ffd640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6081009d0 0x7fd6081025a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800e9f0 tx=0x7fd5f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.356+0000 7fd5effff640 1 -- 172.21.15.7:0/4143404397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800cd80 con 0x7fd6081009d0 2024-09-10T11:00:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.356+0000 7fd5effff640 1 -- 172.21.15.7:0/4143404397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f8010910 con 0x7fd6081009d0 2024-09-10T11:00:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.356+0000 7fd60d34b640 1 -- 172.21.15.7:0/4143404397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6081a6b00 con 0x7fd6081009d0 2024-09-10T11:00:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.356+0000 7fd5effff640 1 -- 172.21.15.7:0/4143404397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8018990 con 0x7fd6081009d0 2024-09-10T11:00:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.356+0000 7fd60d34b640 1 -- 172.21.15.7:0/4143404397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6081a70a0 con 0x7fd6081009d0 2024-09-10T11:00:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.359+0000 7fd5effff640 1 -- 172.21.15.7:0/4143404397 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd5f8018b30 con 0x7fd6081009d0 2024-09-10T11:00:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.359+0000 7fd6101c4740 1 Processor -- start 2024-09-10T11:00:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6101c4740 1 -- start start 2024-09-10T11:00:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e46af0 0x556a77e46ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e47410 0x556a77e51770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6101c4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556a77dab2f0 con 0x556a77e51cb0 2024-09-10T11:00:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6101c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a77d53270 con 0x556a77e46af0 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6101c4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556a77da72b0 con 0x556a77e47410 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e46af0 0x556a77e46ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59564/0 (socket says 172.21.15.7:59564) 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e46af0 0x556a77e46ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:38648/0 (socket says 172.21.15.7:38648) 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6067fc640 1 -- 172.21.15.7:0/2192780583 learned_addr learned my addr 172.21.15.7:0/2192780583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd606ffd640 1 --2- 172.21.15.7:0/2192780583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e47410 0x556a77e51770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6067fc640 1 -- 172.21.15.7:0/2192780583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e47410 msgr2=0x556a77e51770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6067fc640 1 --2- 172.21.15.7:0/2192780583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e47410 0x556a77e51770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6067fc640 1 -- 172.21.15.7:0/2192780583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e46af0 msgr2=0x556a77e46ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6067fc640 1 --2- 172.21.15.7:0/2192780583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e46af0 0x556a77e46ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd6067fc640 1 -- 172.21.15.7:0/2192780583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a77d52340 con 0x556a77e51cb0 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.360+0000 7fd5effff640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5f00a2e20 0x7fd5f00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.361+0000 7fd6067fc640 1 --2- 172.21.15.7:0/2192780583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e540a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009c90 tx=0x7fd5fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699e59b00facb175 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.361+0000 7fd6077fe640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5f00a2e20 0x7fd5f00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.361+0000 7fd5effff640 1 -- 172.21.15.7:0/4143404397 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8564+0+0 (secure 0 0 0) 0x7fd5f80cf680 con 0x7fd6081009d0 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.361+0000 7fd5ed7fa640 1 -- 172.21.15.7:0/2192780583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc01e070 con 0x556a77e51cb0 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.361+0000 7fd5ed7fa640 1 -- 172.21.15.7:0/2192780583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc019b10 con 0x556a77e51cb0 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.361+0000 7fd5ed7fa640 1 -- 172.21.15.7:0/2192780583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00eac0 con 0x556a77e51cb0 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 -- 172.21.15.7:0/2192780583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 msgr2=0x556a77e540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6077fe640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5f00a2e20 0x7fd5f00a52e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd600005e30 tx=0x7fd60003d040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 --2- 172.21.15.7:0/2192780583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e540a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009c90 tx=0x7fd5fc011810 comp rx=0 tx=0).stop 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 -- 172.21.15.7:0/2192780583 shutdown_connections 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 --2- 172.21.15.7:0/2192780583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e540a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 --2- 172.21.15.7:0/2192780583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e47410 0x556a77e51770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 --2- 172.21.15.7:0/2192780583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e46af0 0x556a77e46ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 -- 172.21.15.7:0/2192780583 >> 172.21.15.7:0/2192780583 conn(0x556a77e43060 msgr2=0x556a77e43440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 -- 172.21.15.7:0/2192780583 shutdown_connections 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 -- 172.21.15.7:0/2192780583 wait complete. 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 -- start start 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 Processor -- start 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 0x556a77e4eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 0x556a77e4f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e4f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556a77e548d0 con 0x556a77e51cb0 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a77d34290 con 0x556a77e47410 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.362+0000 7fd6101c4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556a77e54a50 con 0x556a77e46af0 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6101c4740 10 client.0 _ll_register_callbacks cb 0x556a77d65140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:00:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e4f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6067fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 0x556a77e4f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e4f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59580/0 (socket says 172.21.15.7:59580) 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6067fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 0x556a77e4f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:38658/0 (socket says 172.21.15.7:38658) 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6077fe640 1 -- 172.21.15.7:0/1259816921 learned_addr learned my addr 172.21.15.7:0/1259816921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd606ffd640 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 0x556a77e4eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6077fe640 1 -- 172.21.15.7:0/1259816921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 msgr2=0x556a77e4eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6077fe640 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 0x556a77e4eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6077fe640 1 -- 172.21.15.7:0/1259816921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 msgr2=0x556a77e4f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6077fe640 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 0x556a77e4f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6077fe640 1 -- 172.21.15.7:0/1259816921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a77abe210 con 0x556a77e51cb0 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6077fe640 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e4f5e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd600035000 tx=0x7fd60002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c08e958c9fc6d8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd5eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1259816921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600056060 con 0x556a77e51cb0 2024-09-10T11:00:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1259816921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8564+0+0 (secure 0 0 0) 0x7fd6000305a0 con 0x556a77e51cb0 2024-09-10T11:00:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.363+0000 7fd6101c4740 1 -- 172.21.15.7:0/1259816921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5fc019b10 con 0x556a77e51cb0 2024-09-10T11:00:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1259816921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd6000307a0 con 0x556a77e51cb0 2024-09-10T11:00:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd6101c4740 10 client.16413 fetch_fsmap learned FSMap version 46 2024-09-10T11:00:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd6101c4740 1 -- 172.21.15.7:0/1259816921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556a77b8a620 con 0x556a77e51cb0 2024-09-10T11:00:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1259816921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc019b10 con 0x556a77e51cb0 2024-09-10T11:00:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1259816921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fd600030070 con 0x556a77e51cb0 2024-09-10T11:00:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd6101c4740 10 client.16413 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T11:00:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd6101c4740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:00:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd6101c4740 20 client.16413 populate_metadata read hostname 'smithi007' 2024-09-10T11:00:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556a77e61c60 0x556a77e64090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd6101c4740 4 client.16413 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.364+0000 7fd6101c4740 1 -- 172.21.15.7:0/1259816921 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a77db8a20 con 0x556a77e61c60 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.366+0000 7fd606ffd640 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556a77e61c60 0x556a77e64090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.368+0000 7fd5eaff5640 10 client.16413 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.368+0000 7fd606ffd640 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556a77e61c60 0x556a77e64090 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e26c0c72bf0d24 server_cookie=187c62a33a6e8888 in_seq=0 out_seq=0 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.369+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1259816921 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a77db8a20 con 0x556a77e61c60 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.369+0000 7fd5eaff5640 10 client.16413 handle_command_reply: tid=0 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.369+0000 7fd6101c4740 1 client.16413 shutdown 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.369+0000 7fd6101c4740 1 -- 172.21.15.7:0/1259816921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 msgr2=0x556a77e4f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.369+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e4f5e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd600035000 tx=0x7fd60002a8e0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.370+0000 7fd6101c4740 1 -- 172.21.15.7:0/1259816921 shutdown_connections 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.370+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556a77e61c60 0x556a77e64090 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.370+0000 7fd5eaff5640 0 client.16413 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:19.373 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.370+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e51cb0 0x556a77e4f5e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.370+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 0x556a77e4f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.370+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1259816921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 0x556a77e4eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.370+0000 7fd6101c4740 1 -- 172.21.15.7:0/1259816921 >> 172.21.15.7:0/1259816921 conn(0x556a77e43060 msgr2=0x556a77e48d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.370+0000 7fd6101c4740 1 -- 172.21.15.7:0/1259816921 shutdown_connections 2024-09-10T11:00:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.370+0000 7fd6101c4740 1 -- 172.21.15.7:0/1259816921 wait complete. 2024-09-10T11:00:19.374 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.370+0000 7fd6101c4740 20 client.16413 trim_cache size 0 max 16384 2024-09-10T11:00:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.389+0000 7fd6101c4740 1 Processor -- start 2024-09-10T11:00:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.389+0000 7fd6101c4740 1 -- start start 2024-09-10T11:00:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.389+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e46af0 0x556a77e4f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.389+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 0x556a77e699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.389+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e51cb0 0x556a77e69ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.389+0000 7fd6101c4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081b2330 con 0x556a77e46af0 2024-09-10T11:00:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.389+0000 7fd6101c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081afb70 con 0x556a77e47410 2024-09-10T11:00:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.389+0000 7fd6101c4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081ad9e0 con 0x556a77e51cb0 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd606ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e46af0 0x556a77e4f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd606ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e46af0 0x556a77e4f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59592/0 (socket says 172.21.15.7:59592) 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd6067fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 0x556a77e699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e51cb0 0x556a77e69ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd606ffd640 1 -- 172.21.15.7:0/4220509299 learned_addr learned my addr 172.21.15.7:0/4220509299 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd6067fc640 1 -- 172.21.15.7:0/4220509299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e51cb0 msgr2=0x556a77e69ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd6067fc640 1 --2- 172.21.15.7:0/4220509299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e51cb0 0x556a77e69ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd6067fc640 1 -- 172.21.15.7:0/4220509299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e46af0 msgr2=0x556a77e4f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd6067fc640 1 --2- 172.21.15.7:0/4220509299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e46af0 0x556a77e4f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd6067fc640 1 -- 172.21.15.7:0/4220509299 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a77e607e0 con 0x556a77e47410 2024-09-10T11:00:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd606ffd640 1 --2- 172.21.15.7:0/4220509299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e46af0 0x556a77e4f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:00:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.390+0000 7fd6067fc640 1 --2- 172.21.15.7:0/4220509299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 0x556a77e699a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009be0 tx=0x7fd5fc00e6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=439f81712b5989a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.391+0000 7fd5ecff9640 1 -- 172.21.15.7:0/4220509299 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc01e050 con 0x556a77e47410 2024-09-10T11:00:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.391+0000 7fd5ecff9640 1 -- 172.21.15.7:0/4220509299 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc01f040 con 0x556a77e47410 2024-09-10T11:00:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.391+0000 7fd5ecff9640 1 -- 172.21.15.7:0/4220509299 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00e9d0 con 0x556a77e47410 2024-09-10T11:00:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 -- 172.21.15.7:0/4220509299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 msgr2=0x556a77e699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 --2- 172.21.15.7:0/4220509299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 0x556a77e699a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009be0 tx=0x7fd5fc00e6e0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 -- 172.21.15.7:0/4220509299 shutdown_connections 2024-09-10T11:00:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 --2- 172.21.15.7:0/4220509299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e51cb0 0x556a77e69ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 --2- 172.21.15.7:0/4220509299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e47410 0x556a77e699a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 --2- 172.21.15.7:0/4220509299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e46af0 0x556a77e4f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 -- 172.21.15.7:0/4220509299 >> 172.21.15.7:0/4220509299 conn(0x556a77e43060 msgr2=0x556a77e50be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 -- 172.21.15.7:0/4220509299 shutdown_connections 2024-09-10T11:00:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 -- 172.21.15.7:0/4220509299 wait complete. 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 -- start start 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.392+0000 7fd6101c4740 1 Processor -- start 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 0x556a77e5a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e47410 0x556a77e55860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6101c4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e51cb0 0x556a77e55e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6101c4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081ae030 con 0x556a77e47410 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6101c4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081af4a0 con 0x556a77e51cb0 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6101c4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081ae700 con 0x556a77e46af0 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6101c4740 10 client.0 _ll_register_callbacks cb 0x7fd600009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e51cb0 0x556a77e55e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd606ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 0x556a77e5a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6067fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e47410 0x556a77e55860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e51cb0 0x556a77e55e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:38668/0 (socket says 172.21.15.7:38668) 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd606ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 0x556a77e5a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48216/0 (socket says 172.21.15.7:48216) 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6077fe640 1 -- 172.21.15.7:0/1354557451 learned_addr learned my addr 172.21.15.7:0/1354557451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6077fe640 1 -- 172.21.15.7:0/1354557451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 msgr2=0x556a77e5a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6077fe640 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 0x556a77e5a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6077fe640 1 -- 172.21.15.7:0/1354557451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e47410 msgr2=0x556a77e55860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6077fe640 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e47410 0x556a77e55860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6077fe640 1 -- 172.21.15.7:0/1354557451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a77e56680 con 0x556a77e51cb0 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6067fc640 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e47410 0x556a77e55860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd6077fe640 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e51cb0 0x556a77e55e50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd600006010 tx=0x7fd600009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d80aa6db6679fc0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:19.400 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.393+0000 7fd5eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1354557451 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600056040 con 0x556a77e51cb0 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd6101c4740 1 -- 172.21.15.7:0/1354557451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5fc01f040 con 0x556a77e51cb0 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1354557451 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8564+0+0 (secure 0 0 0) 0x7fd6000305a0 con 0x556a77e51cb0 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1354557451 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd60002af20 con 0x556a77e51cb0 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd6101c4740 10 client.26027 fetch_fsmap learned FSMap version 46 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd6101c4740 1 -- 172.21.15.7:0/1354557451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556a77ecca90 con 0x556a77e51cb0 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1354557451 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5fc01f040 con 0x556a77e51cb0 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1354557451 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fd600030040 con 0x556a77e51cb0 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd6101c4740 10 client.26027 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd6101c4740 10 client.26027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd6101c4740 20 client.26027 populate_metadata read hostname 'smithi007' 2024-09-10T11:00:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556a77e61c60 0x556a77ecf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd6101c4740 4 client.26027 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T11:00:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.394+0000 7fd6101c4740 1 -- 172.21.15.7:0/1354557451 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a77db2d70 con 0x556a77e61c60 2024-09-10T11:00:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.395+0000 7fd606ffd640 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556a77e61c60 0x556a77ecf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.395+0000 7fd5eaff5640 10 client.26027 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:19.402 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.395+0000 7fd606ffd640 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556a77e61c60 0x556a77ecf280 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a708703c99e21f83 server_cookie=181cea66c44ed129 in_seq=0 out_seq=0 2024-09-10T11:00:19.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 54754 39.49 MB/sec warmup 79 sec latency 403.783 ms 2024-09-10T11:00:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:19 smithi103 ceph-mon[35160]: pgmap v455: 97 pgs: 97 active+clean; 731 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-09-10T11:00:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:19 smithi103 ceph-mon[35160]: osdmap e103: 12 total, 12 up, 12 in 2024-09-10T11:00:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:19 smithi007 ceph-mon[30873]: pgmap v455: 97 pgs: 97 active+clean; 731 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-09-10T11:00:19.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:19 smithi007 ceph-mon[30873]: osdmap e103: 12 total, 12 up, 12 in 2024-09-10T11:00:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd5eaff5640 1 -- 172.21.15.7:0/1354557451 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a77db2d70 con 0x556a77e61c60 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd5eaff5640 10 client.26027 handle_command_reply: tid=0 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 client.26027 shutdown 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 -- 172.21.15.7:0/1354557451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e51cb0 msgr2=0x556a77e55e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e51cb0 0x556a77e55e50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd600006010 tx=0x7fd600009bd0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 -- 172.21.15.7:0/1354557451 shutdown_connections 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x556a77e61c60 0x556a77ecf280 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd5eaff5640 0 client.26027 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a77e51cb0 0x556a77e55e50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556a77e47410 0x556a77e55860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 --2- 172.21.15.7:0/1354557451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a77e46af0 0x556a77e5a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 -- 172.21.15.7:0/1354557451 >> 172.21.15.7:0/1354557451 conn(0x556a77e43060 msgr2=0x556a77e419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 -- 172.21.15.7:0/1354557451 shutdown_connections 2024-09-10T11:00:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 1 -- 172.21.15.7:0/1354557451 wait complete. 2024-09-10T11:00:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.973+0000 7fd6101c4740 20 client.26027 trim_cache size 0 max 16384 2024-09-10T11:00:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T11:00:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T11:00:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T11:00:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T11:00:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.974+0000 7fd60d34b640 1 -- 172.21.15.7:0/4143404397 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5f00a2e20 msgr2=0x7fd5f00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.974+0000 7fd60d34b640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5f00a2e20 0x7fd5f00a52e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd600005e30 tx=0x7fd60003d040 comp rx=0 tx=0).stop 2024-09-10T11:00:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.974+0000 7fd60d34b640 1 -- 172.21.15.7:0/4143404397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6081009d0 msgr2=0x7fd6081025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.974+0000 7fd60d34b640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6081009d0 0x7fd6081025a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800e9f0 tx=0x7fd5f800eeb0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.974+0000 7fd60d34b640 1 -- 172.21.15.7:0/4143404397 shutdown_connections 2024-09-10T11:00:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.974+0000 7fd60d34b640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd6081069e0 0x7fd608103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.974+0000 7fd60d34b640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd608101370 0x7fd608102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.974+0000 7fd60d34b640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd5f00a2e20 0x7fd5f00a52e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.974+0000 7fd60d34b640 1 --2- 172.21.15.7:0/4143404397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6081009d0 0x7fd6081025a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.974+0000 7fd60d34b640 1 -- 172.21.15.7:0/4143404397 >> 172.21.15.7:0/4143404397 conn(0x7fd6080fc6b0 msgr2=0x7fd6081046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.975+0000 7fd60d34b640 1 -- 172.21.15.7:0/4143404397 shutdown_connections 2024-09-10T11:00:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:19.975+0000 7fd60d34b640 1 -- 172.21.15.7:0/4143404397 wait complete. 2024-09-10T11:00:19.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:19 smithi133 ceph-mon[35282]: pgmap v455: 97 pgs: 97 active+clean; 731 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 1.00k op/s 2024-09-10T11:00:19.978 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:19 smithi133 ceph-mon[35282]: osdmap e103: 12 total, 12 up, 12 in 2024-09-10T11:00:20.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 55657 39.77 MB/sec warmup 80 sec latency 497.458 ms 2024-09-10T11:00:21.225 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 11:00:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: ignoring --setuser ceph since I am not root 2024-09-10T11:00:21.225 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 11:00:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: ignoring --setgroup ceph since I am not root 2024-09-10T11:00:21.225 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 11:00:20 smithi133 ceph-mds[65473]: -- 172.21.15.133:0/1353834922 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dae6ab5860 con 0x55dae6a92800 2024-09-10T11:00:21.225 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 11:00:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: starting mds.f at 2024-09-10T11:00:21.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 56344 39.71 MB/sec warmup 81 sec latency 95.132 ms 2024-09-10T11:00:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:21 smithi103 ceph-mon[35160]: pgmap v457: 97 pgs: 97 active+clean; 763 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 976 op/s 2024-09-10T11:00:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:21 smithi103 ceph-mon[35160]: daemon mds.f restarted 2024-09-10T11:00:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:21 smithi007 ceph-mon[30873]: pgmap v457: 97 pgs: 97 active+clean; 763 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 976 op/s 2024-09-10T11:00:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:21 smithi007 ceph-mon[30873]: daemon mds.f restarted 2024-09-10T11:00:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:21 smithi133 ceph-mon[35282]: pgmap v457: 97 pgs: 97 active+clean; 763 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 976 op/s 2024-09-10T11:00:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:21 smithi133 ceph-mon[35282]: daemon mds.f restarted 2024-09-10T11:00:22.884 INFO:tasks.workunit.client.0.smithi007.stdout: 10 57062 39.64 MB/sec warmup 82 sec latency 48.214 ms 2024-09-10T11:00:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:22 smithi103 ceph-mon[35160]: osdmap e104: 12 total, 12 up, 12 in 2024-09-10T11:00:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:22 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6834/364818207,v1:172.21.15.133:6835/364818207] up:boot 2024-09-10T11:00:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:22 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T11:00:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:22 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T11:00:22.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:22 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:00:22.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:22 smithi007 ceph-mon[30873]: osdmap e104: 12 total, 12 up, 12 in 2024-09-10T11:00:22.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:22 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6834/364818207,v1:172.21.15.133:6835/364818207] up:boot 2024-09-10T11:00:22.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:22 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T11:00:22.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:22 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T11:00:22.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:22 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:00:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:22 smithi133 ceph-mon[35282]: osdmap e104: 12 total, 12 up, 12 in 2024-09-10T11:00:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:22 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6834/364818207,v1:172.21.15.133:6835/364818207] up:boot 2024-09-10T11:00:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:22 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-10T11:00:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:22 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T11:00:22.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:22 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:00:23.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 57899 39.63 MB/sec warmup 83 sec latency 101.023 ms 2024-09-10T11:00:23.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:23 smithi103 ceph-mon[35160]: pgmap v459: 97 pgs: 97 active+clean; 763 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 20 MiB/s wr, 1.22k op/s 2024-09-10T11:00:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:23 smithi007 ceph-mon[30873]: pgmap v459: 97 pgs: 97 active+clean; 763 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 20 MiB/s wr, 1.22k op/s 2024-09-10T11:00:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:23 smithi133 ceph-mon[35282]: pgmap v459: 97 pgs: 97 active+clean; 763 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 20 MiB/s wr, 1.22k op/s 2024-09-10T11:00:24.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 58745 39.71 MB/sec warmup 84 sec latency 233.494 ms 2024-09-10T11:00:25.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 59543 39.82 MB/sec warmup 85 sec latency 285.748 ms 2024-09-10T11:00:25.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:25 smithi103 ceph-mon[35160]: pgmap v460: 97 pgs: 97 active+clean; 763 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 703 op/s 2024-09-10T11:00:25.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:25 smithi007 ceph-mon[30873]: pgmap v460: 97 pgs: 97 active+clean; 763 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 703 op/s 2024-09-10T11:00:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:25 smithi133 ceph-mon[35282]: pgmap v460: 97 pgs: 97 active+clean; 763 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 703 op/s 2024-09-10T11:00:26.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 60357 39.92 MB/sec warmup 86 sec latency 46.105 ms 2024-09-10T11:00:27.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 61223 39.93 MB/sec warmup 87 sec latency 53.838 ms 2024-09-10T11:00:27.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:27 smithi103 ceph-mon[35160]: pgmap v461: 97 pgs: 97 active+clean; 777 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 21 MiB/s wr, 1.26k op/s 2024-09-10T11:00:27.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:27 smithi007 ceph-mon[30873]: pgmap v461: 97 pgs: 97 active+clean; 777 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 21 MiB/s wr, 1.26k op/s 2024-09-10T11:00:27.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:27 smithi133 ceph-mon[35282]: pgmap v461: 97 pgs: 97 active+clean; 777 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 21 MiB/s wr, 1.26k op/s 2024-09-10T11:00:28.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 62184 39.99 MB/sec warmup 88 sec latency 36.366 ms 2024-09-10T11:00:29.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 63082 40.18 MB/sec warmup 89 sec latency 128.842 ms 2024-09-10T11:00:29.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:29 smithi103 ceph-mon[35160]: pgmap v462: 97 pgs: 97 active+clean; 777 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-09-10T11:00:29.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:29 smithi007 ceph-mon[30873]: pgmap v462: 97 pgs: 97 active+clean; 777 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-09-10T11:00:29.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:29 smithi133 ceph-mon[35282]: pgmap v462: 97 pgs: 97 active+clean; 777 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-09-10T11:00:30.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 63714 40.17 MB/sec warmup 90 sec latency 192.364 ms 2024-09-10T11:00:31.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 64385 40.04 MB/sec warmup 91 sec latency 161.900 ms 2024-09-10T11:00:31.924 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:31 smithi103 ceph-mon[35160]: pgmap v463: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 848 op/s 2024-09-10T11:00:31.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:31 smithi007 ceph-mon[30873]: pgmap v463: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 848 op/s 2024-09-10T11:00:31.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:31 smithi133 ceph-mon[35282]: pgmap v463: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 848 op/s 2024-09-10T11:00:32.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 65309 40.11 MB/sec warmup 92 sec latency 63.639 ms 2024-09-10T11:00:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.075+0000 7f6c43ed2640 1 -- 172.21.15.7:0/3640628867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c100890 msgr2=0x7f6c3c100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.075+0000 7f6c43ed2640 1 --2- 172.21.15.7:0/3640628867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c100890 0x7f6c3c100c70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009900 tx=0x7f6c3802a410 comp rx=0 tx=0).stop 2024-09-10T11:00:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.075+0000 7f6c43ed2640 1 -- 172.21.15.7:0/3640628867 shutdown_connections 2024-09-10T11:00:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.075+0000 7f6c43ed2640 1 --2- 172.21.15.7:0/3640628867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10e440 0x7f6c3c110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.075+0000 7f6c43ed2640 1 --2- 172.21.15.7:0/3640628867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c101240 0x7f6c3c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.075+0000 7f6c43ed2640 1 --2- 172.21.15.7:0/3640628867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c100890 0x7f6c3c100c70 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.075+0000 7f6c43ed2640 1 -- 172.21.15.7:0/3640628867 >> 172.21.15.7:0/3640628867 conn(0x7f6c3c077990 msgr2=0x7f6c3c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.075+0000 7f6c43ed2640 1 -- 172.21.15.7:0/3640628867 shutdown_connections 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.075+0000 7f6c43ed2640 1 -- 172.21.15.7:0/3640628867 wait complete. 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c43ed2640 1 Processor -- start 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c43ed2640 1 -- start start 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c43ed2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c100890 0x7f6c3c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c43ed2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c101240 0x7f6c3c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c43ed2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10e440 0x7f6c3c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c43ed2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1140c0 con 0x7f6c3c100890 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c43ed2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c113f40 con 0x7f6c3c10e440 2024-09-10T11:00:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c43ed2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c114240 con 0x7f6c3c101240 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c41c6e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c100890 0x7f6c3c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c41c6e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c100890 0x7f6c3c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47634/0 (socket says 172.21.15.7:47634) 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c41c6e640 1 -- 172.21.15.7:0/1568554762 learned_addr learned my addr 172.21.15.7:0/1568554762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c4146d640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c101240 0x7f6c3c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.076+0000 7f6c4246f640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10e440 0x7f6c3c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c41c6e640 1 -- 172.21.15.7:0/1568554762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c101240 msgr2=0x7f6c3c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c101240 0x7f6c3c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c41c6e640 1 -- 172.21.15.7:0/1568554762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10e440 msgr2=0x7f6c3c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10e440 0x7f6c3c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c41c6e640 1 -- 172.21.15.7:0/1568554762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c19f580 con 0x7f6c3c100890 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c100890 0x7f6c3c19a590 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6c38006010 tx=0x7f6c38004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c32ffd640 1 -- 172.21.15.7:0/1568554762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3803f030 con 0x7f6c3c100890 2024-09-10T11:00:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c32ffd640 1 -- 172.21.15.7:0/1568554762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c38004440 con 0x7f6c3c100890 2024-09-10T11:00:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c43ed2640 1 -- 172.21.15.7:0/1568554762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c19f810 con 0x7f6c3c100890 2024-09-10T11:00:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c32ffd640 1 -- 172.21.15.7:0/1568554762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38035070 con 0x7f6c3c100890 2024-09-10T11:00:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.077+0000 7f6c43ed2640 1 -- 172.21.15.7:0/1568554762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c1a7100 con 0x7f6c3c100890 2024-09-10T11:00:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.079+0000 7f6c32ffd640 1 -- 172.21.15.7:0/1568554762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6c38042030 con 0x7f6c3c100890 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c46c63740 1 Processor -- start 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c46c63740 1 -- start start 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c32ffd640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c280a2da0 0x7f6c280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a84af0 0x557dd7a84ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a85410 0x557dd7a8f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7a920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c46c63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557dd79e92f0 con 0x557dd7a84af0 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c46c63740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557dd7991270 con 0x557dd7a85410 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c46c63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557dd79e52b0 con 0x557dd7a8fcb0 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c32ffd640 1 -- 172.21.15.7:0/1568554762 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8520+0+0 (secure 0 0 0) 0x7f6c380ef6b0 con 0x7f6c3c100890 2024-09-10T11:00:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.080+0000 7f6c4246f640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c280a2da0 0x7f6c280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c41c6e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a84af0 0x557dd7a84ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c41c6e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a84af0 0x557dd7a84ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47640/0 (socket says 172.21.15.7:47640) 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c41c6e640 1 -- 172.21.15.7:0/2183021934 learned_addr learned my addr 172.21.15.7:0/2183021934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c4146d640 1 --2- 172.21.15.7:0/2183021934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a85410 0x557dd7a8f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c4246f640 1 --2- 172.21.15.7:0/2183021934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7a920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c41c6e640 1 -- 172.21.15.7:0/2183021934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 msgr2=0x557dd7a920a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/2183021934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7a920a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c41c6e640 1 -- 172.21.15.7:0/2183021934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a85410 msgr2=0x557dd7a8f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/2183021934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a85410 0x557dd7a8f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c41c6e640 1 -- 172.21.15.7:0/2183021934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dd7990340 con 0x557dd7a84af0 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c4246f640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c280a2da0 0x7f6c280a5260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c074a00 tx=0x7f6c34013b00 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/2183021934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a84af0 0x557dd7a84ed0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6c380ed400 tx=0x7f6c380b7fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d83ae7bea5c50f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c27fff640 1 -- 172.21.15.7:0/2183021934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38049070 con 0x557dd7a84af0 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c27fff640 1 -- 172.21.15.7:0/2183021934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c380bc2d0 con 0x557dd7a84af0 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.081+0000 7f6c27fff640 1 -- 172.21.15.7:0/2183021934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c380b8240 con 0x557dd7a84af0 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.082+0000 7f6c46c63740 1 -- 172.21.15.7:0/2183021934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a84af0 msgr2=0x557dd7a84ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.082+0000 7f6c46c63740 1 --2- 172.21.15.7:0/2183021934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a84af0 0x557dd7a84ed0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6c380ed400 tx=0x7f6c380b7fa0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.082+0000 7f6c46c63740 1 -- 172.21.15.7:0/2183021934 shutdown_connections 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.082+0000 7f6c46c63740 1 --2- 172.21.15.7:0/2183021934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7a920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.082+0000 7f6c46c63740 1 --2- 172.21.15.7:0/2183021934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a85410 0x557dd7a8f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.082+0000 7f6c46c63740 1 --2- 172.21.15.7:0/2183021934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a84af0 0x557dd7a84ed0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.082+0000 7f6c46c63740 1 -- 172.21.15.7:0/2183021934 >> 172.21.15.7:0/2183021934 conn(0x557dd7a81060 msgr2=0x557dd7a81440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.082+0000 7f6c46c63740 1 -- 172.21.15.7:0/2183021934 shutdown_connections 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.082+0000 7f6c46c63740 1 -- 172.21.15.7:0/2183021934 wait complete. 2024-09-10T11:00:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c46c63740 1 -- start start 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c46c63740 1 Processor -- start 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 0x557dd7a8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7a8d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7a8d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c46c63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557dd7a92640 con 0x557dd7a85410 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c46c63740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557dd7972290 con 0x557dd7a84af0 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c46c63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557dd7a927c0 con 0x557dd7a8fcb0 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c46c63740 10 client.0 _ll_register_callbacks cb 0x557dd79a3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c41c6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 0x557dd7a8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c41c6e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 0x557dd7a8cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:41036/0 (socket says 172.21.15.7:41036) 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c4146d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7a8d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c41c6e640 1 -- 172.21.15.7:0/569647158 learned_addr learned my addr 172.21.15.7:0/569647158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.083+0000 7f6c4246f640 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7a8d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c41c6e640 1 -- 172.21.15.7:0/569647158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 msgr2=0x557dd7a8d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7a8d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c41c6e640 1 -- 172.21.15.7:0/569647158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 msgr2=0x557dd7a8d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7a8d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c41c6e640 1 -- 172.21.15.7:0/569647158 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dd76fc210 con 0x557dd7a84af0 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c4246f640 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7a8d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 0x557dd7a8cb00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6c380b85a0 tx=0x7f6c380bff80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c5711d0bf6da0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c257fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c257fa640 1 -- 172.21.15.7:0/569647158 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38049040 con 0x557dd7a84af0 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c257fa640 1 -- 172.21.15.7:0/569647158 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8520+0+0 (secure 0 0 0) 0x7f6c380b8240 con 0x557dd7a84af0 2024-09-10T11:00:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c46c63740 1 -- 172.21.15.7:0/569647158 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c380bc2d0 con 0x557dd7a84af0 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c257fa640 1 -- 172.21.15.7:0/569647158 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6c380bbbd0 con 0x557dd7a84af0 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c46c63740 10 client.26045 fetch_fsmap learned FSMap version 48 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c46c63740 1 -- 172.21.15.7:0/569647158 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557dd77c8620 con 0x557dd7a84af0 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.084+0000 7f6c257fa640 1 -- 172.21.15.7:0/569647158 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c380bc2d0 con 0x557dd7a84af0 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.085+0000 7f6c257fa640 1 -- 172.21.15.7:0/569647158 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f6c380c4440 con 0x557dd7a84af0 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.085+0000 7f6c46c63740 10 client.26045 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.085+0000 7f6c46c63740 10 client.26045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.085+0000 7f6c46c63740 20 client.26045 populate_metadata read hostname 'smithi007' 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.085+0000 7f6c46c63740 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557dd7a9f700 0x557dd7aa1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.085+0000 7f6c46c63740 4 client.26045 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.085+0000 7f6c46c63740 1 -- 172.21.15.7:0/569647158 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dd79f6a20 con 0x557dd7a9f700 2024-09-10T11:00:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.085+0000 7f6c4146d640 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557dd7a9f700 0x557dd7aa1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.086+0000 7f6c4146d640 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557dd7a9f700 0x557dd7aa1af0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6b3f7ca27f9d6 server_cookie=d0350569511bf30e in_seq=0 out_seq=0 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.086+0000 7f6c257fa640 10 client.26045 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c257fa640 1 -- 172.21.15.7:0/569647158 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dd79f6a20 con 0x557dd7a9f700 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c257fa640 10 client.26045 handle_command_reply: tid=0 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 client.26045 shutdown 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 -- 172.21.15.7:0/569647158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 msgr2=0x557dd7a8cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 0x557dd7a8cb00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6c380b85a0 tx=0x7f6c380bff80 comp rx=0 tx=0).stop 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 -- 172.21.15.7:0/569647158 shutdown_connections 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557dd7a9f700 0x557dd7aa1af0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7a8d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7a8d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c257fa640 0 client.26045 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 --2- 172.21.15.7:0/569647158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 0x557dd7a8cb00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 -- 172.21.15.7:0/569647158 >> 172.21.15.7:0/569647158 conn(0x557dd7a81060 msgr2=0x557dd7a86dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 -- 172.21.15.7:0/569647158 shutdown_connections 2024-09-10T11:00:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 1 -- 172.21.15.7:0/569647158 wait complete. 2024-09-10T11:00:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.090+0000 7f6c46c63740 20 client.26045 trim_cache size 0 max 16384 2024-09-10T11:00:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.106+0000 7f6c46c63740 1 Processor -- start 2024-09-10T11:00:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.106+0000 7f6c46c63740 1 -- start start 2024-09-10T11:00:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 0x557dd7a8d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7aa7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7aa7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c46c63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1b22e0 con 0x557dd7a85410 2024-09-10T11:00:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c46c63740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1afb20 con 0x557dd7a84af0 2024-09-10T11:00:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c46c63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1ad970 con 0x557dd7a8fcb0 2024-09-10T11:00:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c4146d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7aa7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c4146d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7aa7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47656/0 (socket says 172.21.15.7:47656) 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c4146d640 1 -- 172.21.15.7:0/195977294 learned_addr learned my addr 172.21.15.7:0/195977294 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/195977294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 0x557dd7a8d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c4246f640 1 --2- 172.21.15.7:0/195977294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7aa7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c4146d640 1 -- 172.21.15.7:0/195977294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 msgr2=0x557dd7aa7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c4146d640 1 --2- 172.21.15.7:0/195977294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7aa7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c4146d640 1 -- 172.21.15.7:0/195977294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 msgr2=0x557dd7a8d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c4146d640 1 --2- 172.21.15.7:0/195977294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 0x557dd7a8d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c4146d640 1 -- 172.21.15.7:0/195977294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dd7a80e90 con 0x557dd7a85410 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.107+0000 7f6c4146d640 1 --2- 172.21.15.7:0/195977294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7aa7210 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x557dd7a81ec0 tx=0x7f6c2c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c854c85ecb0da4cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c277fe640 1 -- 172.21.15.7:0/195977294 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c022070 con 0x557dd7a85410 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c277fe640 1 -- 172.21.15.7:0/195977294 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2c0047a0 con 0x557dd7a85410 2024-09-10T11:00:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c277fe640 1 -- 172.21.15.7:0/195977294 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c01f8f0 con 0x557dd7a85410 2024-09-10T11:00:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c46c63740 1 -- 172.21.15.7:0/195977294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 msgr2=0x557dd7aa7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c46c63740 1 --2- 172.21.15.7:0/195977294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7aa7210 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x557dd7a81ec0 tx=0x7f6c2c005f90 comp rx=0 tx=0).stop 2024-09-10T11:00:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c46c63740 1 -- 172.21.15.7:0/195977294 shutdown_connections 2024-09-10T11:00:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c46c63740 1 --2- 172.21.15.7:0/195977294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a8fcb0 0x557dd7aa7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c46c63740 1 --2- 172.21.15.7:0/195977294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7aa7210 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c46c63740 1 --2- 172.21.15.7:0/195977294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a84af0 0x557dd7a8d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c46c63740 1 -- 172.21.15.7:0/195977294 >> 172.21.15.7:0/195977294 conn(0x557dd7a81060 msgr2=0x557dd7a8eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.108+0000 7f6c46c63740 1 -- 172.21.15.7:0/195977294 shutdown_connections 2024-09-10T11:00:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.109+0000 7f6c46c63740 1 -- 172.21.15.7:0/195977294 wait complete. 2024-09-10T11:00:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.109+0000 7f6c46c63740 1 -- start start 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.109+0000 7f6c46c63740 1 Processor -- start 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.109+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a84af0 0x557dd7a981f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.109+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7a89810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.109+0000 7f6c46c63740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a8fcb0 0x557dd7a89d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.109+0000 7f6c46c63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1adfe0 con 0x557dd7a85410 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.109+0000 7f6c46c63740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1af450 con 0x557dd7a8fcb0 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.109+0000 7f6c46c63740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c1ae6b0 con 0x557dd7a84af0 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.109+0000 7f6c46c63740 10 client.0 _ll_register_callbacks cb 0x7f6c380ecd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4146d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7a89810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4246f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a8fcb0 0x557dd7a89d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c41c6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a84af0 0x557dd7a981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4246f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a8fcb0 0x557dd7a89d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:41052/0 (socket says 172.21.15.7:41052) 2024-09-10T11:00:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4146d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7a89810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47658/0 (socket says 172.21.15.7:47658) 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4246f640 1 -- 172.21.15.7:0/3422913701 learned_addr learned my addr 172.21.15.7:0/3422913701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c41c6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a84af0 0x557dd7a981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:35562/0 (socket says 172.21.15.7:35562) 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4246f640 1 -- 172.21.15.7:0/3422913701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a84af0 msgr2=0x557dd7a981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4246f640 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a84af0 0x557dd7a981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4246f640 1 -- 172.21.15.7:0/3422913701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 msgr2=0x557dd7a89810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4246f640 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7a89810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4246f640 1 -- 172.21.15.7:0/3422913701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dd7a81cf0 con 0x557dd7a8fcb0 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4146d640 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7a89810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c4246f640 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a8fcb0 0x557dd7a89d70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c34003ed0 tx=0x7f6c3401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=114b6d4898ba03d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c257fa640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c257fa640 1 -- 172.21.15.7:0/3422913701 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34004240 con 0x557dd7a8fcb0 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c257fa640 1 -- 172.21.15.7:0/3422913701 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8520+0+0 (secure 0 0 0) 0x7f6c340043e0 con 0x557dd7a8fcb0 2024-09-10T11:00:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.110+0000 7f6c46c63740 1 -- 172.21.15.7:0/3422913701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2c0047a0 con 0x557dd7a8fcb0 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c257fa640 1 -- 172.21.15.7:0/3422913701 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6c340045e0 con 0x557dd7a8fcb0 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c46c63740 10 client.26051 fetch_fsmap learned FSMap version 48 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c46c63740 1 -- 172.21.15.7:0/3422913701 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x557dd7b0a1e0 con 0x557dd7a8fcb0 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c257fa640 1 -- 172.21.15.7:0/3422913701 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2c0047a0 con 0x557dd7a8fcb0 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c257fa640 1 -- 172.21.15.7:0/3422913701 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f6c3401f070 con 0x557dd7a8fcb0 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c46c63740 10 client.26051 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c46c63740 10 client.26051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c46c63740 20 client.26051 populate_metadata read hostname 'smithi007' 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c46c63740 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557dd7a9f700 0x557dd7b0c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c46c63740 4 client.26051 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c46c63740 1 -- 172.21.15.7:0/3422913701 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557dd79f0d70 con 0x557dd7a9f700 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.111+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557dd7a9f700 0x557dd7b0c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.112+0000 7f6c257fa640 10 client.26051 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.112+0000 7f6c41c6e640 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557dd7a9f700 0x557dd7b0c930 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db7247bc69c1a393 server_cookie=4be7e1b21986a7e9 in_seq=0 out_seq=0 2024-09-10T11:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c257fa640 1 -- 172.21.15.7:0/3422913701 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557dd79f0d70 con 0x557dd7a9f700 2024-09-10T11:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c257fa640 10 client.26051 handle_command_reply: tid=0 2024-09-10T11:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c46c63740 1 client.26051 shutdown 2024-09-10T11:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c46c63740 1 -- 172.21.15.7:0/3422913701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a8fcb0 msgr2=0x557dd7a89d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c46c63740 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a8fcb0 0x557dd7a89d70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c34003ed0 tx=0x7f6c3401b000 comp rx=0 tx=0).stop 2024-09-10T11:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c46c63740 1 -- 172.21.15.7:0/3422913701 shutdown_connections 2024-09-10T11:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c46c63740 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x557dd7a9f700 0x557dd7b0c930 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c46c63740 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557dd7a8fcb0 0x557dd7a89d70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c46c63740 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dd7a85410 0x557dd7a89810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c257fa640 0 client.26051 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c46c63740 1 --2- 172.21.15.7:0/3422913701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dd7a84af0 0x557dd7a981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.424+0000 7f6c46c63740 1 -- 172.21.15.7:0/3422913701 >> 172.21.15.7:0/3422913701 conn(0x557dd7a81060 msgr2=0x557dd7a86550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c46c63740 1 -- 172.21.15.7:0/3422913701 shutdown_connections 2024-09-10T11:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c46c63740 1 -- 172.21.15.7:0/3422913701 wait complete. 2024-09-10T11:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c46c63740 20 client.26051 trim_cache size 0 max 16384 2024-09-10T11:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T11:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T11:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T11:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T11:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c43ed2640 1 -- 172.21.15.7:0/1568554762 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c280a2da0 msgr2=0x7f6c280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c43ed2640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c280a2da0 0x7f6c280a5260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c074a00 tx=0x7f6c34013b00 comp rx=0 tx=0).stop 2024-09-10T11:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c43ed2640 1 -- 172.21.15.7:0/1568554762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c100890 msgr2=0x7f6c3c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c43ed2640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c100890 0x7f6c3c19a590 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6c38006010 tx=0x7f6c38004040 comp rx=0 tx=0).stop 2024-09-10T11:00:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c43ed2640 1 -- 172.21.15.7:0/1568554762 shutdown_connections 2024-09-10T11:00:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c43ed2640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6c3c10e440 0x7f6c3c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c43ed2640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c3c101240 0x7f6c3c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c43ed2640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f6c280a2da0 0x7f6c280a5260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c43ed2640 1 --2- 172.21.15.7:0/1568554762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c100890 0x7f6c3c19a590 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.425+0000 7f6c43ed2640 1 -- 172.21.15.7:0/1568554762 >> 172.21.15.7:0/1568554762 conn(0x7f6c3c077990 msgr2=0x7f6c3c0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.426+0000 7f6c43ed2640 1 -- 172.21.15.7:0/1568554762 shutdown_connections 2024-09-10T11:00:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:33.426+0000 7f6c43ed2640 1 -- 172.21.15.7:0/1568554762 wait complete. 2024-09-10T11:00:33.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 66072 40.14 MB/sec warmup 93 sec latency 150.501 ms 2024-09-10T11:00:33.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:33 smithi103 ceph-mon[35160]: pgmap v464: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 818 op/s 2024-09-10T11:00:33.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:33 smithi007 ceph-mon[30873]: pgmap v464: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 818 op/s 2024-09-10T11:00:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:33 smithi133 ceph-mon[35282]: pgmap v464: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 818 op/s 2024-09-10T11:00:34.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 66950 40.25 MB/sec warmup 94 sec latency 53.992 ms 2024-09-10T11:00:35.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 67858 40.35 MB/sec warmup 95 sec latency 81.739 ms 2024-09-10T11:00:35.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:35 smithi103 ceph-mon[35160]: pgmap v465: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 707 op/s 2024-09-10T11:00:35.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:35 smithi007 ceph-mon[30873]: pgmap v465: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 707 op/s 2024-09-10T11:00:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:35 smithi133 ceph-mon[35282]: pgmap v465: 97 pgs: 97 active+clean; 797 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 707 op/s 2024-09-10T11:00:36.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 68757 40.48 MB/sec warmup 96 sec latency 128.163 ms 2024-09-10T11:00:37.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 69592 40.43 MB/sec warmup 97 sec latency 54.120 ms 2024-09-10T11:00:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:37 smithi103 ceph-mon[35160]: pgmap v466: 97 pgs: 97 active+clean; 769 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-09-10T11:00:37.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:37 smithi007 ceph-mon[30873]: pgmap v466: 97 pgs: 97 active+clean; 769 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-09-10T11:00:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:37 smithi133 ceph-mon[35282]: pgmap v466: 97 pgs: 97 active+clean; 769 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 16 MiB/s wr, 1.11k op/s 2024-09-10T11:00:38.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 70559 40.58 MB/sec warmup 98 sec latency 109.935 ms 2024-09-10T11:00:39.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 71522 40.72 MB/sec warmup 99 sec latency 81.210 ms 2024-09-10T11:00:39.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:39 smithi103 ceph-mon[35160]: pgmap v467: 97 pgs: 97 active+clean; 769 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 740 op/s 2024-09-10T11:00:39.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:39 smithi007 ceph-mon[30873]: pgmap v467: 97 pgs: 97 active+clean; 769 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 740 op/s 2024-09-10T11:00:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:39 smithi133 ceph-mon[35282]: pgmap v467: 97 pgs: 97 active+clean; 769 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 740 op/s 2024-09-10T11:00:40.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 72202 40.68 MB/sec warmup 100 sec latency 198.137 ms 2024-09-10T11:00:41.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 72992 40.63 MB/sec warmup 101 sec latency 69.057 ms 2024-09-10T11:00:41.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:41 smithi007 ceph-mon[30873]: pgmap v468: 97 pgs: 97 active+clean; 800 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 13 MiB/s wr, 995 op/s 2024-09-10T11:00:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:41 smithi133 ceph-mon[35282]: pgmap v468: 97 pgs: 97 active+clean; 800 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 13 MiB/s wr, 995 op/s 2024-09-10T11:00:42.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:41 smithi103 ceph-mon[35160]: pgmap v468: 97 pgs: 97 active+clean; 800 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 13 MiB/s wr, 995 op/s 2024-09-10T11:00:42.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 73932 40.69 MB/sec warmup 102 sec latency 112.642 ms 2024-09-10T11:00:43.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 75131 41.09 MB/sec warmup 103 sec latency 55.870 ms 2024-09-10T11:00:43.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:43 smithi007 ceph-mon[30873]: pgmap v469: 97 pgs: 97 active+clean; 800 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 658 op/s 2024-09-10T11:00:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:43 smithi133 ceph-mon[35282]: pgmap v469: 97 pgs: 97 active+clean; 800 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 658 op/s 2024-09-10T11:00:44.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:43 smithi103 ceph-mon[35160]: pgmap v469: 97 pgs: 97 active+clean; 800 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 658 op/s 2024-09-10T11:00:44.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 75735 41.00 MB/sec warmup 104 sec latency 51.112 ms 2024-09-10T11:00:45.885 INFO:tasks.workunit.client.0.smithi007.stdout: 10 76103 40.73 MB/sec warmup 105 sec latency 233.293 ms 2024-09-10T11:00:45.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:45 smithi007 ceph-mon[30873]: pgmap v470: 97 pgs: 97 active+clean; 800 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 658 op/s 2024-09-10T11:00:45.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:45 smithi133 ceph-mon[35282]: pgmap v470: 97 pgs: 97 active+clean; 800 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 658 op/s 2024-09-10T11:00:46.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:45 smithi103 ceph-mon[35160]: pgmap v470: 97 pgs: 97 active+clean; 800 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 658 op/s 2024-09-10T11:00:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.523+0000 7f8a023be640 1 -- 172.21.15.7:0/2865131105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89fc101350 msgr2=0x7f89fc10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.523+0000 7f8a023be640 1 --2- 172.21.15.7:0/2865131105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89fc101350 0x7f89fc10d880 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009cd0 tx=0x7f89f402a8e0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.524+0000 7f8a023be640 1 -- 172.21.15.7:0/2865131105 shutdown_connections 2024-09-10T11:00:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.524+0000 7f8a023be640 1 --2- 172.21.15.7:0/2865131105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89fc101350 0x7f89fc10d880 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.524+0000 7f8a023be640 1 --2- 172.21.15.7:0/2865131105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89fc1009b0 0x7f89fc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.524+0000 7f8a023be640 1 --2- 172.21.15.7:0/2865131105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89fc1069c0 0x7f89fc106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.524+0000 7f8a023be640 1 -- 172.21.15.7:0/2865131105 >> 172.21.15.7:0/2865131105 conn(0x7f89fc077990 msgr2=0x7f89fc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.524+0000 7f8a023be640 1 -- 172.21.15.7:0/2865131105 shutdown_connections 2024-09-10T11:00:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.524+0000 7f8a023be640 1 -- 172.21.15.7:0/2865131105 wait complete. 2024-09-10T11:00:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.524+0000 7f8a023be640 1 Processor -- start 2024-09-10T11:00:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.524+0000 7f8a023be640 1 -- start start 2024-09-10T11:00:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a023be640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89fc1009b0 0x7f89fc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a023be640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89fc101350 0x7f89fc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a023be640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89fc1069c0 0x7f89fc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a023be640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc110e00 con 0x7f89fc1069c0 2024-09-10T11:00:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a023be640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc110c80 con 0x7f89fc1009b0 2024-09-10T11:00:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a023be640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc110f80 con 0x7f89fc101350 2024-09-10T11:00:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a0095b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89fc1069c0 0x7f89fc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a0095b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89fc1069c0 0x7f89fc19edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45668/0 (socket says 172.21.15.7:45668) 2024-09-10T11:00:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a0095b640 1 -- 172.21.15.7:0/3034132482 learned_addr learned my addr 172.21.15.7:0/3034132482 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89fc101350 0x7f89fc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f89fbfff640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89fc1009b0 0x7f89fc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a0095b640 1 -- 172.21.15.7:0/3034132482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89fc101350 msgr2=0x7f89fc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a0095b640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89fc101350 0x7f89fc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a0095b640 1 -- 172.21.15.7:0/3034132482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89fc1009b0 msgr2=0x7f89fc19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a0095b640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89fc1009b0 0x7f89fc19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a0095b640 1 -- 172.21.15.7:0/3034132482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc19f4f0 con 0x7f89fc1069c0 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89fc101350 0x7f89fc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.525+0000 7f8a0095b640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89fc1069c0 0x7f89fc19edc0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f89f4005cc0 tx=0x7f89f40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.526+0000 7f89f97fa640 1 -- 172.21.15.7:0/3034132482 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4003e00 con 0x7f89fc1069c0 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.526+0000 7f89f97fa640 1 -- 172.21.15.7:0/3034132482 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f4004520 con 0x7f89fc1069c0 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.526+0000 7f8a023be640 1 -- 172.21.15.7:0/3034132482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89fc19f780 con 0x7f89fc1069c0 2024-09-10T11:00:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.526+0000 7f89f97fa640 1 -- 172.21.15.7:0/3034132482 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4030db0 con 0x7f89fc1069c0 2024-09-10T11:00:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.526+0000 7f8a023be640 1 -- 172.21.15.7:0/3034132482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89fc1a7070 con 0x7f89fc1069c0 2024-09-10T11:00:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.528+0000 7f89f97fa640 1 -- 172.21.15.7:0/3034132482 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f89f40046d0 con 0x7f89fc1069c0 2024-09-10T11:00:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f8a05230740 1 Processor -- start 2024-09-10T11:00:46.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f8a05230740 1 -- start start 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f8a05230740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cd2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f8a05230740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f8a05230740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53ce00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f8a05230740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cc53c372f0 con 0x55cc53cd3410 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f8a05230740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cc53bdf270 con 0x55cc53cd2af0 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f8a05230740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cc53c332b0 con 0x55cc53cddcb0 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f89fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f89fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45680/0 (socket says 172.21.15.7:45680) 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f89fbfff640 1 -- 172.21.15.7:0/2132983689 learned_addr learned my addr 172.21.15.7:0/2132983689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/2132983689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cd2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f89f97fa640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f89e40a2da0 0x7f89e40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/2132983689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53ce00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.529+0000 7f8a0095b640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f89e40a2da0 0x7f89e40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89fb7fe640 1 -- 172.21.15.7:0/2132983689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 msgr2=0x55cc53ce00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/2132983689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53ce00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89f97fa640 1 -- 172.21.15.7:0/3034132482 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8520+0+0 (secure 0 0 0) 0x7f89f402e030 con 0x7f89fc1069c0 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89fb7fe640 1 -- 172.21.15.7:0/2132983689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 msgr2=0x55cc53cdd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/2132983689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89fb7fe640 1 -- 172.21.15.7:0/2132983689 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc53bde340 con 0x55cc53cd2af0 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89fbfff640 1 --2- 172.21.15.7:0/2132983689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/2132983689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cd2ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f89f0012750 tx=0x7f89f0009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae9e27ba680eabb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f8a0095b640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f89e40a2da0 0x7f89e40a5260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f89f40ee3e0 tx=0x7f89f40dd040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89e27fc640 1 -- 172.21.15.7:0/2132983689 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0012cd0 con 0x55cc53cd2af0 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89e27fc640 1 -- 172.21.15.7:0/2132983689 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f00107d0 con 0x55cc53cd2af0 2024-09-10T11:00:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.530+0000 7f89e27fc640 1 -- 172.21.15.7:0/2132983689 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f00037f0 con 0x55cc53cd2af0 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.531+0000 7f8a05230740 1 -- 172.21.15.7:0/2132983689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 msgr2=0x55cc53cd2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.531+0000 7f8a05230740 1 --2- 172.21.15.7:0/2132983689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cd2ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f89f0012750 tx=0x7f89f0009680 comp rx=0 tx=0).stop 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.531+0000 7f8a05230740 1 -- 172.21.15.7:0/2132983689 shutdown_connections 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.531+0000 7f8a05230740 1 --2- 172.21.15.7:0/2132983689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53ce00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.531+0000 7f8a05230740 1 --2- 172.21.15.7:0/2132983689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.531+0000 7f8a05230740 1 --2- 172.21.15.7:0/2132983689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cd2ed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.531+0000 7f8a05230740 1 -- 172.21.15.7:0/2132983689 >> 172.21.15.7:0/2132983689 conn(0x55cc53ccf060 msgr2=0x55cc53ccf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.531+0000 7f8a05230740 1 -- 172.21.15.7:0/2132983689 shutdown_connections 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.531+0000 7f8a05230740 1 -- 172.21.15.7:0/2132983689 wait complete. 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.531+0000 7f8a05230740 1 -- start start 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f8a05230740 1 Processor -- start 2024-09-10T11:00:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f8a05230740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cdaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f8a05230740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f8a05230740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53cdb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f8a05230740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cc53ce0880 con 0x55cc53cd3410 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f8a05230740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cc53bc0290 con 0x55cc53cd2af0 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f8a05230740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cc53ce0a00 con 0x55cc53cddcb0 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cdaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f8a05230740 10 client.0 _ll_register_callbacks cb 0x55cc53bf1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cdaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:52130/0 (socket says 172.21.15.7:52130) 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45682/0 (socket says 172.21.15.7:45682) 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fb7fe640 1 -- 172.21.15.7:0/112495736 learned_addr learned my addr 172.21.15.7:0/112495736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fb7fe640 1 -- 172.21.15.7:0/112495736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 msgr2=0x55cc53cdb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53cdb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fb7fe640 1 -- 172.21.15.7:0/112495736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 msgr2=0x55cc53cdb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fb7fe640 1 -- 172.21.15.7:0/112495736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc5394a210 con 0x55cc53cd2af0 2024-09-10T11:00:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.532+0000 7f89fbfff640 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:00:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.533+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cdaba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f89f00044c0 tx=0x7f89f00044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=441a38639a7083b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.533+0000 7f89db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T11:00:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.533+0000 7f89db7fe640 1 -- 172.21.15.7:0/112495736 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0005050 con 0x55cc53cd2af0 2024-09-10T11:00:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.533+0000 7f89db7fe640 1 -- 172.21.15.7:0/112495736 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8520+0+0 (secure 0 0 0) 0x7f89f00051f0 con 0x55cc53cd2af0 2024-09-10T11:00:46.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.533+0000 7f8a05230740 1 -- 172.21.15.7:0/112495736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f00107d0 con 0x55cc53cd2af0 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.533+0000 7f89db7fe640 1 -- 172.21.15.7:0/112495736 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f89f0031050 con 0x55cc53cd2af0 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.533+0000 7f8a05230740 10 client.26063 fetch_fsmap learned FSMap version 48 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.533+0000 7f8a05230740 1 -- 172.21.15.7:0/112495736 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cc53a16620 con 0x55cc53cd2af0 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.534+0000 7f89db7fe640 1 -- 172.21.15.7:0/112495736 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f00107d0 con 0x55cc53cd2af0 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.535+0000 7f89db7fe640 1 -- 172.21.15.7:0/112495736 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f89f00035f0 con 0x55cc53cd2af0 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.535+0000 7f8a05230740 10 client.26063 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.535+0000 7f8a05230740 10 client.26063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.535+0000 7f8a05230740 20 client.26063 populate_metadata read hostname 'smithi007' 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.535+0000 7f8a05230740 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cc53cedaa0 0x55cc53cefe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.535+0000 7f8a05230740 4 client.26063 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:00:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.535+0000 7f8a05230740 1 -- 172.21.15.7:0/112495736 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc53c44a20 con 0x55cc53cedaa0 2024-09-10T11:00:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.543+0000 7f89fbfff640 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cc53cedaa0 0x55cc53cefe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.544+0000 7f89db7fe640 10 client.26063 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.544+0000 7f89fbfff640 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cc53cedaa0 0x55cc53cefe90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6616ef03bef19279 server_cookie=79fcf4e33470f9a3 in_seq=0 out_seq=0 2024-09-10T11:00:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.546+0000 7f89db7fe640 1 -- 172.21.15.7:0/112495736 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc53c44a20 con 0x55cc53cedaa0 2024-09-10T11:00:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.546+0000 7f89db7fe640 10 client.26063 handle_command_reply: tid=0 2024-09-10T11:00:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.546+0000 7f8a05230740 1 client.26063 shutdown 2024-09-10T11:00:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.546+0000 7f8a05230740 1 -- 172.21.15.7:0/112495736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 msgr2=0x55cc53cdaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.546+0000 7f8a05230740 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cdaba0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f89f00044c0 tx=0x7f89f00044f0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.547+0000 7f8a05230740 1 -- 172.21.15.7:0/112495736 shutdown_connections 2024-09-10T11:00:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.547+0000 7f8a05230740 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cc53cedaa0 0x55cc53cefe90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.547+0000 7f8a05230740 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53cdb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.547+0000 7f8a05230740 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53cdb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.547+0000 7f8a05230740 1 --2- 172.21.15.7:0/112495736 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53cdaba0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.547+0000 7f8a05230740 1 -- 172.21.15.7:0/112495736 >> 172.21.15.7:0/112495736 conn(0x55cc53ccf060 msgr2=0x55cc53cd4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.547+0000 7f89db7fe640 0 client.26063 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.547+0000 7f8a05230740 1 -- 172.21.15.7:0/112495736 shutdown_connections 2024-09-10T11:00:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.547+0000 7f8a05230740 1 -- 172.21.15.7:0/112495736 wait complete. 2024-09-10T11:00:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.547+0000 7f8a05230740 20 client.26063 trim_cache size 0 max 16384 2024-09-10T11:00:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.566+0000 7f8a05230740 1 Processor -- start 2024-09-10T11:00:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.566+0000 7f8a05230740 1 -- start start 2024-09-10T11:00:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.566+0000 7f8a05230740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cd2af0 0x55cc53cd5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.566+0000 7f8a05230740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd3410 0x55cc53cdb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.566+0000 7f8a05230740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cddcb0 0x55cc53cdcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.566+0000 7f8a05230740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1b2290 con 0x55cc53cddcb0 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.566+0000 7f8a05230740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1afad0 con 0x55cc53cd3410 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.566+0000 7f8a05230740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1ad920 con 0x55cc53cd2af0 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.566+0000 7f89fb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cddcb0 0x55cc53cdcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.566+0000 7f89fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd3410 0x55cc53cdb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fbfff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd3410 0x55cc53cdb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:52146/0 (socket says 172.21.15.7:52146) 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cddcb0 0x55cc53cdcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45698/0 (socket says 172.21.15.7:45698) 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fbfff640 1 -- 172.21.15.7:0/3342794838 learned_addr learned my addr 172.21.15.7:0/3342794838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/3342794838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cd2af0 0x55cc53cd5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fbfff640 1 -- 172.21.15.7:0/3342794838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cd2af0 msgr2=0x55cc53cd5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fbfff640 1 --2- 172.21.15.7:0/3342794838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cd2af0 0x55cc53cd5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fbfff640 1 -- 172.21.15.7:0/3342794838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cddcb0 msgr2=0x55cc53cdcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fbfff640 1 --2- 172.21.15.7:0/3342794838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cddcb0 0x55cc53cdcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fbfff640 1 -- 172.21.15.7:0/3342794838 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc53cd0110 con 0x55cc53cd3410 2024-09-10T11:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/3342794838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cddcb0 0x55cc53cdcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89fbfff640 1 --2- 172.21.15.7:0/3342794838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd3410 0x55cc53cdb450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f89ec004620 tx=0x7f89ec005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21f4548a57e3b404 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89e1ffb640 1 -- 172.21.15.7:0/3342794838 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec011840 con 0x55cc53cd3410 2024-09-10T11:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89e1ffb640 1 -- 172.21.15.7:0/3342794838 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ec026400 con 0x55cc53cd3410 2024-09-10T11:00:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.567+0000 7f89e1ffb640 1 -- 172.21.15.7:0/3342794838 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec01f660 con 0x55cc53cd3410 2024-09-10T11:00:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.568+0000 7f8a05230740 1 -- 172.21.15.7:0/3342794838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd3410 msgr2=0x55cc53cdb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.568+0000 7f8a05230740 1 --2- 172.21.15.7:0/3342794838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd3410 0x55cc53cdb450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f89ec004620 tx=0x7f89ec005f90 comp rx=0 tx=0).stop 2024-09-10T11:00:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.568+0000 7f8a05230740 1 -- 172.21.15.7:0/3342794838 shutdown_connections 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.568+0000 7f8a05230740 1 --2- 172.21.15.7:0/3342794838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cddcb0 0x55cc53cdcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.568+0000 7f8a05230740 1 --2- 172.21.15.7:0/3342794838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd3410 0x55cc53cdb450 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.568+0000 7f8a05230740 1 --2- 172.21.15.7:0/3342794838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cd2af0 0x55cc53cd5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.568+0000 7f8a05230740 1 -- 172.21.15.7:0/3342794838 >> 172.21.15.7:0/3342794838 conn(0x55cc53ccf060 msgr2=0x55cc53cdc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.569+0000 7f8a05230740 1 -- 172.21.15.7:0/3342794838 shutdown_connections 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.569+0000 7f8a05230740 1 -- 172.21.15.7:0/3342794838 wait complete. 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.569+0000 7f8a05230740 1 -- start start 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f8a05230740 1 Processor -- start 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f8a05230740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53ce5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f8a05230740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53ce6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f8a05230740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53cd6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f8a05230740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1adf90 con 0x55cc53cd3410 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f8a05230740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1af400 con 0x55cc53cd2af0 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f8a05230740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc1ae660 con 0x55cc53cddcb0 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f89fb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53ce5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f8a05230740 10 client.0 _ll_register_callbacks cb 0x7f89ec009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f89fb7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53ce5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:52152/0 (socket says 172.21.15.7:52152) 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f89fb7fe640 1 -- 172.21.15.7:0/2764295415 learned_addr learned my addr 172.21.15.7:0/2764295415 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.570+0000 7f89fbfff640 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53ce6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53cd6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89fb7fe640 1 -- 172.21.15.7:0/2764295415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 msgr2=0x55cc53cd6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53cd6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89fb7fe640 1 -- 172.21.15.7:0/2764295415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 msgr2=0x55cc53ce6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53ce6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89fb7fe640 1 -- 172.21.15.7:0/2764295415 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc53cd8a70 con 0x55cc53cd2af0 2024-09-10T11:00:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89fb7fe640 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53ce5cc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f89f0003b60 tx=0x7f89f001b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=633ddb90573dd5aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:00:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T11:00:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89db7fe640 1 -- 172.21.15.7:0/2764295415 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0010e90 con 0x55cc53cd2af0 2024-09-10T11:00:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89db7fe640 1 -- 172.21.15.7:0/2764295415 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8520+0+0 (secure 0 0 0) 0x7f89f0005040 con 0x55cc53cd2af0 2024-09-10T11:00:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f8a05230740 1 -- 172.21.15.7:0/2764295415 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ec026400 con 0x55cc53cd2af0 2024-09-10T11:00:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.571+0000 7f89db7fe640 1 -- 172.21.15.7:0/2764295415 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f89f0045050 con 0x55cc53cd2af0 2024-09-10T11:00:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.572+0000 7f8a05230740 10 client.26069 fetch_fsmap learned FSMap version 48 2024-09-10T11:00:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.572+0000 7f8a05230740 1 -- 172.21.15.7:0/2764295415 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cc53d60290 con 0x55cc53cd2af0 2024-09-10T11:00:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.572+0000 7f89db7fe640 1 -- 172.21.15.7:0/2764295415 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ec026400 con 0x55cc53cd2af0 2024-09-10T11:00:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.572+0000 7f89db7fe640 1 -- 172.21.15.7:0/2764295415 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f89f0004ba0 con 0x55cc53cd2af0 2024-09-10T11:00:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.572+0000 7f8a05230740 10 client.26069 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T11:00:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.572+0000 7f8a05230740 10 client.26069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:00:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.572+0000 7f8a05230740 20 client.26069 populate_metadata read hostname 'smithi007' 2024-09-10T11:00:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.572+0000 7f8a05230740 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cc53d60980 0x55cc53d62da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:00:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.573+0000 7f8a05230740 4 client.26069 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T11:00:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.573+0000 7f8a05230740 1 -- 172.21.15.7:0/2764295415 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc53c3ed70 con 0x55cc53d60980 2024-09-10T11:00:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.576+0000 7f89fbfff640 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cc53d60980 0x55cc53d62da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:00:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.577+0000 7f89db7fe640 10 client.26069 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.577+0000 7f89fbfff640 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cc53d60980 0x55cc53d62da0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d91ae90f871ff14 server_cookie=488ce30190e4b3f3 in_seq=0 out_seq=0 2024-09-10T11:00:46.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 76643 40.53 MB/sec warmup 106 sec latency 236.369 ms 2024-09-10T11:00:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.940+0000 7f89db7fe640 1 -- 172.21.15.7:0/2764295415 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc53c3ed70 con 0x55cc53d60980 2024-09-10T11:00:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.940+0000 7f89db7fe640 10 client.26069 handle_command_reply: tid=0 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.940+0000 7f8a05230740 1 client.26069 shutdown 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 1 -- 172.21.15.7:0/2764295415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 msgr2=0x55cc53ce5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53ce5cc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f89f0003b60 tx=0x7f89f001b040 comp rx=0 tx=0).stop 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 1 -- 172.21.15.7:0/2764295415 shutdown_connections 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55cc53d60980 0x55cc53d62da0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc53cddcb0 0x55cc53cd6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc53cd3410 0x55cc53ce6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f89db7fe640 0 client.26069 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 1 --2- 172.21.15.7:0/2764295415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc53cd2af0 0x55cc53ce5cc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 1 -- 172.21.15.7:0/2764295415 >> 172.21.15.7:0/2764295415 conn(0x55cc53ccf060 msgr2=0x55cc53cce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 1 -- 172.21.15.7:0/2764295415 shutdown_connections 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 1 -- 172.21.15.7:0/2764295415 wait complete. 2024-09-10T11:00:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.941+0000 7f8a05230740 20 client.26069 trim_cache size 0 max 16384 2024-09-10T11:00:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T11:00:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T11:00:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T11:00:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T11:00:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 -- 172.21.15.7:0/3034132482 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f89e40a2da0 msgr2=0x7f89e40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f89e40a2da0 0x7f89e40a5260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f89f40ee3e0 tx=0x7f89f40dd040 comp rx=0 tx=0).stop 2024-09-10T11:00:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 -- 172.21.15.7:0/3034132482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89fc1069c0 msgr2=0x7f89fc19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:00:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89fc1069c0 0x7f89fc19edc0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f89f4005cc0 tx=0x7f89f40093d0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 -- 172.21.15.7:0/3034132482 shutdown_connections 2024-09-10T11:00:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f89e40a2da0 0x7f89e40a5260 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89fc1069c0 0x7f89fc19edc0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89fc101350 0x7f89fc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 --2- 172.21.15.7:0/3034132482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89fc1009b0 0x7f89fc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:00:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 -- 172.21.15.7:0/3034132482 >> 172.21.15.7:0/3034132482 conn(0x7f89fc077990 msgr2=0x7f89fc0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:00:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 -- 172.21.15.7:0/3034132482 shutdown_connections 2024-09-10T11:00:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:00:46.942+0000 7f8a023be640 1 -- 172.21.15.7:0/3034132482 wait complete. 2024-09-10T11:00:47.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 77689 40.70 MB/sec warmup 107 sec latency 329.967 ms 2024-09-10T11:00:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:47 smithi133 ceph-mon[35282]: pgmap v471: 97 pgs: 97 active+clean; 833 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 12 MiB/s wr, 940 op/s 2024-09-10T11:00:47.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:47 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:00:48.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:47 smithi103 ceph-mon[35160]: pgmap v471: 97 pgs: 97 active+clean; 833 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 12 MiB/s wr, 940 op/s 2024-09-10T11:00:48.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:47 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:00:48.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:47 smithi007 ceph-mon[30873]: pgmap v471: 97 pgs: 97 active+clean; 833 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 12 MiB/s wr, 940 op/s 2024-09-10T11:00:48.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:47 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:00:48.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 78718 40.98 MB/sec warmup 108 sec latency 71.572 ms 2024-09-10T11:00:49.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 79422 40.86 MB/sec warmup 109 sec latency 52.402 ms 2024-09-10T11:00:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:49 smithi133 ceph-mon[35282]: pgmap v472: 97 pgs: 97 active+clean; 833 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 536 op/s 2024-09-10T11:00:50.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:49 smithi103 ceph-mon[35160]: pgmap v472: 97 pgs: 97 active+clean; 833 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 536 op/s 2024-09-10T11:00:50.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:49 smithi007 ceph-mon[30873]: pgmap v472: 97 pgs: 97 active+clean; 833 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 536 op/s 2024-09-10T11:00:50.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 79978 40.75 MB/sec warmup 110 sec latency 353.129 ms 2024-09-10T11:00:51.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 81015 40.88 MB/sec warmup 111 sec latency 75.165 ms 2024-09-10T11:00:51.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:51 smithi133 ceph-mon[35282]: pgmap v473: 97 pgs: 97 active+clean; 848 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 14 MiB/s wr, 1k op/s 2024-09-10T11:00:52.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:51 smithi103 ceph-mon[35160]: pgmap v473: 97 pgs: 97 active+clean; 848 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 14 MiB/s wr, 1k op/s 2024-09-10T11:00:52.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:51 smithi007 ceph-mon[30873]: pgmap v473: 97 pgs: 97 active+clean; 848 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 14 MiB/s wr, 1k op/s 2024-09-10T11:00:52.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 82038 41.08 MB/sec warmup 112 sec latency 40.517 ms 2024-09-10T11:00:53.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 82697 41.02 MB/sec warmup 113 sec latency 100.957 ms 2024-09-10T11:00:53.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:53 smithi133 ceph-mon[35282]: pgmap v474: 97 pgs: 97 active+clean; 848 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 745 op/s 2024-09-10T11:00:54.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:53 smithi103 ceph-mon[35160]: pgmap v474: 97 pgs: 97 active+clean; 848 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 745 op/s 2024-09-10T11:00:54.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:53 smithi007 ceph-mon[30873]: pgmap v474: 97 pgs: 97 active+clean; 848 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 745 op/s 2024-09-10T11:00:54.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 83623 41.01 MB/sec warmup 114 sec latency 77.257 ms 2024-09-10T11:00:55.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 84331 41.01 MB/sec warmup 115 sec latency 307.753 ms 2024-09-10T11:00:55.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:55 smithi133 ceph-mon[35282]: pgmap v475: 97 pgs: 97 active+clean; 848 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 745 op/s 2024-09-10T11:00:56.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:55 smithi103 ceph-mon[35160]: pgmap v475: 97 pgs: 97 active+clean; 848 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 745 op/s 2024-09-10T11:00:56.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:55 smithi007 ceph-mon[30873]: pgmap v475: 97 pgs: 97 active+clean; 848 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 745 op/s 2024-09-10T11:00:56.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 85132 41.04 MB/sec warmup 116 sec latency 116.984 ms 2024-09-10T11:00:57.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 85971 41.11 MB/sec warmup 117 sec latency 45.114 ms 2024-09-10T11:00:57.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:57 smithi133 ceph-mon[35282]: pgmap v476: 97 pgs: 97 active+clean; 876 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 17 MiB/s wr, 1.09k op/s 2024-09-10T11:00:58.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:57 smithi103 ceph-mon[35160]: pgmap v476: 97 pgs: 97 active+clean; 876 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 17 MiB/s wr, 1.09k op/s 2024-09-10T11:00:58.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:57 smithi007 ceph-mon[30873]: pgmap v476: 97 pgs: 97 active+clean; 876 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 17 MiB/s wr, 1.09k op/s 2024-09-10T11:00:58.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 86814 41.12 MB/sec warmup 118 sec latency 45.553 ms 2024-09-10T11:00:59.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 87709 41.15 MB/sec warmup 119 sec latency 78.659 ms 2024-09-10T11:00:59.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:00:59 smithi133 ceph-mon[35282]: pgmap v477: 97 pgs: 97 active+clean; 876 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 813 op/s 2024-09-10T11:01:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.044+0000 7fd300e8b640 1 -- 172.21.15.7:0/1972729180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2fc10c0d0 msgr2=0x7fd2fc10e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.044+0000 7fd300e8b640 1 --2- 172.21.15.7:0/1972729180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2fc10c0d0 0x7fd2fc10e550 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009ca0 tx=0x7fd2f402a950 comp rx=0 tx=0).stop 2024-09-10T11:01:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.045+0000 7fd300e8b640 1 -- 172.21.15.7:0/1972729180 shutdown_connections 2024-09-10T11:01:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.045+0000 7fd300e8b640 1 --2- 172.21.15.7:0/1972729180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2fc10c0d0 0x7fd2fc10e550 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.045+0000 7fd300e8b640 1 --2- 172.21.15.7:0/1972729180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2fc1035a0 0x7fd2fc10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.045+0000 7fd300e8b640 1 --2- 172.21.15.7:0/1972729180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2fc102c80 0x7fd2fc103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.045+0000 7fd300e8b640 1 -- 172.21.15.7:0/1972729180 >> 172.21.15.7:0/1972729180 conn(0x7fd2fc077990 msgr2=0x7fd2fc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.045+0000 7fd300e8b640 1 -- 172.21.15.7:0/1972729180 shutdown_connections 2024-09-10T11:01:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.045+0000 7fd300e8b640 1 -- 172.21.15.7:0/1972729180 wait complete. 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.045+0000 7fd300e8b640 1 Processor -- start 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd300e8b640 1 -- start start 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd300e8b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2fc102c80 0x7fd2fc1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd300e8b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2fc1035a0 0x7fd2fc198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd300e8b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2fc10c0d0 0x7fd2fc19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd300e8b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc111ee0 con 0x7fd2fc10c0d0 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd300e8b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc111d60 con 0x7fd2fc102c80 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd300e8b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc112060 con 0x7fd2fc1035a0 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd2fad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2fc10c0d0 0x7fd2fc19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd2fad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2fc10c0d0 0x7fd2fc19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54024/0 (socket says 172.21.15.7:54024) 2024-09-10T11:01:00.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd2fad9d640 1 -- 172.21.15.7:0/314033093 learned_addr learned my addr 172.21.15.7:0/314033093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd2fa59c640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2fc102c80 0x7fd2fc1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2fc1035a0 0x7fd2fc198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd2fad9d640 1 -- 172.21.15.7:0/314033093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2fc1035a0 msgr2=0x7fd2fc198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd2fad9d640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2fc1035a0 0x7fd2fc198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd2fad9d640 1 -- 172.21.15.7:0/314033093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2fc102c80 msgr2=0x7fd2fc1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd2fad9d640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2fc102c80 0x7fd2fc1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.046+0000 7fd2fad9d640 1 -- 172.21.15.7:0/314033093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc19d2b0 con 0x7fd2fc10c0d0 2024-09-10T11:01:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.047+0000 7fd2fad9d640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2fc10c0d0 0x7fd2fc19cb80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009dc0 tx=0x7fd2f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.047+0000 7fd2eb7fe640 1 -- 172.21.15.7:0/314033093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4034040 con 0x7fd2fc10c0d0 2024-09-10T11:01:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.047+0000 7fd300e8b640 1 -- 172.21.15.7:0/314033093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc19d540 con 0x7fd2fc10c0d0 2024-09-10T11:01:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.047+0000 7fd2eb7fe640 1 -- 172.21.15.7:0/314033093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f402f940 con 0x7fd2fc10c0d0 2024-09-10T11:01:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.047+0000 7fd2eb7fe640 1 -- 172.21.15.7:0/314033093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40046a0 con 0x7fd2fc10c0d0 2024-09-10T11:01:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.047+0000 7fd300e8b640 1 -- 172.21.15.7:0/314033093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc1a4e30 con 0x7fd2fc10c0d0 2024-09-10T11:01:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.050+0000 7fd303c24740 1 Processor -- start 2024-09-10T11:01:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.050+0000 7fd2eb7fe640 1 -- 172.21.15.7:0/314033093 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd2f4037020 con 0x7fd2fc10c0d0 2024-09-10T11:01:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd303c24740 1 -- start start 2024-09-10T11:01:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd303c24740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852edd9af0 0x55852edd9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd303c24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852ede4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd303c24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852ede4cb0 0x55852ede70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd303c24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55852ed3e2f0 con 0x55852ede4cb0 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd303c24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55852ece6270 con 0x55852edda410 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd303c24740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55852ed3a2b0 con 0x55852edd9af0 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd2f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852ede4cb0 0x55852ede70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd2f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852ede4cb0 0x55852ede70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54030/0 (socket says 172.21.15.7:54030) 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/2442309661 learned_addr learned my addr 172.21.15.7:0/2442309661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd2fa59c640 1 --2- 172.21.15.7:0/2442309661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852ede4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.051+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/2442309661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852edd9af0 0x55852edd9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/2442309661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852edd9af0 msgr2=0x55852edd9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2eb7fe640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd2e40a2da0 0x7fd2e40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/2442309661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852edd9af0 0x55852edd9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/2442309661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 msgr2=0x55852ede4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/2442309661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852ede4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/2442309661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852ece5340 con 0x55852ede4cb0 2024-09-10T11:01:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2fad9d640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd2e40a2da0 0x7fd2e40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/2442309661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852ede4cb0 0x55852ede70a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0011780 tx=0x7fd2f0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1195448f773b93ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2eb7fe640 1 -- 172.21.15.7:0/314033093 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8520+0+0 (secure 0 0 0) 0x7fd2f40f0050 con 0x7fd2fc10c0d0 2024-09-10T11:01:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2e8ff9640 1 -- 172.21.15.7:0/2442309661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f000f7c0 con 0x55852ede4cb0 2024-09-10T11:01:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2e8ff9640 1 -- 172.21.15.7:0/2442309661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f000fe40 con 0x55852ede4cb0 2024-09-10T11:01:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.052+0000 7fd2e8ff9640 1 -- 172.21.15.7:0/2442309661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f001fe40 con 0x55852ede4cb0 2024-09-10T11:01:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd303c24740 1 -- 172.21.15.7:0/2442309661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852ede4cb0 msgr2=0x55852ede70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd303c24740 1 --2- 172.21.15.7:0/2442309661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852ede4cb0 0x55852ede70a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0011780 tx=0x7fd2f0005850 comp rx=0 tx=0).stop 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd2fad9d640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd2e40a2da0 0x7fd2e40a5260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40b9050 tx=0x7fd2f402a950 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd303c24740 1 -- 172.21.15.7:0/2442309661 shutdown_connections 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd303c24740 1 --2- 172.21.15.7:0/2442309661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852ede4cb0 0x55852ede70a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd303c24740 1 --2- 172.21.15.7:0/2442309661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852ede4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd303c24740 1 --2- 172.21.15.7:0/2442309661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852edd9af0 0x55852edd9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd303c24740 1 -- 172.21.15.7:0/2442309661 >> 172.21.15.7:0/2442309661 conn(0x55852edd6060 msgr2=0x55852edd6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd303c24740 1 -- 172.21.15.7:0/2442309661 shutdown_connections 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd303c24740 1 -- 172.21.15.7:0/2442309661 wait complete. 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.053+0000 7fd303c24740 1 -- start start 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd303c24740 1 Processor -- start 2024-09-10T11:01:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd303c24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd303c24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852ede2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd303c24740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852ede4cb0 0x55852ede2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd303c24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55852ede7880 con 0x55852edd9af0 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd303c24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55852ecc7290 con 0x55852edda410 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd303c24740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55852ede7a00 con 0x55852ede4cb0 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd303c24740 10 client.0 _ll_register_callbacks cb 0x55852ecf8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd2f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd2f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54044/0 (socket says 172.21.15.7:54044) 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/4269817757 learned_addr learned my addr 172.21.15.7:0/4269817757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd2fa59c640 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852ede2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852ede4cb0 0x55852ede2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/4269817757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852ede4cb0 msgr2=0x55852ede2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852ede4cb0 0x55852ede2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/4269817757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 msgr2=0x55852ede2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852ede2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.054+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/4269817757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852ea51210 con 0x55852edd9af0 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.055+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede1c00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009c80 tx=0x7fd2f001fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24913988e2a0b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.055+0000 7fd2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.055+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/4269817757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f001f440 con 0x55852edd9af0 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.055+0000 7fd303c24740 1 -- 172.21.15.7:0/4269817757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2f000fe40 con 0x55852edd9af0 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.055+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/4269817757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8520+0+0 (secure 0 0 0) 0x7fd2f001fbc0 con 0x55852edd9af0 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.055+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/4269817757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd2f0035050 con 0x55852edd9af0 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.055+0000 7fd303c24740 10 client.16473 fetch_fsmap learned FSMap version 49 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.055+0000 7fd303c24740 1 -- 172.21.15.7:0/4269817757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55852eb1d620 con 0x55852edd9af0 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.055+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/4269817757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f000fe40 con 0x55852edd9af0 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.055+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/4269817757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fd2f001d020 con 0x55852edd9af0 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.056+0000 7fd303c24740 10 client.16473 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T11:01:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.056+0000 7fd303c24740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.056+0000 7fd303c24740 20 client.16473 populate_metadata read hostname 'smithi007' 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.056+0000 7fd303c24740 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55852edf4b30 0x55852edf6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.056+0000 7fd303c24740 4 client.16473 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.056+0000 7fd303c24740 1 -- 172.21.15.7:0/4269817757 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55852ed4ba20 con 0x55852edf4b30 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.056+0000 7fd2fa59c640 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55852edf4b30 0x55852edf6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.056+0000 7fd2fa59c640 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55852edf4b30 0x55852edf6f20 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86b2784a43545ce server_cookie=d6254ecc7e1ed8c3 in_seq=0 out_seq=0 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.056+0000 7fd2e1ffb640 10 client.16473 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/4269817757 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55852ed4ba20 con 0x55852edf4b30 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd2e1ffb640 10 client.16473 handle_command_reply: tid=0 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 client.16473 shutdown 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 -- 172.21.15.7:0/4269817757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 msgr2=0x55852ede1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede1c00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009c80 tx=0x7fd2f001fae0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 -- 172.21.15.7:0/4269817757 shutdown_connections 2024-09-10T11:01:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55852edf4b30 0x55852edf6f20 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852ede4cb0 0x55852ede2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852ede2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 --2- 172.21.15.7:0/4269817757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede1c00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd2e1ffb640 0 client.16473 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 -- 172.21.15.7:0/4269817757 >> 172.21.15.7:0/4269817757 conn(0x55852edd6060 msgr2=0x55852eddbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 -- 172.21.15.7:0/4269817757 shutdown_connections 2024-09-10T11:01:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 1 -- 172.21.15.7:0/4269817757 wait complete. 2024-09-10T11:01:00.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.057+0000 7fd303c24740 20 client.16473 trim_cache size 0 max 16384 2024-09-10T11:01:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd303c24740 1 Processor -- start 2024-09-10T11:01:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd303c24740 1 -- start start 2024-09-10T11:01:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd303c24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd303c24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852eddbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd303c24740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852ede4cb0 0x55852edfc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd303c24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1b0010 con 0x55852edd9af0 2024-09-10T11:01:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd303c24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1ad820 con 0x55852edda410 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd303c24740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1ab670 con 0x55852ede4cb0 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd2f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd2f9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54056/0 (socket says 172.21.15.7:54056) 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/3704931399 learned_addr learned my addr 172.21.15.7:0/3704931399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd2fa59c640 1 --2- 172.21.15.7:0/3704931399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852eddbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.074+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/3704931399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852ede4cb0 0x55852edfc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.075+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/3704931399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852ede4cb0 msgr2=0x55852edfc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.075+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/3704931399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852ede4cb0 0x55852edfc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.075+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/3704931399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 msgr2=0x55852eddbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.075+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/3704931399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852eddbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.075+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/3704931399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852edda240 con 0x55852edd9af0 2024-09-10T11:01:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.075+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/3704931399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede24b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0006010 tx=0x7fd2f0005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e06d5d2ad6735a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.075+0000 7fd2e3fff640 1 -- 172.21.15.7:0/3704931399 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f003e070 con 0x55852edd9af0 2024-09-10T11:01:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.075+0000 7fd2e3fff640 1 -- 172.21.15.7:0/3704931399 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f001f920 con 0x55852edd9af0 2024-09-10T11:01:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.075+0000 7fd2e3fff640 1 -- 172.21.15.7:0/3704931399 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f001fdb0 con 0x55852edd9af0 2024-09-10T11:01:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 -- 172.21.15.7:0/3704931399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 msgr2=0x55852ede24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 --2- 172.21.15.7:0/3704931399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede24b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0006010 tx=0x7fd2f0005970 comp rx=0 tx=0).stop 2024-09-10T11:01:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 -- 172.21.15.7:0/3704931399 shutdown_connections 2024-09-10T11:01:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 --2- 172.21.15.7:0/3704931399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852ede4cb0 0x55852edfc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 --2- 172.21.15.7:0/3704931399 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852edda410 0x55852eddbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 --2- 172.21.15.7:0/3704931399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edd9af0 0x55852ede24b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 -- 172.21.15.7:0/3704931399 >> 172.21.15.7:0/3704931399 conn(0x55852edd6060 msgr2=0x55852eddb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 -- 172.21.15.7:0/3704931399 shutdown_connections 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 -- 172.21.15.7:0/3704931399 wait complete. 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 -- start start 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.076+0000 7fd303c24740 1 Processor -- start 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd303c24740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852edd9af0 0x55852eded110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd303c24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edda410 0x55852ede8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd303c24740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852ede4cb0 0x55852ede8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd303c24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1abce0 con 0x55852edda410 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd303c24740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1ad150 con 0x55852ede4cb0 2024-09-10T11:01:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd303c24740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1ac3b0 con 0x55852edd9af0 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd303c24740 10 client.0 _ll_register_callbacks cb 0x7fd2ec0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edda410 0x55852ede8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2f9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852ede4cb0 0x55852ede8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2f9d9b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852ede4cb0 0x55852ede8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:38112/0 (socket says 172.21.15.7:38112) 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edda410 0x55852ede8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54062/0 (socket says 172.21.15.7:54062) 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2f9d9b640 1 -- 172.21.15.7:0/2544927558 learned_addr learned my addr 172.21.15.7:0/2544927558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852edd9af0 0x55852eded110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2fa59c640 1 -- 172.21.15.7:0/2544927558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852edd9af0 msgr2=0x55852eded110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2fa59c640 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852edd9af0 0x55852eded110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2fa59c640 1 -- 172.21.15.7:0/2544927558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852ede4cb0 msgr2=0x55852ede8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2fa59c640 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852ede4cb0 0x55852ede8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2fa59c640 1 -- 172.21.15.7:0/2544927558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852ede8eb0 con 0x55852edda410 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2fa59c640 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edda410 0x55852ede8430 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x55852edf7650 tx=0x7fd2ec01fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9eddeb3d7e7531 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.077+0000 7fd2e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/2544927558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec010a60 con 0x55852edda410 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/2544927558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8520+0+0 (secure 0 0 0) 0x7fd2ec010c00 con 0x55852edda410 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd303c24740 1 -- 172.21.15.7:0/2544927558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55852ed45d70 con 0x55852edda410 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/2544927558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd2ec010e40 con 0x55852edda410 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd303c24740 10 client.16479 fetch_fsmap learned FSMap version 49 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd303c24740 1 -- 172.21.15.7:0/2544927558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55852ee5f320 con 0x55852edda410 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/2544927558 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55852ed45d70 con 0x55852edda410 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/2544927558 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fd2ec02e5c0 con 0x55852edda410 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd303c24740 10 client.16479 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T11:01:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd303c24740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:01:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd303c24740 20 client.16479 populate_metadata read hostname 'smithi007' 2024-09-10T11:01:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd303c24740 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55852ee5fad0 0x55852ee61ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd303c24740 4 client.16479 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T11:01:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd303c24740 1 -- 172.21.15.7:0/2544927558 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55852ecc2900 con 0x55852ee5fad0 2024-09-10T11:01:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.078+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55852ee5fad0 0x55852ee61ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.079+0000 7fd2f9d9b640 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55852ee5fad0 0x55852ee61ef0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1200aba30c065a3c server_cookie=a0265732e9b6bd8f in_seq=0 out_seq=0 2024-09-10T11:01:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.079+0000 7fd2e1ffb640 10 client.16479 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:00.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:00:59 smithi103 ceph-mon[35160]: pgmap v477: 97 pgs: 97 active+clean; 876 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 813 op/s 2024-09-10T11:01:00.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:00:59 smithi007 ceph-mon[30873]: pgmap v477: 97 pgs: 97 active+clean; 876 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 813 op/s 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd2e1ffb640 1 -- 172.21.15.7:0/2544927558 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55852ecc2900 con 0x55852ee5fad0 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd2e1ffb640 10 client.16479 handle_command_reply: tid=0 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 client.16479 shutdown 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 -- 172.21.15.7:0/2544927558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edda410 msgr2=0x55852ede8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edda410 0x55852ede8430 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x55852edf7650 tx=0x7fd2ec01fb70 comp rx=0 tx=0).stop 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 -- 172.21.15.7:0/2544927558 shutdown_connections 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55852ee5fad0 0x55852ee61ef0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55852ede4cb0 0x55852ede8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55852edda410 0x55852ede8430 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 --2- 172.21.15.7:0/2544927558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55852edd9af0 0x55852eded110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 -- 172.21.15.7:0/2544927558 >> 172.21.15.7:0/2544927558 conn(0x55852edd6060 msgr2=0x55852eddbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd2e1ffb640 0 client.16479 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 -- 172.21.15.7:0/2544927558 shutdown_connections 2024-09-10T11:01:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 1 -- 172.21.15.7:0/2544927558 wait complete. 2024-09-10T11:01:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.456+0000 7fd303c24740 20 client.16479 trim_cache size 0 max 16384 2024-09-10T11:01:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T11:01:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T11:01:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T11:01:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T11:01:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 -- 172.21.15.7:0/314033093 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd2e40a2da0 msgr2=0x7fd2e40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd2e40a2da0 0x7fd2e40a5260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40b9050 tx=0x7fd2f402a950 comp rx=0 tx=0).stop 2024-09-10T11:01:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 -- 172.21.15.7:0/314033093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2fc10c0d0 msgr2=0x7fd2fc19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2fc10c0d0 0x7fd2fc19cb80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009dc0 tx=0x7fd2f4003c10 comp rx=0 tx=0).stop 2024-09-10T11:01:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 -- 172.21.15.7:0/314033093 shutdown_connections 2024-09-10T11:01:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fd2e40a2da0 0x7fd2e40a5260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2fc10c0d0 0x7fd2fc19cb80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2fc1035a0 0x7fd2fc198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 --2- 172.21.15.7:0/314033093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2fc102c80 0x7fd2fc1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 -- 172.21.15.7:0/314033093 >> 172.21.15.7:0/314033093 conn(0x7fd2fc077990 msgr2=0x7fd2fc0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 -- 172.21.15.7:0/314033093 shutdown_connections 2024-09-10T11:01:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:00.457+0000 7fd300e8b640 1 -- 172.21.15.7:0/314033093 wait complete. 2024-09-10T11:01:01.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 88449 15.88 MB/sec execute 1 sec latency 336.453 ms 2024-09-10T11:01:01.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:01 smithi007 ceph-mon[30873]: pgmap v478: 97 pgs: 97 active+clean; 886 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-09-10T11:01:01.960 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:01 smithi103 ceph-mon[35160]: pgmap v478: 97 pgs: 97 active+clean; 886 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-09-10T11:01:01.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:01 smithi133 ceph-mon[35282]: pgmap v478: 97 pgs: 97 active+clean; 886 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-09-10T11:01:02.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 88841 24.54 MB/sec execute 2 sec latency 194.415 ms 2024-09-10T11:01:03.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 89421 26.45 MB/sec execute 3 sec latency 346.304 ms 2024-09-10T11:01:03.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:03 smithi133 ceph-mon[35282]: pgmap v479: 97 pgs: 97 active+clean; 886 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 669 op/s 2024-09-10T11:01:04.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:03 smithi103 ceph-mon[35160]: pgmap v479: 97 pgs: 97 active+clean; 886 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 669 op/s 2024-09-10T11:01:04.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:03 smithi007 ceph-mon[30873]: pgmap v479: 97 pgs: 97 active+clean; 886 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 669 op/s 2024-09-10T11:01:04.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 89659 22.25 MB/sec execute 4 sec latency 123.684 ms 2024-09-10T11:01:04.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:04 smithi133 ceph-mon[35282]: osdmap e105: 12 total, 12 up, 12 in 2024-09-10T11:01:05.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:04 smithi103 ceph-mon[35160]: osdmap e105: 12 total, 12 up, 12 in 2024-09-10T11:01:05.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:04 smithi007 ceph-mon[30873]: osdmap e105: 12 total, 12 up, 12 in 2024-09-10T11:01:05.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 89959 21.17 MB/sec execute 5 sec latency 260.167 ms 2024-09-10T11:01:06.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:05 smithi103 ceph-mon[35160]: pgmap v481: 97 pgs: 97 active+clean; 886 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 803 op/s 2024-09-10T11:01:06.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:05 smithi007 ceph-mon[30873]: pgmap v481: 97 pgs: 97 active+clean; 886 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 803 op/s 2024-09-10T11:01:06.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:05 smithi133 ceph-mon[35282]: pgmap v481: 97 pgs: 97 active+clean; 886 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 803 op/s 2024-09-10T11:01:06.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 90457 22.08 MB/sec execute 6 sec latency 323.475 ms 2024-09-10T11:01:07.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 91306 24.22 MB/sec execute 7 sec latency 92.756 ms 2024-09-10T11:01:08.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:07 smithi103 ceph-mon[35160]: pgmap v482: 97 pgs: 16 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-09-10T11:01:08.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:07 smithi007 ceph-mon[30873]: pgmap v482: 97 pgs: 16 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-09-10T11:01:08.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:07 smithi133 ceph-mon[35282]: pgmap v482: 97 pgs: 16 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-09-10T11:01:08.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 92397 29.41 MB/sec execute 8 sec latency 56.347 ms 2024-09-10T11:01:09.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 93229 32.09 MB/sec execute 9 sec latency 97.946 ms 2024-09-10T11:01:10.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:09 smithi103 ceph-mon[35160]: pgmap v483: 97 pgs: 16 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-09-10T11:01:10.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:09 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:01:10.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:09 smithi007 ceph-mon[30873]: pgmap v483: 97 pgs: 16 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-09-10T11:01:10.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:09 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:01:10.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:09 smithi133 ceph-mon[35282]: pgmap v483: 97 pgs: 16 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 52 active+clean; 883 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-09-10T11:01:10.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:09 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:01:10.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 93818 31.04 MB/sec execute 10 sec latency 155.370 ms 2024-09-10T11:01:11.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 94333 30.78 MB/sec execute 11 sec latency 234.914 ms 2024-09-10T11:01:12.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:11 smithi103 ceph-mon[35160]: pgmap v484: 97 pgs: 97 active+clean; 913 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-10T11:01:12.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:11 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:01:12.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:11 smithi007 ceph-mon[30873]: pgmap v484: 97 pgs: 97 active+clean; 913 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-10T11:01:12.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:11 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:01:12.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:11 smithi133 ceph-mon[35282]: pgmap v484: 97 pgs: 97 active+clean; 913 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-10T11:01:12.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:11 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:01:12.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 95370 32.68 MB/sec execute 12 sec latency 102.028 ms 2024-09-10T11:01:13.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:12 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:01:13.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:12 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:01:13.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:12 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T11:01:13.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:12 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:01:13.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:12 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:01:13.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:12 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T11:01:13.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:12 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:01:13.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:12 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:01:13.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:12 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T11:01:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.569+0000 7fc8af3b9640 1 -- 172.21.15.7:0/2939799580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a8102c80 msgr2=0x7fc8a8103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.569+0000 7fc8af3b9640 1 --2- 172.21.15.7:0/2939799580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a8102c80 0x7fc8a8103060 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009900 tx=0x7fc8a402a480 comp rx=0 tx=0).stop 2024-09-10T11:01:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.569+0000 7fc8af3b9640 1 -- 172.21.15.7:0/2939799580 shutdown_connections 2024-09-10T11:01:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.569+0000 7fc8af3b9640 1 --2- 172.21.15.7:0/2939799580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a810c0d0 0x7fc8a810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.569+0000 7fc8af3b9640 1 --2- 172.21.15.7:0/2939799580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a81035a0 0x7fc8a810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.569+0000 7fc8af3b9640 1 --2- 172.21.15.7:0/2939799580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a8102c80 0x7fc8a8103060 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.569+0000 7fc8af3b9640 1 -- 172.21.15.7:0/2939799580 >> 172.21.15.7:0/2939799580 conn(0x7fc8a8077990 msgr2=0x7fc8a80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.569+0000 7fc8af3b9640 1 -- 172.21.15.7:0/2939799580 shutdown_connections 2024-09-10T11:01:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.570+0000 7fc8af3b9640 1 -- 172.21.15.7:0/2939799580 wait complete. 2024-09-10T11:01:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.571+0000 7fc8af3b9640 1 Processor -- start 2024-09-10T11:01:13.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.571+0000 7fc8af3b9640 1 -- start start 2024-09-10T11:01:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.571+0000 7fc8af3b9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a8102c80 0x7fc8a81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.571+0000 7fc8af3b9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a81035a0 0x7fc8a8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.571+0000 7fc8af3b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a810c0d0 0x7fc8a819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.571+0000 7fc8af3b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a8111f00 con 0x7fc8a810c0d0 2024-09-10T11:01:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8af3b9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a8111d80 con 0x7fc8a8102c80 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8af3b9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a8112080 con 0x7fc8a81035a0 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8ac954640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a81035a0 0x7fc8a8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8ac954640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a81035a0 0x7fc8a8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:60106/0 (socket says 172.21.15.7:60106) 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8ac954640 1 -- 172.21.15.7:0/3222607443 learned_addr learned my addr 172.21.15.7:0/3222607443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8ad956640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a810c0d0 0x7fc8a819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8ad155640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a8102c80 0x7fc8a81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8ac954640 1 -- 172.21.15.7:0/3222607443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a8102c80 msgr2=0x7fc8a81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8ac954640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a8102c80 0x7fc8a81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8ac954640 1 -- 172.21.15.7:0/3222607443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a810c0d0 msgr2=0x7fc8a819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8ac954640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a810c0d0 0x7fc8a819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.572+0000 7fc8ac954640 1 -- 172.21.15.7:0/3222607443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a819d3e0 con 0x7fc8a81035a0 2024-09-10T11:01:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.573+0000 7fc8ac954640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a81035a0 0x7fc8a8198930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc89800e9f0 tx=0x7fc89800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.574+0000 7fc89e7fc640 1 -- 172.21.15.7:0/3222607443 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89800ccc0 con 0x7fc8a81035a0 2024-09-10T11:01:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.574+0000 7fc89e7fc640 1 -- 172.21.15.7:0/3222607443 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc89800ce60 con 0x7fc8a81035a0 2024-09-10T11:01:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.575+0000 7fc89e7fc640 1 -- 172.21.15.7:0/3222607443 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc898010690 con 0x7fc8a81035a0 2024-09-10T11:01:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.575+0000 7fc8af3b9640 1 -- 172.21.15.7:0/3222607443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a819d6d0 con 0x7fc8a81035a0 2024-09-10T11:01:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.575+0000 7fc8af3b9640 1 -- 172.21.15.7:0/3222607443 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a81a4f60 con 0x7fc8a81035a0 2024-09-10T11:01:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.579+0000 7fc89e7fc640 1 -- 172.21.15.7:0/3222607443 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc898004680 con 0x7fc8a81035a0 2024-09-10T11:01:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.581+0000 7fc8b2226740 1 Processor -- start 2024-09-10T11:01:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc89e7fc640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc8940a2e20 0x7fc8940a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8b2226740 1 -- start start 2024-09-10T11:01:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 0x5562508beed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508bf410 0x5562508c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 0x5562508cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8b2226740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5562508232f0 con 0x5562508beaf0 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8b2226740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5562507cb270 con 0x5562508c9cb0 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8b2226740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55625081f2b0 con 0x5562508bf410 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8ad956640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc8940a2e20 0x7fc8940a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc89e7fc640 1 -- 172.21.15.7:0/3222607443 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8608+0+0 (secure 0 0 0) 0x7fc8980cf850 con 0x7fc8a81035a0 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8ad155640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 0x5562508beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8ad155640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 0x5562508beed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57158/0 (socket says 172.21.15.7:57158) 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8ad155640 1 -- 172.21.15.7:0/2887690666 learned_addr learned my addr 172.21.15.7:0/2887690666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8ac954640 1 --2- 172.21.15.7:0/2887690666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508bf410 0x5562508c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.582+0000 7fc8ad956640 1 --2- 172.21.15.7:0/2887690666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 0x5562508cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.583+0000 7fc8ac954640 1 -- 172.21.15.7:0/2887690666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 msgr2=0x5562508cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.583+0000 7fc8ac954640 1 --2- 172.21.15.7:0/2887690666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 0x5562508cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.583+0000 7fc8ac954640 1 -- 172.21.15.7:0/2887690666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 msgr2=0x5562508beed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.583+0000 7fc8ac954640 1 --2- 172.21.15.7:0/2887690666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 0x5562508beed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.583+0000 7fc8ac954640 1 -- 172.21.15.7:0/2887690666 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562507ca340 con 0x5562508bf410 2024-09-10T11:01:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.583+0000 7fc8ad155640 1 --2- 172.21.15.7:0/2887690666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 0x5562508beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:01:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.583+0000 7fc8ad956640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc8940a2e20 0x7fc8940a52e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0005c40 tx=0x7fc8a0009d70 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.583+0000 7fc8ac954640 1 --2- 172.21.15.7:0/2887690666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508bf410 0x5562508c9770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc8980ccf10 tx=0x7fc89804a7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8edb2e94f4341426 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.583+0000 7fc8937fe640 1 -- 172.21.15.7:0/2887690666 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89809bce0 con 0x5562508bf410 2024-09-10T11:01:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.584+0000 7fc8937fe640 1 -- 172.21.15.7:0/2887690666 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8980a0090 con 0x5562508bf410 2024-09-10T11:01:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.584+0000 7fc8937fe640 1 -- 172.21.15.7:0/2887690666 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89801f340 con 0x5562508bf410 2024-09-10T11:01:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 -- 172.21.15.7:0/2887690666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508bf410 msgr2=0x5562508c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2887690666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508bf410 0x5562508c9770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc8980ccf10 tx=0x7fc89804a7f0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 -- 172.21.15.7:0/2887690666 shutdown_connections 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2887690666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 0x5562508cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2887690666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508bf410 0x5562508c9770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2887690666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 0x5562508beed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 -- 172.21.15.7:0/2887690666 >> 172.21.15.7:0/2887690666 conn(0x5562508bb060 msgr2=0x5562508bb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 -- 172.21.15.7:0/2887690666 shutdown_connections 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 -- 172.21.15.7:0/2887690666 wait complete. 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 -- start start 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.585+0000 7fc8b2226740 1 Processor -- start 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.586+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 0x5562508c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.586+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508bf410 0x5562508c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.586+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 0x5562508c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.586+0000 7fc8b2226740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5562508cc880 con 0x5562508beaf0 2024-09-10T11:01:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.586+0000 7fc8b2226740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5562507ac290 con 0x5562508bf410 2024-09-10T11:01:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.586+0000 7fc8b2226740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5562508cca00 con 0x5562508c9cb0 2024-09-10T11:01:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.586+0000 7fc8b2226740 10 client.0 _ll_register_callbacks cb 0x5562507dd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:01:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ac954640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508bf410 0x5562508c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ac954640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508bf410 0x5562508c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:35194/0 (socket says 172.21.15.7:35194) 2024-09-10T11:01:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ac954640 1 -- 172.21.15.7:0/3374361335 learned_addr learned my addr 172.21.15.7:0/3374361335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ad956640 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 0x5562508c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ad155640 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 0x5562508c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ac954640 1 -- 172.21.15.7:0/3374361335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 msgr2=0x5562508c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ac954640 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 0x5562508c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ac954640 1 -- 172.21.15.7:0/3374361335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 msgr2=0x5562508c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ac954640 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 0x5562508c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ac954640 1 -- 172.21.15.7:0/3374361335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556250536210 con 0x5562508bf410 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ad956640 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 0x5562508c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc8ac954640 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508bf410 0x5562508c70e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc89800c6a0 tx=0x7fc898009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5c4dcc127e2509c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.587+0000 7fc890ff9640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.588+0000 7fc890ff9640 1 -- 172.21.15.7:0/3374361335 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89809bce0 con 0x5562508bf410 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.588+0000 7fc890ff9640 1 -- 172.21.15.7:0/3374361335 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8608+0+0 (secure 0 0 0) 0x7fc898098350 con 0x5562508bf410 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.588+0000 7fc8b2226740 1 -- 172.21.15.7:0/3374361335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8980a0090 con 0x5562508bf410 2024-09-10T11:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.588+0000 7fc890ff9640 1 -- 172.21.15.7:0/3374361335 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc8980b9050 con 0x5562508bf410 2024-09-10T11:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.589+0000 7fc8b2226740 10 client.26099 fetch_fsmap learned FSMap version 50 2024-09-10T11:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.589+0000 7fc8b2226740 1 -- 172.21.15.7:0/3374361335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556250602620 con 0x5562508bf410 2024-09-10T11:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.589+0000 7fc890ff9640 1 -- 172.21.15.7:0/3374361335 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8980a0090 con 0x5562508bf410 2024-09-10T11:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.589+0000 7fc890ff9640 1 -- 172.21.15.7:0/3374361335 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7fc89809b1c0 con 0x5562508bf410 2024-09-10T11:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.589+0000 7fc8b2226740 10 client.26099 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T11:01:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.589+0000 7fc8b2226740 10 client.26099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:01:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.589+0000 7fc8b2226740 20 client.26099 populate_metadata read hostname 'smithi007' 2024-09-10T11:01:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.589+0000 7fc8b2226740 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5562508d9a90 0x5562508dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.589+0000 7fc8b2226740 4 client.26099 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:01:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.589+0000 7fc8b2226740 1 -- 172.21.15.7:0/3374361335 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556250830a20 con 0x5562508d9a90 2024-09-10T11:01:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.590+0000 7fc8ad155640 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5562508d9a90 0x5562508dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.591+0000 7fc8ad155640 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5562508d9a90 0x5562508dbe80 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551e718b2ee892bc server_cookie=44109badac781511 in_seq=0 out_seq=0 2024-09-10T11:01:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.591+0000 7fc890ff9640 10 client.26099 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.594+0000 7fc890ff9640 1 -- 172.21.15.7:0/3374361335 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556250830a20 con 0x5562508d9a90 2024-09-10T11:01:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.594+0000 7fc890ff9640 10 client.26099 handle_command_reply: tid=0 2024-09-10T11:01:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.594+0000 7fc8b2226740 1 client.26099 shutdown 2024-09-10T11:01:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.596+0000 7fc8b2226740 1 -- 172.21.15.7:0/3374361335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508bf410 msgr2=0x5562508c70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.596+0000 7fc8b2226740 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508bf410 0x5562508c70e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc89800c6a0 tx=0x7fc898009310 comp rx=0 tx=0).stop 2024-09-10T11:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.596+0000 7fc8b2226740 1 -- 172.21.15.7:0/3374361335 shutdown_connections 2024-09-10T11:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.596+0000 7fc8b2226740 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5562508d9a90 0x5562508dbe80 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.596+0000 7fc8b2226740 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 0x5562508c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.596+0000 7fc8b2226740 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508bf410 0x5562508c70e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.596+0000 7fc8b2226740 1 --2- 172.21.15.7:0/3374361335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508beaf0 0x5562508c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.596+0000 7fc8b2226740 1 -- 172.21.15.7:0/3374361335 >> 172.21.15.7:0/3374361335 conn(0x5562508bb060 msgr2=0x5562508c0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.596+0000 7fc890ff9640 0 client.26099 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.597+0000 7fc8b2226740 1 -- 172.21.15.7:0/3374361335 shutdown_connections 2024-09-10T11:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.598+0000 7fc8b2226740 1 -- 172.21.15.7:0/3374361335 wait complete. 2024-09-10T11:01:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.598+0000 7fc8b2226740 20 client.26099 trim_cache size 0 max 16384 2024-09-10T11:01:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.622+0000 7fc8b2226740 1 Processor -- start 2024-09-10T11:01:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.622+0000 7fc8b2226740 1 -- start start 2024-09-10T11:01:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.622+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508beaf0 0x5562508c7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.622+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 0x5562508e1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8b2226740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a81add50 con 0x5562508bf410 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8b2226740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a81ab590 con 0x5562508beaf0 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8b2226740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a81a93c0 con 0x5562508c9cb0 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ac954640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ad155640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508beaf0 0x5562508c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ac954640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57184/0 (socket says 172.21.15.7:57184) 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ad155640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508beaf0 0x5562508c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:35196/0 (socket says 172.21.15.7:35196) 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ad956640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 0x5562508e1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ac954640 1 -- 172.21.15.7:0/2483914755 learned_addr learned my addr 172.21.15.7:0/2483914755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ac954640 1 -- 172.21.15.7:0/2483914755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 msgr2=0x5562508e1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ac954640 1 --2- 172.21.15.7:0/2483914755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 0x5562508e1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ac954640 1 -- 172.21.15.7:0/2483914755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508beaf0 msgr2=0x5562508c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ac954640 1 --2- 172.21.15.7:0/2483914755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508beaf0 0x5562508c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ac954640 1 -- 172.21.15.7:0/2483914755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562508bfc90 con 0x5562508bf410 2024-09-10T11:01:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ad155640 1 --2- 172.21.15.7:0/2483914755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508beaf0 0x5562508c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:01:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc8ac954640 1 --2- 172.21.15.7:0/2483914755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508c8b10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc89809b9e0 tx=0x7fc89809a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da1efe90e954d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.623+0000 7fc892ffd640 1 -- 172.21.15.7:0/2483914755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc898050040 con 0x5562508bf410 2024-09-10T11:01:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc892ffd640 1 -- 172.21.15.7:0/2483914755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8980a0090 con 0x5562508bf410 2024-09-10T11:01:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc892ffd640 1 -- 172.21.15.7:0/2483914755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89801f340 con 0x5562508bf410 2024-09-10T11:01:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc8b2226740 1 -- 172.21.15.7:0/2483914755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 msgr2=0x5562508c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2483914755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508c8b10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc89809b9e0 tx=0x7fc89809a1f0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc8b2226740 1 -- 172.21.15.7:0/2483914755 shutdown_connections 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2483914755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508c9cb0 0x5562508e1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2483914755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508c8b10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2483914755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508beaf0 0x5562508c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc8b2226740 1 -- 172.21.15.7:0/2483914755 >> 172.21.15.7:0/2483914755 conn(0x5562508bb060 msgr2=0x5562508c8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc8b2226740 1 -- 172.21.15.7:0/2483914755 shutdown_connections 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.624+0000 7fc8b2226740 1 -- 172.21.15.7:0/2483914755 wait complete. 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.625+0000 7fc8b2226740 1 -- start start 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.625+0000 7fc8b2226740 1 Processor -- start 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.625+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508beaf0 0x5562508cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.625+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8b2226740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 0x5562509423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8b2226740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a81a9a50 con 0x5562508bf410 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8b2226740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a81aaec0 con 0x5562508c9cb0 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8b2226740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a81aa120 con 0x5562508beaf0 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8b2226740 10 client.0 _ll_register_callbacks cb 0x7fc8a00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ac954640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ac954640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57196/0 (socket says 172.21.15.7:57196) 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ad956640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 0x5562509423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ac954640 1 -- 172.21.15.7:0/2626673346 learned_addr learned my addr 172.21.15.7:0/2626673346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ad155640 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508beaf0 0x5562508cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ad155640 1 -- 172.21.15.7:0/2626673346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 msgr2=0x5562509423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ad155640 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 0x5562509423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ad155640 1 -- 172.21.15.7:0/2626673346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 msgr2=0x5562508cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ad155640 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ad155640 1 -- 172.21.15.7:0/2626673346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562509429d0 con 0x5562508beaf0 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ad956640 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 0x5562509423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ac954640 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc8ad155640 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508beaf0 0x5562508cd1f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4005dc0 tx=0x7fc8a40043e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5cd1e2a43bde395 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.626+0000 7fc890ff9640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc890ff9640 1 -- 172.21.15.7:0/2626673346 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4035040 con 0x5562508beaf0 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc8b2226740 1 -- 172.21.15.7:0/2626673346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8980a0090 con 0x5562508beaf0 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc890ff9640 1 -- 172.21.15.7:0/2626673346 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8608+0+0 (secure 0 0 0) 0x7fc8a4005450 con 0x5562508beaf0 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc890ff9640 1 -- 172.21.15.7:0/2626673346 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc8a4054050 con 0x5562508beaf0 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc8b2226740 10 client.26140 fetch_fsmap learned FSMap version 50 2024-09-10T11:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc8b2226740 1 -- 172.21.15.7:0/2626673346 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556250943d70 con 0x5562508beaf0 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc890ff9640 1 -- 172.21.15.7:0/2626673346 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8980a0090 con 0x5562508beaf0 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc890ff9640 1 -- 172.21.15.7:0/2626673346 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7fc8a4031750 con 0x5562508beaf0 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc8b2226740 10 client.26140 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc8b2226740 10 client.26140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc8b2226740 20 client.26140 populate_metadata read hostname 'smithi007' 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.627+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5562509444f0 0x556250946910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.628+0000 7fc8b2226740 4 client.26140 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.628+0000 7fc8b2226740 1 -- 172.21.15.7:0/2626673346 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55625082ad70 con 0x5562509444f0 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.628+0000 7fc8ac954640 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5562509444f0 0x556250946910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.628+0000 7fc890ff9640 10 client.26140 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.628+0000 7fc8ac954640 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5562509444f0 0x556250946910 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d31cc9e2c1c09fb server_cookie=fe94463d76d19d8c in_seq=0 out_seq=0 2024-09-10T11:01:13.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 96083 34.26 MB/sec execute 13 sec latency 178.479 ms 2024-09-10T11:01:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.956+0000 7fc890ff9640 1 -- 172.21.15.7:0/2626673346 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55625082ad70 con 0x5562509444f0 2024-09-10T11:01:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.956+0000 7fc890ff9640 10 client.26140 handle_command_reply: tid=0 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 client.26140 shutdown 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 -- 172.21.15.7:0/2626673346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508beaf0 msgr2=0x5562508cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508beaf0 0x5562508cd1f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4005dc0 tx=0x7fc8a40043e0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 -- 172.21.15.7:0/2626673346 shutdown_connections 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x5562509444f0 0x556250946910 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5562508c9cb0 0x5562509423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc890ff9640 0 client.26140 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562508bf410 0x5562508cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 --2- 172.21.15.7:0/2626673346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562508beaf0 0x5562508cd1f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 -- 172.21.15.7:0/2626673346 >> 172.21.15.7:0/2626673346 conn(0x5562508bb060 msgr2=0x5562508ba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 -- 172.21.15.7:0/2626673346 shutdown_connections 2024-09-10T11:01:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 1 -- 172.21.15.7:0/2626673346 wait complete. 2024-09-10T11:01:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.957+0000 7fc8b2226740 20 client.26140 trim_cache size 0 max 16384 2024-09-10T11:01:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T11:01:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T11:01:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T11:01:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T11:01:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.958+0000 7fc8af3b9640 1 -- 172.21.15.7:0/3222607443 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc8940a2e20 msgr2=0x7fc8940a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.958+0000 7fc8af3b9640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc8940a2e20 0x7fc8940a52e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0005c40 tx=0x7fc8a0009d70 comp rx=0 tx=0).stop 2024-09-10T11:01:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.958+0000 7fc8af3b9640 1 -- 172.21.15.7:0/3222607443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a81035a0 msgr2=0x7fc8a8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.958+0000 7fc8af3b9640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a81035a0 0x7fc8a8198930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc89800e9f0 tx=0x7fc89800eeb0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.958+0000 7fc8af3b9640 1 -- 172.21.15.7:0/3222607443 shutdown_connections 2024-09-10T11:01:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.958+0000 7fc8af3b9640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc8940a2e20 0x7fc8940a52e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.958+0000 7fc8af3b9640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a810c0d0 0x7fc8a819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.958+0000 7fc8af3b9640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8a81035a0 0x7fc8a8198930 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.958+0000 7fc8af3b9640 1 --2- 172.21.15.7:0/3222607443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8a8102c80 0x7fc8a81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.958+0000 7fc8af3b9640 1 -- 172.21.15.7:0/3222607443 >> 172.21.15.7:0/3222607443 conn(0x7fc8a8077990 msgr2=0x7fc8a80fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.959+0000 7fc8af3b9640 1 -- 172.21.15.7:0/3222607443 shutdown_connections 2024-09-10T11:01:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:13.959+0000 7fc8af3b9640 1 -- 172.21.15.7:0/3222607443 wait complete. 2024-09-10T11:01:14.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:13 smithi103 ceph-mon[35160]: pgmap v485: 97 pgs: 97 active+clean; 913 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-10T11:01:14.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:13 smithi103 ceph-mon[35160]: osdmap e106: 12 total, 12 up, 12 in 2024-09-10T11:01:14.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:13 smithi007 ceph-mon[30873]: pgmap v485: 97 pgs: 97 active+clean; 913 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-10T11:01:14.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:13 smithi007 ceph-mon[30873]: osdmap e106: 12 total, 12 up, 12 in 2024-09-10T11:01:14.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:13 smithi133 ceph-mon[35282]: pgmap v485: 97 pgs: 97 active+clean; 913 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-10T11:01:14.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:13 smithi133 ceph-mon[35282]: osdmap e106: 12 total, 12 up, 12 in 2024-09-10T11:01:14.886 INFO:tasks.workunit.client.0.smithi007.stdout: 10 96661 33.32 MB/sec execute 14 sec latency 193.854 ms 2024-09-10T11:01:15.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 97376 33.63 MB/sec execute 15 sec latency 67.101 ms 2024-09-10T11:01:16.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:15 smithi103 ceph-mon[35160]: pgmap v487: 97 pgs: 97 active+clean; 913 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-10T11:01:16.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:15 smithi007 ceph-mon[30873]: pgmap v487: 97 pgs: 97 active+clean; 913 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-10T11:01:16.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:15 smithi133 ceph-mon[35282]: pgmap v487: 97 pgs: 97 active+clean; 913 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-10T11:01:16.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 98139 33.75 MB/sec execute 16 sec latency 259.536 ms 2024-09-10T11:01:17.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 99095 35.10 MB/sec execute 17 sec latency 78.600 ms 2024-09-10T11:01:18.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:17 smithi103 ceph-mon[35160]: pgmap v488: 97 pgs: 97 active+clean; 929 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-10T11:01:18.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:17 smithi007 ceph-mon[30873]: pgmap v488: 97 pgs: 97 active+clean; 929 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-10T11:01:18.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:17 smithi133 ceph-mon[35282]: pgmap v488: 97 pgs: 97 active+clean; 929 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-10T11:01:18.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 99866 35.68 MB/sec execute 18 sec latency 76.669 ms 2024-09-10T11:01:19.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 100666 35.96 MB/sec execute 19 sec latency 76.882 ms 2024-09-10T11:01:20.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:19 smithi103 ceph-mon[35160]: pgmap v489: 97 pgs: 97 active+clean; 929 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-10T11:01:20.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:19 smithi007 ceph-mon[30873]: pgmap v489: 97 pgs: 97 active+clean; 929 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-10T11:01:20.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:19 smithi133 ceph-mon[35282]: pgmap v489: 97 pgs: 97 active+clean; 929 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-10T11:01:20.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 101615 36.90 MB/sec execute 20 sec latency 53.683 ms 2024-09-10T11:01:21.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 102024 35.90 MB/sec execute 21 sec latency 347.071 ms 2024-09-10T11:01:22.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:21 smithi103 ceph-mon[35160]: pgmap v490: 97 pgs: 97 active+clean; 953 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 936 op/s 2024-09-10T11:01:22.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:21 smithi007 ceph-mon[30873]: pgmap v490: 97 pgs: 97 active+clean; 953 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 936 op/s 2024-09-10T11:01:22.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:21 smithi133 ceph-mon[35282]: pgmap v490: 97 pgs: 97 active+clean; 953 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 936 op/s 2024-09-10T11:01:22.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 102854 36.33 MB/sec execute 22 sec latency 69.340 ms 2024-09-10T11:01:23.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 103698 36.84 MB/sec execute 23 sec latency 63.251 ms 2024-09-10T11:01:24.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:23 smithi103 ceph-mon[35160]: pgmap v491: 97 pgs: 97 active+clean; 953 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 936 op/s 2024-09-10T11:01:24.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:23 smithi007 ceph-mon[30873]: pgmap v491: 97 pgs: 97 active+clean; 953 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 936 op/s 2024-09-10T11:01:24.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:23 smithi133 ceph-mon[35282]: pgmap v491: 97 pgs: 97 active+clean; 953 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 936 op/s 2024-09-10T11:01:24.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 104491 37.15 MB/sec execute 24 sec latency 74.826 ms 2024-09-10T11:01:25.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 105280 37.08 MB/sec execute 25 sec latency 49.638 ms 2024-09-10T11:01:26.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:25 smithi103 ceph-mon[35160]: pgmap v492: 97 pgs: 97 active+clean; 953 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 902 op/s 2024-09-10T11:01:26.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:25 smithi007 ceph-mon[30873]: pgmap v492: 97 pgs: 97 active+clean; 953 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 902 op/s 2024-09-10T11:01:26.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:25 smithi133 ceph-mon[35282]: pgmap v492: 97 pgs: 97 active+clean; 953 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 902 op/s 2024-09-10T11:01:26.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 105824 36.67 MB/sec execute 26 sec latency 392.692 ms 2024-09-10T11:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.070+0000 7f4a11884640 1 -- 172.21.15.7:0/2479773530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c1009d0 msgr2=0x7f4a0c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.070+0000 7f4a11884640 1 --2- 172.21.15.7:0/2479773530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c1009d0 0x7f4a0c100e30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009930 tx=0x7f4a0002a420 comp rx=0 tx=0).stop 2024-09-10T11:01:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.070+0000 7f4a11884640 1 -- 172.21.15.7:0/2479773530 shutdown_connections 2024-09-10T11:01:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.070+0000 7f4a11884640 1 --2- 172.21.15.7:0/2479773530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a0c101370 0x7f4a0c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.070+0000 7f4a11884640 1 --2- 172.21.15.7:0/2479773530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c1009d0 0x7f4a0c100e30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.070+0000 7f4a11884640 1 --2- 172.21.15.7:0/2479773530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0c1069e0 0x7f4a0c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.070+0000 7f4a11884640 1 -- 172.21.15.7:0/2479773530 >> 172.21.15.7:0/2479773530 conn(0x7f4a0c0fc6b0 msgr2=0x7f4a0c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.070+0000 7f4a11884640 1 -- 172.21.15.7:0/2479773530 shutdown_connections 2024-09-10T11:01:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.070+0000 7f4a11884640 1 -- 172.21.15.7:0/2479773530 wait complete. 2024-09-10T11:01:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.070+0000 7f4a11884640 1 Processor -- start 2024-09-10T11:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a11884640 1 -- start start 2024-09-10T11:01:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a11884640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0c1009d0 0x7f4a0c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a11884640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c101370 0x7f4a0c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a11884640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a0c1069e0 0x7f4a0c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a11884640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c110e20 con 0x7f4a0c101370 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a11884640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c110ca0 con 0x7f4a0c1069e0 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a11884640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c110fa0 con 0x7f4a0c1009d0 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a0a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c101370 0x7f4a0c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a0a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c101370 0x7f4a0c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57626/0 (socket says 172.21.15.7:57626) 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a0a7fc640 1 -- 172.21.15.7:0/1802917599 learned_addr learned my addr 172.21.15.7:0/1802917599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a0b7fe640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a0c1069e0 0x7f4a0c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a0affd640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0c1009d0 0x7f4a0c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a0a7fc640 1 -- 172.21.15.7:0/1802917599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0c1009d0 msgr2=0x7f4a0c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a0a7fc640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0c1009d0 0x7f4a0c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a0a7fc640 1 -- 172.21.15.7:0/1802917599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a0c1069e0 msgr2=0x7f4a0c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a0a7fc640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a0c1069e0 0x7f4a0c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.071+0000 7f4a0a7fc640 1 -- 172.21.15.7:0/1802917599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c19f540 con 0x7f4a0c101370 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.072+0000 7f4a0a7fc640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c101370 0x7f4a0c19aa90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009230 tx=0x7f4a00004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.072+0000 7f4a10882640 1 -- 172.21.15.7:0/1802917599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0003f030 con 0x7f4a0c101370 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.072+0000 7f4a11884640 1 -- 172.21.15.7:0/1802917599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c19f7d0 con 0x7f4a0c101370 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.072+0000 7f4a10882640 1 -- 172.21.15.7:0/1802917599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a00003ad0 con 0x7f4a0c101370 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.072+0000 7f4a10882640 1 -- 172.21.15.7:0/1802917599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00035070 con 0x7f4a0c101370 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.072+0000 7f4a11884640 1 -- 172.21.15.7:0/1802917599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c1a70c0 con 0x7f4a0c101370 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.074+0000 7f4a10882640 1 -- 172.21.15.7:0/1802917599 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4a00042030 con 0x7f4a0c101370 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.074+0000 7f4a1461d740 1 Processor -- start 2024-09-10T11:01:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a1461d740 1 -- start start 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a787ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a792770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 0x55fb0a7950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a1461d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fb0a6ec2f0 con 0x55fb0a792cb0 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a1461d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fb0a694270 con 0x55fb0a787af0 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a1461d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fb0a6e82b0 con 0x55fb0a788410 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a0a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 0x55fb0a7950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a0b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a787ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a0b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a787ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:37934/0 (socket says 172.21.15.7:37934) 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a0b7fe640 1 -- 172.21.15.7:0/1017251127 learned_addr learned my addr 172.21.15.7:0/1017251127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a0b7fe640 1 -- 172.21.15.7:0/1017251127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 msgr2=0x55fb0a792770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a0b7fe640 1 --2- 172.21.15.7:0/1017251127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a792770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a0b7fe640 1 -- 172.21.15.7:0/1017251127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 msgr2=0x55fb0a7950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.075+0000 7f4a10882640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f49f40a2da0 0x7f49f40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.076+0000 7f4a0b7fe640 1 --2- 172.21.15.7:0/1017251127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 0x55fb0a7950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.076+0000 7f4a0affd640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f49f40a2da0 0x7f49f40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.076+0000 7f4a0b7fe640 1 -- 172.21.15.7:0/1017251127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb0a693340 con 0x55fb0a787af0 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.076+0000 7f4a10882640 1 -- 172.21.15.7:0/1802917599 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8564+0+0 (secure 0 0 0) 0x7f4a000efa10 con 0x7f4a0c101370 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.076+0000 7f4a0a7fc640 1 --2- 172.21.15.7:0/1017251127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 0x55fb0a7950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.076+0000 7f4a0b7fe640 1 --2- 172.21.15.7:0/1017251127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a787ed0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009fd0 tx=0x7f4a040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48036b7910fd4c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.076+0000 7f49f1ffb640 1 -- 172.21.15.7:0/1017251127 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04004d20 con 0x55fb0a787af0 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.076+0000 7f4a0affd640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f49f40a2da0 0x7f49f40a5260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b620 tx=0x7f49fc00b5b0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.076+0000 7f49f1ffb640 1 -- 172.21.15.7:0/1017251127 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a04004050 con 0x55fb0a787af0 2024-09-10T11:01:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.076+0000 7f49f1ffb640 1 -- 172.21.15.7:0/1017251127 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04005260 con 0x55fb0a787af0 2024-09-10T11:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.077+0000 7f4a1461d740 1 -- 172.21.15.7:0/1017251127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 msgr2=0x55fb0a787ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.077+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1017251127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a787ed0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009fd0 tx=0x7f4a040118d0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.077+0000 7f4a1461d740 1 -- 172.21.15.7:0/1017251127 shutdown_connections 2024-09-10T11:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.077+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1017251127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 0x55fb0a7950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.077+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1017251127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a792770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.077+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1017251127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a787ed0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.077+0000 7f4a1461d740 1 -- 172.21.15.7:0/1017251127 >> 172.21.15.7:0/1017251127 conn(0x55fb0a784060 msgr2=0x55fb0a784440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.077+0000 7f4a1461d740 1 -- 172.21.15.7:0/1017251127 shutdown_connections 2024-09-10T11:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.077+0000 7f4a1461d740 1 -- 172.21.15.7:0/1017251127 wait complete. 2024-09-10T11:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a1461d740 1 -- start start 2024-09-10T11:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a1461d740 1 Processor -- start 2024-09-10T11:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a78fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a790070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a792cb0 0x55fb0a7905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a1461d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fb0a795880 con 0x55fb0a787af0 2024-09-10T11:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a1461d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fb0a675290 con 0x55fb0a792cb0 2024-09-10T11:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a1461d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fb0a795a00 con 0x55fb0a788410 2024-09-10T11:01:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a78fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a1461d740 10 client.0 _ll_register_callbacks cb 0x55fb0a6a6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a0b7fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a792cb0 0x55fb0a7905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a78fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57630/0 (socket says 172.21.15.7:57630) 2024-09-10T11:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.078+0000 7f4a0affd640 1 -- 172.21.15.7:0/2917428115 learned_addr learned my addr 172.21.15.7:0/2917428115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f4a0a7fc640 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a790070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f4a0affd640 1 -- 172.21.15.7:0/2917428115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 msgr2=0x55fb0a790070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f4a0affd640 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a790070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f4a0affd640 1 -- 172.21.15.7:0/2917428115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a792cb0 msgr2=0x55fb0a7905b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f4a0affd640 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a792cb0 0x55fb0a7905b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f4a0affd640 1 -- 172.21.15.7:0/2917428115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb0a3ff210 con 0x55fb0a787af0 2024-09-10T11:01:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f4a0b7fe640 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a792cb0 0x55fb0a7905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:01:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f4a0affd640 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a78fb30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f49fc003d50 tx=0x7f49fc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969e70169d2a500b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f49eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:01:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f49eaffd640 1 -- 172.21.15.7:0/2917428115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc01e620 con 0x55fb0a787af0 2024-09-10T11:01:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f49eaffd640 1 -- 172.21.15.7:0/2917428115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8564+0+0 (secure 0 0 0) 0x7f49fc01e7c0 con 0x55fb0a787af0 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.079+0000 7f4a1461d740 1 -- 172.21.15.7:0/2917428115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a04004050 con 0x55fb0a787af0 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.080+0000 7f49eaffd640 1 -- 172.21.15.7:0/2917428115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f49fc01ea00 con 0x55fb0a787af0 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.080+0000 7f4a1461d740 10 client.16506 fetch_fsmap learned FSMap version 50 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.080+0000 7f4a1461d740 1 -- 172.21.15.7:0/2917428115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55fb0a4cb620 con 0x55fb0a787af0 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.080+0000 7f49eaffd640 1 -- 172.21.15.7:0/2917428115 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a04004050 con 0x55fb0a787af0 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.081+0000 7f49eaffd640 1 -- 172.21.15.7:0/2917428115 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7f49fc02d070 con 0x55fb0a787af0 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.081+0000 7f4a1461d740 10 client.16506 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.081+0000 7f4a1461d740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.081+0000 7f4a1461d740 20 client.16506 populate_metadata read hostname 'smithi007' 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.081+0000 7f4a1461d740 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fb0a7a2b00 0x55fb0a7a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.081+0000 7f4a1461d740 4 client.16506 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:01:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.081+0000 7f4a1461d740 1 -- 172.21.15.7:0/2917428115 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb0a6f9a20 con 0x55fb0a7a2b00 2024-09-10T11:01:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.081+0000 7f4a0a7fc640 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fb0a7a2b00 0x55fb0a7a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.083+0000 7f4a0a7fc640 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fb0a7a2b00 0x55fb0a7a4ef0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d7bdf89895d7f8 server_cookie=4526173f7ae98462 in_seq=0 out_seq=0 2024-09-10T11:01:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.083+0000 7f49eaffd640 10 client.16506 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.086+0000 7f49eaffd640 1 -- 172.21.15.7:0/2917428115 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb0a6f9a20 con 0x55fb0a7a2b00 2024-09-10T11:01:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.086+0000 7f49eaffd640 10 client.16506 handle_command_reply: tid=0 2024-09-10T11:01:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.086+0000 7f4a1461d740 1 client.16506 shutdown 2024-09-10T11:01:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.086+0000 7f4a1461d740 1 -- 172.21.15.7:0/2917428115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 msgr2=0x55fb0a78fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.086+0000 7f4a1461d740 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a78fb30 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f49fc003d50 tx=0x7f49fc00b390 comp rx=0 tx=0).stop 2024-09-10T11:01:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.086+0000 7f4a1461d740 1 -- 172.21.15.7:0/2917428115 shutdown_connections 2024-09-10T11:01:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.086+0000 7f4a1461d740 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fb0a7a2b00 0x55fb0a7a4ef0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.086+0000 7f4a1461d740 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a792cb0 0x55fb0a7905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.086+0000 7f49eaffd640 0 client.16506 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.086+0000 7f4a1461d740 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a790070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.087+0000 7f4a1461d740 1 --2- 172.21.15.7:0/2917428115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a78fb30 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.087+0000 7f4a1461d740 1 -- 172.21.15.7:0/2917428115 >> 172.21.15.7:0/2917428115 conn(0x55fb0a784060 msgr2=0x55fb0a789d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.087+0000 7f4a1461d740 1 -- 172.21.15.7:0/2917428115 shutdown_connections 2024-09-10T11:01:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.087+0000 7f4a1461d740 1 -- 172.21.15.7:0/2917428115 wait complete. 2024-09-10T11:01:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.088+0000 7f4a1461d740 20 client.16506 trim_cache size 0 max 16384 2024-09-10T11:01:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.112+0000 7f4a1461d740 1 Processor -- start 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a1461d740 1 -- start start 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a78a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a7903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 0x55fb0a791aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a1461d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1b23b0 con 0x55fb0a792cb0 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a1461d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1afbf0 con 0x55fb0a787af0 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a1461d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1ada60 con 0x55fb0a788410 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a0affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a78a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a0affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a78a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:37956/0 (socket says 172.21.15.7:37956) 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a0affd640 1 -- 172.21.15.7:0/1129976258 learned_addr learned my addr 172.21.15.7:0/1129976258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a0b7fe640 1 --2- 172.21.15.7:0/1129976258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 0x55fb0a791aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a0a7fc640 1 --2- 172.21.15.7:0/1129976258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a7903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a0affd640 1 -- 172.21.15.7:0/1129976258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 msgr2=0x55fb0a7903e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a0affd640 1 --2- 172.21.15.7:0/1129976258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a7903e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a0affd640 1 -- 172.21.15.7:0/1129976258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 msgr2=0x55fb0a791aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a0affd640 1 --2- 172.21.15.7:0/1129976258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 0x55fb0a791aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.113+0000 7f4a0affd640 1 -- 172.21.15.7:0/1129976258 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb0a783e90 con 0x55fb0a787af0 2024-09-10T11:01:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.114+0000 7f4a0affd640 1 --2- 172.21.15.7:0/1129976258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a78a560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49fc004010 tx=0x7f49fc00b4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0d471acd3982c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.114+0000 7f49f17fa640 1 -- 172.21.15.7:0/1129976258 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc03d070 con 0x55fb0a787af0 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.114+0000 7f49f17fa640 1 -- 172.21.15.7:0/1129976258 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49fc02e630 con 0x55fb0a787af0 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.114+0000 7f49f17fa640 1 -- 172.21.15.7:0/1129976258 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc02ebc0 con 0x55fb0a787af0 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.115+0000 7f4a1461d740 1 -- 172.21.15.7:0/1129976258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 msgr2=0x55fb0a78a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.115+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1129976258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a78a560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49fc004010 tx=0x7f49fc00b4e0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.115+0000 7f4a1461d740 1 -- 172.21.15.7:0/1129976258 shutdown_connections 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.115+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1129976258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a792cb0 0x55fb0a791aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.115+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1129976258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a788410 0x55fb0a7903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.115+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1129976258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a787af0 0x55fb0a78a560 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.115+0000 7f4a1461d740 1 -- 172.21.15.7:0/1129976258 >> 172.21.15.7:0/1129976258 conn(0x55fb0a784060 msgr2=0x55fb0a791390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.115+0000 7f4a1461d740 1 -- 172.21.15.7:0/1129976258 shutdown_connections 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.115+0000 7f4a1461d740 1 -- 172.21.15.7:0/1129976258 wait complete. 2024-09-10T11:01:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.116+0000 7f4a1461d740 1 -- start start 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.116+0000 7f4a1461d740 1 Processor -- start 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a79b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a788410 0x55fb0a79b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a1461d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a792cb0 0x55fb0a7965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a1461d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1ae0b0 con 0x55fb0a787af0 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a1461d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1af520 con 0x55fb0a788410 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a1461d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c1ae780 con 0x55fb0a792cb0 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a1461d740 10 client.0 _ll_register_callbacks cb 0x7f49fc005c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a79b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a0affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a79b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57650/0 (socket says 172.21.15.7:57650) 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a0affd640 1 -- 172.21.15.7:0/1499759838 learned_addr learned my addr 172.21.15.7:0/1499759838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a0b7fe640 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a792cb0 0x55fb0a7965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a0affd640 1 -- 172.21.15.7:0/1499759838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a792cb0 msgr2=0x55fb0a7965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a0affd640 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a792cb0 0x55fb0a7965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a0affd640 1 -- 172.21.15.7:0/1499759838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a788410 msgr2=0x55fb0a79b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a0affd640 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a788410 0x55fb0a79b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.117+0000 7f4a0affd640 1 -- 172.21.15.7:0/1499759838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb0a796de0 con 0x55fb0a787af0 2024-09-10T11:01:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.118+0000 7f4a0affd640 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a79b040 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f49fc032010 tx=0x7f49fc011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b532e7ab6eec59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.118+0000 7f49eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:01:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.119+0000 7f49eaffd640 1 -- 172.21.15.7:0/1499759838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc046060 con 0x55fb0a787af0 2024-09-10T11:01:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.119+0000 7f49eaffd640 1 -- 172.21.15.7:0/1499759838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8564+0+0 (secure 0 0 0) 0x7f49fc02e9e0 con 0x55fb0a787af0 2024-09-10T11:01:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.119+0000 7f4a1461d740 1 -- 172.21.15.7:0/1499759838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49fc02e630 con 0x55fb0a787af0 2024-09-10T11:01:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.120+0000 7f49eaffd640 1 -- 172.21.15.7:0/1499759838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f49fc03d050 con 0x55fb0a787af0 2024-09-10T11:01:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.120+0000 7f4a1461d740 10 client.16512 fetch_fsmap learned FSMap version 50 2024-09-10T11:01:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.120+0000 7f4a1461d740 1 -- 172.21.15.7:0/1499759838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55fb0a815c60 con 0x55fb0a787af0 2024-09-10T11:01:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.121+0000 7f49eaffd640 1 -- 172.21.15.7:0/1499759838 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49fc02e630 con 0x55fb0a787af0 2024-09-10T11:01:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.121+0000 7f49eaffd640 1 -- 172.21.15.7:0/1499759838 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7f49fc02d040 con 0x55fb0a787af0 2024-09-10T11:01:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.121+0000 7f4a1461d740 10 client.16512 fetch_fsmap finished waiting for FSMap version 50 2024-09-10T11:01:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.121+0000 7f4a1461d740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:01:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.121+0000 7f4a1461d740 20 client.16512 populate_metadata read hostname 'smithi007' 2024-09-10T11:01:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.121+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fb0a7a2b00 0x55fb0a818250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.121+0000 7f4a1461d740 4 client.16512 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T11:01:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.121+0000 7f4a1461d740 1 -- 172.21.15.7:0/1499759838 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb0a6f3d70 con 0x55fb0a7a2b00 2024-09-10T11:01:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.122+0000 7f4a0a7fc640 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fb0a7a2b00 0x55fb0a818250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.125+0000 7f4a0a7fc640 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fb0a7a2b00 0x55fb0a818250 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d327a172d115ed53 server_cookie=18fc03cbaecad840 in_seq=0 out_seq=0 2024-09-10T11:01:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.125+0000 7f49eaffd640 10 client.16512 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f49eaffd640 1 -- 172.21.15.7:0/1499759838 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb0a6f3d70 con 0x55fb0a7a2b00 2024-09-10T11:01:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f49eaffd640 10 client.16512 handle_command_reply: tid=0 2024-09-10T11:01:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 client.16512 shutdown 2024-09-10T11:01:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 -- 172.21.15.7:0/1499759838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 msgr2=0x55fb0a79b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a79b040 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f49fc032010 tx=0x7f49fc011d20 comp rx=0 tx=0).stop 2024-09-10T11:01:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 -- 172.21.15.7:0/1499759838 shutdown_connections 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fb0a7a2b00 0x55fb0a818250 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb0a792cb0 0x55fb0a7965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fb0a788410 0x55fb0a79b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 --2- 172.21.15.7:0/1499759838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fb0a787af0 0x55fb0a79b040 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 -- 172.21.15.7:0/1499759838 >> 172.21.15.7:0/1499759838 conn(0x55fb0a784060 msgr2=0x55fb0a7831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f49eaffd640 0 client.16512 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 -- 172.21.15.7:0/1499759838 shutdown_connections 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.599+0000 7f4a1461d740 1 -- 172.21.15.7:0/1499759838 wait complete. 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.600+0000 7f4a1461d740 20 client.16512 trim_cache size 0 max 16384 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T11:01:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T11:01:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T11:01:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T11:01:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 -- 172.21.15.7:0/1802917599 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f49f40a2da0 msgr2=0x7f49f40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f49f40a2da0 0x7f49f40a5260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b620 tx=0x7f49fc00b5b0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 -- 172.21.15.7:0/1802917599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c101370 msgr2=0x7f4a0c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c101370 0x7f4a0c19aa90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009230 tx=0x7f4a00004270 comp rx=0 tx=0).stop 2024-09-10T11:01:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 -- 172.21.15.7:0/1802917599 shutdown_connections 2024-09-10T11:01:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a0c1069e0 0x7f4a0c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f49f40a2da0 0x7f49f40a5260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c101370 0x7f4a0c19aa90 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 --2- 172.21.15.7:0/1802917599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a0c1009d0 0x7f4a0c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 -- 172.21.15.7:0/1802917599 >> 172.21.15.7:0/1802917599 conn(0x7f4a0c0fc6b0 msgr2=0x7f4a0c06a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 -- 172.21.15.7:0/1802917599 shutdown_connections 2024-09-10T11:01:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:27.601+0000 7f4a11884640 1 -- 172.21.15.7:0/1802917599 wait complete. 2024-09-10T11:01:27.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 106392 36.59 MB/sec execute 27 sec latency 330.754 ms 2024-09-10T11:01:28.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:27 smithi103 ceph-mon[35160]: pgmap v493: 97 pgs: 97 active+clean; 964 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2024-09-10T11:01:28.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:27 smithi007 ceph-mon[30873]: pgmap v493: 97 pgs: 97 active+clean; 964 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2024-09-10T11:01:28.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:27 smithi133 ceph-mon[35282]: pgmap v493: 97 pgs: 97 active+clean; 964 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 20 MiB/s wr, 1.11k op/s 2024-09-10T11:01:28.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 107299 37.28 MB/sec execute 28 sec latency 56.536 ms 2024-09-10T11:01:29.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 108035 37.40 MB/sec execute 29 sec latency 72.240 ms 2024-09-10T11:01:30.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:29 smithi103 ceph-mon[35160]: pgmap v494: 97 pgs: 97 active+clean; 964 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 674 op/s 2024-09-10T11:01:30.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:29 smithi007 ceph-mon[30873]: pgmap v494: 97 pgs: 97 active+clean; 964 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 674 op/s 2024-09-10T11:01:30.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:29 smithi133 ceph-mon[35282]: pgmap v494: 97 pgs: 97 active+clean; 964 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 674 op/s 2024-09-10T11:01:30.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 108843 37.42 MB/sec execute 30 sec latency 104.859 ms 2024-09-10T11:01:31.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 109656 37.47 MB/sec execute 31 sec latency 193.335 ms 2024-09-10T11:01:32.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:31 smithi103 ceph-mon[35160]: pgmap v495: 97 pgs: 97 active+clean; 969 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-09-10T11:01:32.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:31 smithi007 ceph-mon[30873]: pgmap v495: 97 pgs: 97 active+clean; 969 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-09-10T11:01:32.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:31 smithi133 ceph-mon[35282]: pgmap v495: 97 pgs: 97 active+clean; 969 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-09-10T11:01:32.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 110406 37.96 MB/sec execute 32 sec latency 196.486 ms 2024-09-10T11:01:33.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 111100 37.79 MB/sec execute 33 sec latency 132.364 ms 2024-09-10T11:01:34.027 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:33 smithi133 ceph-mon[35282]: pgmap v496: 97 pgs: 97 active+clean; 969 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 773 op/s 2024-09-10T11:01:34.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:33 smithi103 ceph-mon[35160]: pgmap v496: 97 pgs: 97 active+clean; 969 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 773 op/s 2024-09-10T11:01:34.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:33 smithi007 ceph-mon[30873]: pgmap v496: 97 pgs: 97 active+clean; 969 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 773 op/s 2024-09-10T11:01:34.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 111972 37.85 MB/sec execute 34 sec latency 222.826 ms 2024-09-10T11:01:35.073 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:34 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:01:35.084 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:34 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:01:35.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:34 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:01:35.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 112946 38.41 MB/sec execute 35 sec latency 53.736 ms 2024-09-10T11:01:36.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:35 smithi103 ceph-mon[35160]: pgmap v497: 97 pgs: 97 active+clean; 969 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 773 op/s 2024-09-10T11:01:36.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:35 smithi007 ceph-mon[30873]: pgmap v497: 97 pgs: 97 active+clean; 969 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 773 op/s 2024-09-10T11:01:36.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:35 smithi133 ceph-mon[35282]: pgmap v497: 97 pgs: 97 active+clean; 969 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 773 op/s 2024-09-10T11:01:36.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 113711 38.63 MB/sec execute 36 sec latency 116.210 ms 2024-09-10T11:01:37.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 114476 38.64 MB/sec execute 37 sec latency 113.430 ms 2024-09-10T11:01:38.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:37 smithi103 ceph-mon[35160]: pgmap v498: 97 pgs: 97 active+clean; 1001 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-09-10T11:01:38.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:37 smithi007 ceph-mon[30873]: pgmap v498: 97 pgs: 97 active+clean; 1001 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-09-10T11:01:38.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:37 smithi133 ceph-mon[35282]: pgmap v498: 97 pgs: 97 active+clean; 1001 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-09-10T11:01:38.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 115476 38.91 MB/sec execute 38 sec latency 70.031 ms 2024-09-10T11:01:39.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 116231 39.18 MB/sec execute 39 sec latency 46.285 ms 2024-09-10T11:01:40.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:39 smithi103 ceph-mon[35160]: pgmap v499: 97 pgs: 97 active+clean; 1001 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 763 op/s 2024-09-10T11:01:40.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:39 smithi007 ceph-mon[30873]: pgmap v499: 97 pgs: 97 active+clean; 1001 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 763 op/s 2024-09-10T11:01:40.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:39 smithi133 ceph-mon[35282]: pgmap v499: 97 pgs: 97 active+clean; 1001 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 763 op/s 2024-09-10T11:01:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.714+0000 7f98ee752640 1 -- 172.21.15.7:0/3176393510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98e81035e0 msgr2=0x7f98e810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.714+0000 7f98ee752640 1 --2- 172.21.15.7:0/3176393510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98e81035e0 0x7f98e810bac0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009900 tx=0x7f98dc02a480 comp rx=0 tx=0).stop 2024-09-10T11:01:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.714+0000 7f98ee752640 1 -- 172.21.15.7:0/3176393510 shutdown_connections 2024-09-10T11:01:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.714+0000 7f98ee752640 1 --2- 172.21.15.7:0/3176393510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98e810c110 0x7f98e810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.714+0000 7f98ee752640 1 --2- 172.21.15.7:0/3176393510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98e81035e0 0x7f98e810bac0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.714+0000 7f98ee752640 1 --2- 172.21.15.7:0/3176393510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e8102cc0 0x7f98e81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.714+0000 7f98ee752640 1 -- 172.21.15.7:0/3176393510 >> 172.21.15.7:0/3176393510 conn(0x7f98e80fc6b0 msgr2=0x7f98e80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.714+0000 7f98ee752640 1 -- 172.21.15.7:0/3176393510 shutdown_connections 2024-09-10T11:01:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.714+0000 7f98ee752640 1 -- 172.21.15.7:0/3176393510 wait complete. 2024-09-10T11:01:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.715+0000 7f98ee752640 1 Processor -- start 2024-09-10T11:01:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.715+0000 7f98ee752640 1 -- start start 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98ee752640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98e8102cc0 0x7f98e81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98ee752640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98e81035e0 0x7f98e81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98ee752640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e810c110 0x7f98e819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98ee752640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8111ef0 con 0x7f98e81035e0 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98ee752640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8111d70 con 0x7f98e810c110 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98ee752640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f98e8112070 con 0x7f98e8102cc0 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98e77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98e81035e0 0x7f98e81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98e77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98e81035e0 0x7f98e81988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58512/0 (socket says 172.21.15.7:58512) 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98e77fe640 1 -- 172.21.15.7:0/2213093266 learned_addr learned my addr 172.21.15.7:0/2213093266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98e7fff640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98e8102cc0 0x7f98e81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98e77fe640 1 -- 172.21.15.7:0/2213093266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98e8102cc0 msgr2=0x7f98e81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98e77fe640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98e8102cc0 0x7f98e81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98e77fe640 1 -- 172.21.15.7:0/2213093266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e810c110 msgr2=0x7f98e819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98e77fe640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e810c110 0x7f98e819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98e77fe640 1 -- 172.21.15.7:0/2213093266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e819d390 con 0x7f98e81035e0 2024-09-10T11:01:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.716+0000 7f98e77fe640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98e81035e0 0x7f98e81988e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f98dc006010 tx=0x7f98dc031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.717+0000 7f98e57fa640 1 -- 172.21.15.7:0/2213093266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc03f030 con 0x7f98e81035e0 2024-09-10T11:01:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.717+0000 7f98ee752640 1 -- 172.21.15.7:0/2213093266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e819d620 con 0x7f98e81035e0 2024-09-10T11:01:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.717+0000 7f98e57fa640 1 -- 172.21.15.7:0/2213093266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98dc0043d0 con 0x7f98e81035e0 2024-09-10T11:01:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.717+0000 7f98e57fa640 1 -- 172.21.15.7:0/2213093266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc035070 con 0x7f98e81035e0 2024-09-10T11:01:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.717+0000 7f98ee752640 1 -- 172.21.15.7:0/2213093266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e81a4f10 con 0x7f98e81035e0 2024-09-10T11:01:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.720+0000 7f98f15ca740 1 Processor -- start 2024-09-10T11:01:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.720+0000 7f98f15ca740 1 -- start start 2024-09-10T11:01:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.720+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8c9f0 0x558ccfa8cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.720+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfa9bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.720+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa9c0c0 0x558ccfa9e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.720+0000 7f98f15ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558ccf9f12f0 con 0x558ccfa9c0c0 2024-09-10T11:01:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.720+0000 7f98f15ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558ccf999270 con 0x558ccfa8c9f0 2024-09-10T11:01:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.720+0000 7f98f15ca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558ccf9ed2b0 con 0x558ccfa8d310 2024-09-10T11:01:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98e57fa640 1 -- 172.21.15.7:0/2213093266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f98dc042030 con 0x7f98e81035e0 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfa9bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98eccef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8c9f0 0x558ccfa8cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfa9bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:52214/0 (socket says 172.21.15.7:52214) 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98eccef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8c9f0 0x558ccfa8cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:41414/0 (socket says 172.21.15.7:41414) 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98e7fff640 1 -- 172.21.15.7:0/695332565 learned_addr learned my addr 172.21.15.7:0/695332565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98e77fe640 1 --2- 172.21.15.7:0/695332565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa9c0c0 0x558ccfa9e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98eccef640 1 -- 172.21.15.7:0/695332565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 msgr2=0x558ccfa9bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98eccef640 1 --2- 172.21.15.7:0/695332565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfa9bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98eccef640 1 -- 172.21.15.7:0/695332565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa9c0c0 msgr2=0x558ccfa9e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98eccef640 1 --2- 172.21.15.7:0/695332565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa9c0c0 0x558ccfa9e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98eccef640 1 -- 172.21.15.7:0/695332565 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ccf998340 con 0x558ccfa8c9f0 2024-09-10T11:01:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98e77fe640 1 --2- 172.21.15.7:0/695332565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa9c0c0 0x558ccfa9e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:01:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98eccef640 1 --2- 172.21.15.7:0/695332565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8c9f0 0x558ccfa8cdd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009d70 tx=0x7f98e000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c83550d527eda14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98ce7fc640 1 -- 172.21.15.7:0/695332565 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0004b20 con 0x558ccfa8c9f0 2024-09-10T11:01:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.721+0000 7f98e57fa640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98d00a2e20 0x7f98d00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.722+0000 7f98ce7fc640 1 -- 172.21.15.7:0/695332565 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98e0003e50 con 0x558ccfa8c9f0 2024-09-10T11:01:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.722+0000 7f98ce7fc640 1 -- 172.21.15.7:0/695332565 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0005020 con 0x558ccfa8c9f0 2024-09-10T11:01:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.722+0000 7f98e7fff640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98d00a2e20 0x7f98d00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.722+0000 7f98e57fa640 1 -- 172.21.15.7:0/2213093266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8564+0+0 (secure 0 0 0) 0x7f98dc0ef7b0 con 0x7f98e81035e0 2024-09-10T11:01:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.722+0000 7f98f15ca740 1 -- 172.21.15.7:0/695332565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8c9f0 msgr2=0x558ccfa8cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.722+0000 7f98e7fff640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98d00a2e20 0x7f98d00a52e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x558ccfa8f6c0 tx=0x7f98d8005f90 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.722+0000 7f98f15ca740 1 --2- 172.21.15.7:0/695332565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8c9f0 0x558ccfa8cdd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009d70 tx=0x7f98e000e720 comp rx=0 tx=0).stop 2024-09-10T11:01:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.723+0000 7f98f15ca740 1 -- 172.21.15.7:0/695332565 shutdown_connections 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.723+0000 7f98f15ca740 1 --2- 172.21.15.7:0/695332565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa9c0c0 0x558ccfa9e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.723+0000 7f98f15ca740 1 --2- 172.21.15.7:0/695332565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfa9bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.723+0000 7f98f15ca740 1 --2- 172.21.15.7:0/695332565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8c9f0 0x558ccfa8cdd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.723+0000 7f98f15ca740 1 -- 172.21.15.7:0/695332565 >> 172.21.15.7:0/695332565 conn(0x558ccfa89060 msgr2=0x558ccfa89440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.723+0000 7f98f15ca740 1 -- 172.21.15.7:0/695332565 shutdown_connections 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.723+0000 7f98f15ca740 1 -- 172.21.15.7:0/695332565 wait complete. 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.723+0000 7f98f15ca740 1 -- start start 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.723+0000 7f98f15ca740 1 Processor -- start 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfa98f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8d310 0x558ccfa994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa9c0c0 0x558ccfa999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98f15ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558ccfa9ed20 con 0x558ccfa8c9f0 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98f15ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558ccf97a290 con 0x558ccfa8d310 2024-09-10T11:01:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98f15ca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558ccfa9eea0 con 0x558ccfa9c0c0 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98f15ca740 10 client.0 _ll_register_callbacks cb 0x558ccf9ab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfa98f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfa98f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58526/0 (socket says 172.21.15.7:58526) 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98e7fff640 1 -- 172.21.15.7:0/19490530 learned_addr learned my addr 172.21.15.7:0/19490530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98e77fe640 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8d310 0x558ccfa994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98eccef640 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa9c0c0 0x558ccfa999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98e7fff640 1 -- 172.21.15.7:0/19490530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa9c0c0 msgr2=0x558ccfa999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98e7fff640 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa9c0c0 0x558ccfa999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98e7fff640 1 -- 172.21.15.7:0/19490530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8d310 msgr2=0x558ccfa994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98e7fff640 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8d310 0x558ccfa994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98e7fff640 1 -- 172.21.15.7:0/19490530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ccf704210 con 0x558ccfa8c9f0 2024-09-10T11:01:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.724+0000 7f98eccef640 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa9c0c0 0x558ccfa999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.725+0000 7f98e7fff640 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfa98f70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f98d801ea00 tx=0x7f98d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a104505bc3db5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.725+0000 7f98c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.725+0000 7f98c77fe640 1 -- 172.21.15.7:0/19490530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d802e060 con 0x558ccfa8c9f0 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.725+0000 7f98c77fe640 1 -- 172.21.15.7:0/19490530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8564+0+0 (secure 0 0 0) 0x7f98d8018030 con 0x558ccfa8c9f0 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.725+0000 7f98f15ca740 1 -- 172.21.15.7:0/19490530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98e0003e50 con 0x558ccfa8c9f0 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.725+0000 7f98c77fe640 1 -- 172.21.15.7:0/19490530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f98d8009b50 con 0x558ccfa8c9f0 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.725+0000 7f98f15ca740 10 client.16524 fetch_fsmap learned FSMap version 51 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.725+0000 7f98f15ca740 1 -- 172.21.15.7:0/19490530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558ccf7d0620 con 0x558ccfa8c9f0 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.725+0000 7f98c77fe640 1 -- 172.21.15.7:0/19490530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98d8029900 con 0x558ccfa8c9f0 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.726+0000 7f98c77fe640 1 -- 172.21.15.7:0/19490530 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 51) ==== 8362+0+0 (secure 0 0 0) 0x7f98d8031830 con 0x558ccfa8c9f0 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.726+0000 7f98f15ca740 10 client.16524 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.726+0000 7f98f15ca740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:01:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.726+0000 7f98f15ca740 20 client.16524 populate_metadata read hostname 'smithi007' 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.726+0000 7f98f15ca740 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558ccfaac0c0 0x558ccfaae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.726+0000 7f98f15ca740 4 client.16524 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.726+0000 7f98f15ca740 1 -- 172.21.15.7:0/19490530 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f98e0003e50 con 0x558ccfaac0c0 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.726+0000 7f98e77fe640 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558ccfaac0c0 0x558ccfaae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.726+0000 7f98e77fe640 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558ccfaac0c0 0x558ccfaae4d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0aee091efa14cd server_cookie=7a3cfb390a308f1b in_seq=0 out_seq=0 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.726+0000 7f98c77fe640 10 client.16524 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98c77fe640 1 -- 172.21.15.7:0/19490530 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f98e0003e50 con 0x558ccfaac0c0 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98c77fe640 10 client.16524 handle_command_reply: tid=0 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 client.16524 shutdown 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 -- 172.21.15.7:0/19490530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 msgr2=0x558ccfa98f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfa98f70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f98d801ea00 tx=0x7f98d8004730 comp rx=0 tx=0).stop 2024-09-10T11:01:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 -- 172.21.15.7:0/19490530 shutdown_connections 2024-09-10T11:01:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558ccfaac0c0 0x558ccfaae4d0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa9c0c0 0x558ccfa999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa8d310 0x558ccfa994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 --2- 172.21.15.7:0/19490530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfa98f70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 -- 172.21.15.7:0/19490530 >> 172.21.15.7:0/19490530 conn(0x558ccfa89060 msgr2=0x558ccfa8ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98c77fe640 0 client.16524 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 -- 172.21.15.7:0/19490530 shutdown_connections 2024-09-10T11:01:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 1 -- 172.21.15.7:0/19490530 wait complete. 2024-09-10T11:01:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.728+0000 7f98f15ca740 20 client.16524 trim_cache size 0 max 16384 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.749+0000 7f98f15ca740 1 Processor -- start 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.749+0000 7f98f15ca740 1 -- start start 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfab3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfab4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa9c0c0 0x558ccfa961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98f15ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81ae000 con 0x558ccfa8c9f0 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98f15ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81ab870 con 0x558ccfa9c0c0 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98f15ca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81a96e0 con 0x558ccfa8d310 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfab3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfab3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58532/0 (socket says 172.21.15.7:58532) 2024-09-10T11:01:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98e7fff640 1 -- 172.21.15.7:0/1480027096 learned_addr learned my addr 172.21.15.7:0/1480027096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98e77fe640 1 --2- 172.21.15.7:0/1480027096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfab4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98eccef640 1 --2- 172.21.15.7:0/1480027096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa9c0c0 0x558ccfa961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98e7fff640 1 -- 172.21.15.7:0/1480027096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 msgr2=0x558ccfab4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98e7fff640 1 --2- 172.21.15.7:0/1480027096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfab4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98e7fff640 1 -- 172.21.15.7:0/1480027096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa9c0c0 msgr2=0x558ccfa961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98e7fff640 1 --2- 172.21.15.7:0/1480027096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa9c0c0 0x558ccfa961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98e7fff640 1 -- 172.21.15.7:0/1480027096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ccfa88e90 con 0x558ccfa8c9f0 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.750+0000 7f98e7fff640 1 --2- 172.21.15.7:0/1480027096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfab3de0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f98d80092e0 tx=0x7f98d8009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8483cb2ebf13da69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98cdffb640 1 -- 172.21.15.7:0/1480027096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d80315f0 con 0x558ccfa8c9f0 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98cdffb640 1 -- 172.21.15.7:0/1480027096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98d801c6c0 con 0x558ccfa8c9f0 2024-09-10T11:01:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98cdffb640 1 -- 172.21.15.7:0/1480027096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8032720 con 0x558ccfa8c9f0 2024-09-10T11:01:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98f15ca740 1 -- 172.21.15.7:0/1480027096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 msgr2=0x558ccfab3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98f15ca740 1 --2- 172.21.15.7:0/1480027096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfab3de0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f98d80092e0 tx=0x7f98d8009cf0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98f15ca740 1 -- 172.21.15.7:0/1480027096 shutdown_connections 2024-09-10T11:01:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98f15ca740 1 --2- 172.21.15.7:0/1480027096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa9c0c0 0x558ccfa961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98f15ca740 1 --2- 172.21.15.7:0/1480027096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfab4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98f15ca740 1 --2- 172.21.15.7:0/1480027096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfab3de0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98f15ca740 1 -- 172.21.15.7:0/1480027096 >> 172.21.15.7:0/1480027096 conn(0x558ccfa89060 msgr2=0x558ccfa8ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98f15ca740 1 -- 172.21.15.7:0/1480027096 shutdown_connections 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.751+0000 7f98f15ca740 1 -- 172.21.15.7:0/1480027096 wait complete. 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98f15ca740 1 -- start start 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98f15ca740 1 Processor -- start 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfaa0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfaa0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98f15ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa9c0c0 0x558ccfb15db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98f15ca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81a9d30 con 0x558ccfa8c9f0 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98f15ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81ab1a0 con 0x558ccfa9c0c0 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98f15ca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f98e81aa400 con 0x558ccfa8d310 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98f15ca740 10 client.0 _ll_register_callbacks cb 0x7f98d801e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfaa0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98e7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfaa0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58534/0 (socket says 172.21.15.7:58534) 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98e7fff640 1 -- 172.21.15.7:0/2473868667 learned_addr learned my addr 172.21.15.7:0/2473868667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.752+0000 7f98eccef640 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa9c0c0 0x558ccfb15db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98e7fff640 1 -- 172.21.15.7:0/2473868667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 msgr2=0x558ccfaa0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98e7fff640 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfaa0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98e7fff640 1 -- 172.21.15.7:0/2473868667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa9c0c0 msgr2=0x558ccfb15db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98e7fff640 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa9c0c0 0x558ccfb15db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98e7fff640 1 -- 172.21.15.7:0/2473868667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ccfb163a0 con 0x558ccfa8c9f0 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98e7fff640 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfaa0630 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f98d8019040 tx=0x7f98d8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46980841b42d3671 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98c77fe640 1 -- 172.21.15.7:0/2473868667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8031ab0 con 0x558ccfa8c9f0 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98c77fe640 1 -- 172.21.15.7:0/2473868667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8564+0+0 (secure 0 0 0) 0x7f98d8005070 con 0x558ccfa8c9f0 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98f15ca740 1 -- 172.21.15.7:0/2473868667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d801c6c0 con 0x558ccfa8c9f0 2024-09-10T11:01:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98c77fe640 1 -- 172.21.15.7:0/2473868667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f98d801ce70 con 0x558ccfa8c9f0 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98f15ca740 10 client.16530 fetch_fsmap learned FSMap version 51 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.753+0000 7f98f15ca740 1 -- 172.21.15.7:0/2473868667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558ccfb17670 con 0x558ccfa8c9f0 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98c77fe640 1 -- 172.21.15.7:0/2473868667 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98d801c6c0 con 0x558ccfa8c9f0 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98c77fe640 1 -- 172.21.15.7:0/2473868667 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 51) ==== 8362+0+0 (secure 0 0 0) 0x7f98d80464c0 con 0x558ccfa8c9f0 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98f15ca740 10 client.16530 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98f15ca740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98f15ca740 20 client.16530 populate_metadata read hostname 'smithi007' 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98f15ca740 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558ccfb17d30 0x558ccfb1a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98f15ca740 4 client.16530 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98f15ca740 1 -- 172.21.15.7:0/2473868667 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ccf9fea20 con 0x558ccfb17d30 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98e77fe640 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558ccfb17d30 0x558ccfb1a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98c77fe640 10 client.16530 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:40.754+0000 7f98e77fe640 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558ccfb17d30 0x558ccfb1a150 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8949b004e461c8d0 server_cookie=da6294169249c280 in_seq=0 out_seq=0 2024-09-10T11:01:40.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 116936 38.92 MB/sec execute 40 sec latency 117.857 ms 2024-09-10T11:01:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.148+0000 7f98c77fe640 1 -- 172.21.15.7:0/2473868667 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ccf9fea20 con 0x558ccfb17d30 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.148+0000 7f98c77fe640 10 client.16530 handle_command_reply: tid=0 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.148+0000 7f98f15ca740 1 client.16530 shutdown 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 1 -- 172.21.15.7:0/2473868667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 msgr2=0x558ccfaa0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfaa0630 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f98d8019040 tx=0x7f98d8005970 comp rx=0 tx=0).stop 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 1 -- 172.21.15.7:0/2473868667 shutdown_connections 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x558ccfb17d30 0x558ccfb1a150 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98c77fe640 0 client.16530 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ccfa9c0c0 0x558ccfb15db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ccfa8d310 0x558ccfaa0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 1 --2- 172.21.15.7:0/2473868667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558ccfa8c9f0 0x558ccfaa0630 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 1 -- 172.21.15.7:0/2473868667 >> 172.21.15.7:0/2473868667 conn(0x558ccfa89060 msgr2=0x558ccfa879a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 1 -- 172.21.15.7:0/2473868667 shutdown_connections 2024-09-10T11:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 1 -- 172.21.15.7:0/2473868667 wait complete. 2024-09-10T11:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.149+0000 7f98f15ca740 20 client.16530 trim_cache size 0 max 16384 2024-09-10T11:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T11:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T11:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T11:01:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T11:01:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.150+0000 7f98ee752640 1 -- 172.21.15.7:0/2213093266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98d00a2e20 msgr2=0x7f98d00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.150+0000 7f98ee752640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98d00a2e20 0x7f98d00a52e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x558ccfa8f6c0 tx=0x7f98d8005f90 comp rx=0 tx=0).stop 2024-09-10T11:01:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.150+0000 7f98ee752640 1 -- 172.21.15.7:0/2213093266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98e81035e0 msgr2=0x7f98e81988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.150+0000 7f98ee752640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98e81035e0 0x7f98e81988e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f98dc006010 tx=0x7f98dc031b20 comp rx=0 tx=0).stop 2024-09-10T11:01:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.150+0000 7f98ee752640 1 -- 172.21.15.7:0/2213093266 shutdown_connections 2024-09-10T11:01:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.150+0000 7f98ee752640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e810c110 0x7f98e819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.150+0000 7f98ee752640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98d00a2e20 0x7f98d00a52e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.150+0000 7f98ee752640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98e81035e0 0x7f98e81988e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.150+0000 7f98ee752640 1 --2- 172.21.15.7:0/2213093266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98e8102cc0 0x7f98e81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.150+0000 7f98ee752640 1 -- 172.21.15.7:0/2213093266 >> 172.21.15.7:0/2213093266 conn(0x7f98e80fc6b0 msgr2=0x7f98e80fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.151+0000 7f98ee752640 1 -- 172.21.15.7:0/2213093266 shutdown_connections 2024-09-10T11:01:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:41.151+0000 7f98ee752640 1 -- 172.21.15.7:0/2213093266 wait complete. 2024-09-10T11:01:41.887 INFO:tasks.workunit.client.0.smithi007.stdout: 10 117278 38.46 MB/sec execute 41 sec latency 386.577 ms 2024-09-10T11:01:42.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:41 smithi103 ceph-mon[35160]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-10T11:01:42.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:41 smithi007 ceph-mon[30873]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-10T11:01:42.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:41 smithi133 ceph-mon[35282]: pgmap v500: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-10T11:01:42.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 118147 38.59 MB/sec execute 42 sec latency 68.726 ms 2024-09-10T11:01:43.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:42 smithi103 ceph-mon[35160]: pgmap v501: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 598 op/s 2024-09-10T11:01:43.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:42 smithi007 ceph-mon[30873]: pgmap v501: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 598 op/s 2024-09-10T11:01:43.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:42 smithi133 ceph-mon[35282]: pgmap v501: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 598 op/s 2024-09-10T11:01:43.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 119043 38.91 MB/sec execute 43 sec latency 85.865 ms 2024-09-10T11:01:44.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 119838 39.05 MB/sec execute 44 sec latency 50.575 ms 2024-09-10T11:01:45.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:45 smithi103 ceph-mon[35160]: pgmap v502: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 598 op/s 2024-09-10T11:01:45.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:45 smithi007 ceph-mon[30873]: pgmap v502: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 598 op/s 2024-09-10T11:01:45.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:45 smithi133 ceph-mon[35282]: pgmap v502: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 598 op/s 2024-09-10T11:01:45.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 120792 39.21 MB/sec execute 45 sec latency 56.575 ms 2024-09-10T11:01:46.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 121538 39.37 MB/sec execute 46 sec latency 273.012 ms 2024-09-10T11:01:47.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:47 smithi103 ceph-mon[35160]: pgmap v503: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-09-10T11:01:47.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:47 smithi007 ceph-mon[30873]: pgmap v503: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-09-10T11:01:47.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:47 smithi133 ceph-mon[35282]: pgmap v503: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 16 MiB/s wr, 1.06k op/s 2024-09-10T11:01:47.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 122294 39.36 MB/sec execute 47 sec latency 99.240 ms 2024-09-10T11:01:48.889 INFO:tasks.workunit.client.0.smithi007.stdout: 10 123079 39.44 MB/sec execute 48 sec latency 39.458 ms 2024-09-10T11:01:49.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:49 smithi103 ceph-mon[35160]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-10T11:01:49.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:49 smithi007 ceph-mon[30873]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-10T11:01:49.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:49 smithi133 ceph-mon[35282]: pgmap v504: 97 pgs: 97 active+clean; 1.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 742 op/s 2024-09-10T11:01:49.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 124011 39.54 MB/sec execute 49 sec latency 48.657 ms 2024-09-10T11:01:50.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 124921 39.88 MB/sec execute 50 sec latency 56.690 ms 2024-09-10T11:01:51.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:51 smithi103 ceph-mon[35160]: pgmap v505: 97 pgs: 97 active+clean; 1.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-10T11:01:51.436 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:51 smithi007 ceph-mon[30873]: pgmap v505: 97 pgs: 97 active+clean; 1.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-10T11:01:51.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:51 smithi133 ceph-mon[35282]: pgmap v505: 97 pgs: 97 active+clean; 1.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-10T11:01:51.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 125591 39.76 MB/sec execute 51 sec latency 182.724 ms 2024-09-10T11:01:52.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 126285 39.62 MB/sec execute 52 sec latency 256.339 ms 2024-09-10T11:01:53.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:53 smithi103 ceph-mon[35160]: pgmap v506: 97 pgs: 97 active+clean; 1.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 792 op/s 2024-09-10T11:01:53.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:53 smithi007 ceph-mon[30873]: pgmap v506: 97 pgs: 97 active+clean; 1.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 792 op/s 2024-09-10T11:01:53.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:53 smithi133 ceph-mon[35282]: pgmap v506: 97 pgs: 97 active+clean; 1.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 792 op/s 2024-09-10T11:01:53.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 127168 39.98 MB/sec execute 53 sec latency 94.158 ms 2024-09-10T11:01:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.267+0000 7f0aaaaae640 1 -- 172.21.15.7:0/526901906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0aa4101ac0 msgr2=0x7f0aa4101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.267+0000 7f0aaaaae640 1 --2- 172.21.15.7:0/526901906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0aa4101ac0 0x7f0aa4101f20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009930 tx=0x7f0a9802a420 comp rx=0 tx=0).stop 2024-09-10T11:01:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.267+0000 7f0aaaaae640 1 -- 172.21.15.7:0/526901906 shutdown_connections 2024-09-10T11:01:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.267+0000 7f0aaaaae640 1 --2- 172.21.15.7:0/526901906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aa4102460 0x7f0aa410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.267+0000 7f0aaaaae640 1 --2- 172.21.15.7:0/526901906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0aa4101ac0 0x7f0aa4101f20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.267+0000 7f0aaaaae640 1 --2- 172.21.15.7:0/526901906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa4107ad0 0x7f0aa4107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.267+0000 7f0aaaaae640 1 -- 172.21.15.7:0/526901906 >> 172.21.15.7:0/526901906 conn(0x7f0aa40fd7e0 msgr2=0x7f0aa40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.267+0000 7f0aaaaae640 1 -- 172.21.15.7:0/526901906 shutdown_connections 2024-09-10T11:01:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.267+0000 7f0aaaaae640 1 -- 172.21.15.7:0/526901906 wait complete. 2024-09-10T11:01:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aaaaae640 1 Processor -- start 2024-09-10T11:01:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aaaaae640 1 -- start start 2024-09-10T11:01:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aaaaae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0aa4101ac0 0x7f0aa419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aaaaae640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa4102460 0x7f0aa419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aaaaae640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aa4107ad0 0x7f0aa419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aaaaae640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4111f60 con 0x7f0aa4101ac0 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aaaaae640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4111de0 con 0x7f0aa4107ad0 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aaaaae640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa41120e0 con 0x7f0aa4102460 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aa884a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0aa4101ac0 0x7f0aa419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aa884a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0aa4101ac0 0x7f0aa419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42044/0 (socket says 172.21.15.7:42044) 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.268+0000 7f0aa884a640 1 -- 172.21.15.7:0/1391267076 learned_addr learned my addr 172.21.15.7:0/1391267076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aa904b640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aa4107ad0 0x7f0aa419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aa3fff640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa4102460 0x7f0aa419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aa884a640 1 -- 172.21.15.7:0/1391267076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa4102460 msgr2=0x7f0aa419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aa884a640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa4102460 0x7f0aa419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aa884a640 1 -- 172.21.15.7:0/1391267076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aa4107ad0 msgr2=0x7f0aa419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aa884a640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aa4107ad0 0x7f0aa419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aa884a640 1 -- 172.21.15.7:0/1391267076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa419f5a0 con 0x7f0aa4101ac0 2024-09-10T11:01:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aa904b640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aa4107ad0 0x7f0aa419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:01:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aa884a640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0aa4101ac0 0x7f0aa419a5b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400e9f0 tx=0x7f0a9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aa1ffb640 1 -- 172.21.15.7:0/1391267076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400cd80 con 0x7f0aa4101ac0 2024-09-10T11:01:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.269+0000 7f0aaaaae640 1 -- 172.21.15.7:0/1391267076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa419f890 con 0x7f0aa4101ac0 2024-09-10T11:01:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.270+0000 7f0aaaaae640 1 -- 172.21.15.7:0/1391267076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa41a7120 con 0x7f0aa4101ac0 2024-09-10T11:01:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.270+0000 7f0aa1ffb640 1 -- 172.21.15.7:0/1391267076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a94005500 con 0x7f0aa4101ac0 2024-09-10T11:01:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.270+0000 7f0aa1ffb640 1 -- 172.21.15.7:0/1391267076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94010690 con 0x7f0aa4101ac0 2024-09-10T11:01:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.273+0000 7f0aad848740 1 Processor -- start 2024-09-10T11:01:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.273+0000 7f0aad848740 1 -- start start 2024-09-10T11:01:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aad848740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 0x55fde5582ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aad848740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde558d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aad848740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde558dcb0 0x55fde55900a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aad848740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fde54e72f0 con 0x55fde5582af0 2024-09-10T11:01:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aad848740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fde548f270 con 0x55fde558dcb0 2024-09-10T11:01:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aad848740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fde54e32b0 con 0x55fde5583410 2024-09-10T11:01:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aa3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde558dcb0 0x55fde55900a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aa3fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde558dcb0 0x55fde55900a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:37774/0 (socket says 172.21.15.7:37774) 2024-09-10T11:01:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aa3fff640 1 -- 172.21.15.7:0/944515532 learned_addr learned my addr 172.21.15.7:0/944515532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aa904b640 1 --2- 172.21.15.7:0/944515532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 0x55fde5582ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aa3fff640 1 -- 172.21.15.7:0/944515532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 msgr2=0x55fde558d770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T11:01:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aa3fff640 1 --2- 172.21.15.7:0/944515532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde558d770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aa3fff640 1 -- 172.21.15.7:0/944515532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 msgr2=0x55fde5582ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aa3fff640 1 --2- 172.21.15.7:0/944515532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 0x55fde5582ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.274+0000 7f0aa3fff640 1 -- 172.21.15.7:0/944515532 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde548e340 con 0x55fde558dcb0 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.275+0000 7f0aa1ffb640 1 -- 172.21.15.7:0/1391267076 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0a940046c0 con 0x7f0aa4101ac0 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.275+0000 7f0aa3fff640 1 --2- 172.21.15.7:0/944515532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde558dcb0 0x55fde55900a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009930 tx=0x7f0a98005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6759c3accf3d058 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.275+0000 7f0a8affd640 1 -- 172.21.15.7:0/944515532 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98035040 con 0x55fde558dcb0 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.275+0000 7f0a8affd640 1 -- 172.21.15.7:0/944515532 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a98005520 con 0x55fde558dcb0 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.275+0000 7f0a8affd640 1 -- 172.21.15.7:0/944515532 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a980035f0 con 0x55fde558dcb0 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aa1ffb640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0a8c0a2da0 0x7f0a8c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aad848740 1 -- 172.21.15.7:0/944515532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde558dcb0 msgr2=0x55fde55900a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aad848740 1 --2- 172.21.15.7:0/944515532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde558dcb0 0x55fde55900a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009930 tx=0x7f0a98005de0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aad848740 1 -- 172.21.15.7:0/944515532 shutdown_connections 2024-09-10T11:01:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aad848740 1 --2- 172.21.15.7:0/944515532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde558dcb0 0x55fde55900a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aad848740 1 --2- 172.21.15.7:0/944515532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde558d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aad848740 1 --2- 172.21.15.7:0/944515532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 0x55fde5582ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aad848740 1 -- 172.21.15.7:0/944515532 >> 172.21.15.7:0/944515532 conn(0x55fde557f060 msgr2=0x55fde557f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aa904b640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0a8c0a2da0 0x7f0a8c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aa1ffb640 1 -- 172.21.15.7:0/1391267076 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8564+0+0 (secure 0 0 0) 0x7f0a940cf790 con 0x7f0aa4101ac0 2024-09-10T11:01:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.276+0000 7f0aad848740 1 -- 172.21.15.7:0/944515532 shutdown_connections 2024-09-10T11:01:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.277+0000 7f0aad848740 1 -- 172.21.15.7:0/944515532 wait complete. 2024-09-10T11:01:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.277+0000 7f0aad848740 1 -- start start 2024-09-10T11:01:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.277+0000 7f0aad848740 1 Processor -- start 2024-09-10T11:01:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa904b640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0a8c0a2da0 0x7f0a8c0a5260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009fd0 tx=0x7f0a9c009500 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aad848740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aad848740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde558b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aad848740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 0x55fde558b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aad848740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fde5590880 con 0x55fde558dcb0 2024-09-10T11:01:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aad848740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55fde5470290 con 0x55fde5582af0 2024-09-10T11:01:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aad848740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fde5590a00 con 0x55fde5583410 2024-09-10T11:01:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aad848740 10 client.0 _ll_register_callbacks cb 0x55fde54a1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa884a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa884a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:37780/0 (socket says 172.21.15.7:37780) 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa884a640 1 -- 172.21.15.7:0/308718121 learned_addr learned my addr 172.21.15.7:0/308718121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa904b640 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 0x55fde558b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa3fff640 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde558b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa884a640 1 -- 172.21.15.7:0/308718121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 msgr2=0x55fde558b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa884a640 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde558b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa884a640 1 -- 172.21.15.7:0/308718121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 msgr2=0x55fde558b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa884a640 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 0x55fde558b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.278+0000 7f0aa884a640 1 -- 172.21.15.7:0/308718121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde51fa210 con 0x55fde5582af0 2024-09-10T11:01:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.279+0000 7f0aa884a640 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558ab80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a940cceb0 tx=0x7f0a9409c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fa03e40242f6f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.279+0000 7f0a83fff640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.279+0000 7f0a83fff640 1 -- 172.21.15.7:0/308718121 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a940a4890 con 0x55fde5582af0 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.279+0000 7f0a83fff640 1 -- 172.21.15.7:0/308718121 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8564+0+0 (secure 0 0 0) 0x7f0a940a4400 con 0x55fde5582af0 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.279+0000 7f0aad848740 1 -- 172.21.15.7:0/308718121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a98005520 con 0x55fde5582af0 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.279+0000 7f0a83fff640 1 -- 172.21.15.7:0/308718121 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0a940a4640 con 0x55fde5582af0 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.279+0000 7f0aad848740 10 client.26147 fetch_fsmap learned FSMap version 51 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.279+0000 7f0aad848740 1 -- 172.21.15.7:0/308718121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55fde52c6620 con 0x55fde5582af0 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.280+0000 7f0a83fff640 1 -- 172.21.15.7:0/308718121 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a9409b070 con 0x55fde5582af0 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.280+0000 7f0a83fff640 1 -- 172.21.15.7:0/308718121 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 51) ==== 8362+0+0 (secure 0 0 0) 0x7f0a940ba2f0 con 0x55fde5582af0 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.280+0000 7f0aad848740 10 client.26147 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.280+0000 7f0aad848740 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.281+0000 7f0aad848740 20 client.26147 populate_metadata read hostname 'smithi007' 2024-09-10T11:01:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.281+0000 7f0aad848740 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fde559dce0 0x55fde55a0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.281+0000 7f0aad848740 4 client.26147 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:01:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.281+0000 7f0aad848740 1 -- 172.21.15.7:0/308718121 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0a98005520 con 0x55fde559dce0 2024-09-10T11:01:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.281+0000 7f0aa3fff640 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fde559dce0 0x55fde55a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.281+0000 7f0aa3fff640 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fde559dce0 0x55fde55a0100 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350fa7e859733c6f server_cookie=6dbdd4385c79faa5 in_seq=0 out_seq=0 2024-09-10T11:01:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.281+0000 7f0a83fff640 10 client.26147 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.282+0000 7f0a83fff640 1 -- 172.21.15.7:0/308718121 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0a98005520 con 0x55fde559dce0 2024-09-10T11:01:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.282+0000 7f0a83fff640 10 client.26147 handle_command_reply: tid=0 2024-09-10T11:01:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.282+0000 7f0aad848740 1 client.26147 shutdown 2024-09-10T11:01:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 1 -- 172.21.15.7:0/308718121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 msgr2=0x55fde558ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558ab80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a940cceb0 tx=0x7f0a9409c3e0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 1 -- 172.21.15.7:0/308718121 shutdown_connections 2024-09-10T11:01:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fde559dce0 0x55fde55a0100 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 0x55fde558b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde558b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 1 --2- 172.21.15.7:0/308718121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558ab80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0a83fff640 0 client.26147 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 1 -- 172.21.15.7:0/308718121 >> 172.21.15.7:0/308718121 conn(0x55fde557f060 msgr2=0x55fde5584dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 1 -- 172.21.15.7:0/308718121 shutdown_connections 2024-09-10T11:01:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 1 -- 172.21.15.7:0/308718121 wait complete. 2024-09-10T11:01:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.283+0000 7f0aad848740 20 client.26147 trim_cache size 0 max 16384 2024-09-10T11:01:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aad848740 1 Processor -- start 2024-09-10T11:01:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aad848740 1 -- start start 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aad848740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aad848740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde55a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aad848740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 0x55fde55a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aad848740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90011120 con 0x55fde558dcb0 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aad848740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9000e990 con 0x55fde5582af0 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aad848740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9000c800 con 0x55fde5583410 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde55a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa884a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa904b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 0x55fde55a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde55a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:60672/0 (socket says 172.21.15.7:60672) 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa884a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:37792/0 (socket says 172.21.15.7:37792) 2024-09-10T11:01:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa3fff640 1 -- 172.21.15.7:0/4147818598 learned_addr learned my addr 172.21.15.7:0/4147818598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa904b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 0x55fde55a5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42084/0 (socket says 172.21.15.7:42084) 2024-09-10T11:01:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa904b640 1 -- 172.21.15.7:0/4147818598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 msgr2=0x55fde55a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa904b640 1 --2- 172.21.15.7:0/4147818598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde55a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa904b640 1 -- 172.21.15.7:0/4147818598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 msgr2=0x55fde558b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa904b640 1 --2- 172.21.15.7:0/4147818598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.302+0000 7f0aa904b640 1 -- 172.21.15.7:0/4147818598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fde557ee90 con 0x55fde558dcb0 2024-09-10T11:01:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.303+0000 7f0aa904b640 1 --2- 172.21.15.7:0/4147818598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 0x55fde55a5f30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c013010 tx=0x7f0a9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7d5c896e97cb21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.303+0000 7f0a8a7fc640 1 -- 172.21.15.7:0/4147818598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c004b20 con 0x55fde558dcb0 2024-09-10T11:01:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.303+0000 7f0a8a7fc640 1 -- 172.21.15.7:0/4147818598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a9c004cc0 con 0x55fde558dcb0 2024-09-10T11:01:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.303+0000 7f0a8a7fc640 1 -- 172.21.15.7:0/4147818598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c005020 con 0x55fde558dcb0 2024-09-10T11:01:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.304+0000 7f0aad848740 1 -- 172.21.15.7:0/4147818598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 msgr2=0x55fde55a5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.304+0000 7f0aad848740 1 --2- 172.21.15.7:0/4147818598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 0x55fde55a5f30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c013010 tx=0x7f0a9c017000 comp rx=0 tx=0).stop 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.304+0000 7f0aad848740 1 -- 172.21.15.7:0/4147818598 shutdown_connections 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.304+0000 7f0aad848740 1 --2- 172.21.15.7:0/4147818598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde558dcb0 0x55fde55a5f30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.304+0000 7f0aad848740 1 --2- 172.21.15.7:0/4147818598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde5583410 0x55fde55a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.304+0000 7f0aad848740 1 --2- 172.21.15.7:0/4147818598 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5582af0 0x55fde558b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.304+0000 7f0aad848740 1 -- 172.21.15.7:0/4147818598 >> 172.21.15.7:0/4147818598 conn(0x55fde557f060 msgr2=0x55fde558cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.304+0000 7f0aad848740 1 -- 172.21.15.7:0/4147818598 shutdown_connections 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.304+0000 7f0aad848740 1 -- 172.21.15.7:0/4147818598 wait complete. 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.304+0000 7f0aad848740 1 -- start start 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aad848740 1 Processor -- start 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aad848740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 0x55fde5587ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aad848740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5583410 0x55fde55880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aad848740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde558dcb0 0x55fde5588620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aad848740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9000ce50 con 0x55fde5582af0 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aad848740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9000e2c0 con 0x55fde5583410 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aad848740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9000d520 con 0x55fde558dcb0 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aa884a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 0x55fde5587ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aad848740 10 client.0 _ll_register_callbacks cb 0x7f0a94009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aa884a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 0x55fde5587ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42094/0 (socket says 172.21.15.7:42094) 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aa884a640 1 -- 172.21.15.7:0/270410537 learned_addr learned my addr 172.21.15.7:0/270410537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aa3fff640 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5583410 0x55fde55880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.305+0000 7f0aa904b640 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde558dcb0 0x55fde5588620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0aa884a640 1 -- 172.21.15.7:0/270410537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde558dcb0 msgr2=0x55fde5588620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0aa884a640 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde558dcb0 0x55fde5588620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0aa884a640 1 -- 172.21.15.7:0/270410537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5583410 msgr2=0x55fde55880e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:01:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0aa884a640 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5583410 0x55fde55880e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0aa884a640 1 -- 172.21.15.7:0/270410537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fde557a2d0 con 0x55fde5582af0 2024-09-10T11:01:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0aa884a640 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 0x55fde5587ba0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0a9409b920 tx=0x7f0a9409b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472cde0a8898de24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:01:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0a83fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:01:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0a83fff640 1 -- 172.21.15.7:0/270410537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a940a4890 con 0x55fde5582af0 2024-09-10T11:01:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0a83fff640 1 -- 172.21.15.7:0/270410537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8564+0+0 (secure 0 0 0) 0x7f0a940a4ae0 con 0x55fde5582af0 2024-09-10T11:01:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0aad848740 1 -- 172.21.15.7:0/270410537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a9c004cc0 con 0x55fde5582af0 2024-09-10T11:01:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.306+0000 7f0a83fff640 1 -- 172.21.15.7:0/270410537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0a940a4ce0 con 0x55fde5582af0 2024-09-10T11:01:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.307+0000 7f0aad848740 10 client.16548 fetch_fsmap learned FSMap version 51 2024-09-10T11:01:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.307+0000 7f0aad848740 1 -- 172.21.15.7:0/270410537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55fde5608ae0 con 0x55fde5582af0 2024-09-10T11:01:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.307+0000 7f0a83fff640 1 -- 172.21.15.7:0/270410537 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a9c004cc0 con 0x55fde5582af0 2024-09-10T11:01:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.308+0000 7f0a83fff640 1 -- 172.21.15.7:0/270410537 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 51) ==== 8362+0+0 (secure 0 0 0) 0x7f0a940a1160 con 0x55fde5582af0 2024-09-10T11:01:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.308+0000 7f0aad848740 10 client.16548 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T11:01:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.308+0000 7f0aad848740 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:01:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.308+0000 7f0aad848740 20 client.16548 populate_metadata read hostname 'smithi007' 2024-09-10T11:01:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.308+0000 7f0aad848740 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fde56092c0 0x55fde560b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:01:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.308+0000 7f0aad848740 4 client.16548 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T11:01:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.308+0000 7f0aad848740 1 -- 172.21.15.7:0/270410537 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fde54f4a20 con 0x55fde56092c0 2024-09-10T11:01:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.308+0000 7f0aa3fff640 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fde56092c0 0x55fde560b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:01:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.309+0000 7f0a83fff640 10 client.16548 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.309+0000 7f0aa3fff640 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fde56092c0 0x55fde560b6e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1783bd05b5cf0f82 server_cookie=32de6bb485693727 in_seq=0 out_seq=0 2024-09-10T11:01:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.709+0000 7f0a83fff640 1 -- 172.21.15.7:0/270410537 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fde54f4a20 con 0x55fde56092c0 2024-09-10T11:01:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.709+0000 7f0a83fff640 10 client.16548 handle_command_reply: tid=0 2024-09-10T11:01:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.709+0000 7f0aad848740 1 client.16548 shutdown 2024-09-10T11:01:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.709+0000 7f0aad848740 1 -- 172.21.15.7:0/270410537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 msgr2=0x55fde5587ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.709+0000 7f0aad848740 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 0x55fde5587ba0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0a9409b920 tx=0x7f0a9409b5a0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.709+0000 7f0aad848740 1 -- 172.21.15.7:0/270410537 shutdown_connections 2024-09-10T11:01:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.709+0000 7f0aad848740 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55fde56092c0 0x55fde560b6e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.709+0000 7f0aad848740 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fde558dcb0 0x55fde5588620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.709+0000 7f0a83fff640 0 client.16548 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:01:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.710+0000 7f0aad848740 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55fde5583410 0x55fde55880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.710+0000 7f0aad848740 1 --2- 172.21.15.7:0/270410537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fde5582af0 0x55fde5587ba0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.710+0000 7f0aad848740 1 -- 172.21.15.7:0/270410537 >> 172.21.15.7:0/270410537 conn(0x55fde557f060 msgr2=0x55fde5584550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.710+0000 7f0aad848740 1 -- 172.21.15.7:0/270410537 shutdown_connections 2024-09-10T11:01:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.710+0000 7f0aad848740 1 -- 172.21.15.7:0/270410537 wait complete. 2024-09-10T11:01:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.710+0000 7f0aad848740 20 client.16548 trim_cache size 0 max 16384 2024-09-10T11:01:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T11:01:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T11:01:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T11:01:54.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T11:01:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 -- 172.21.15.7:0/1391267076 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0a8c0a2da0 msgr2=0x7f0a8c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0a8c0a2da0 0x7f0a8c0a5260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009fd0 tx=0x7f0a9c009500 comp rx=0 tx=0).stop 2024-09-10T11:01:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 -- 172.21.15.7:0/1391267076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0aa4101ac0 msgr2=0x7f0aa419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:01:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0aa4101ac0 0x7f0aa419a5b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400e9f0 tx=0x7f0a9400edf0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 -- 172.21.15.7:0/1391267076 shutdown_connections 2024-09-10T11:01:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0aa4107ad0 0x7f0aa419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0aa4102460 0x7f0aa419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f0a8c0a2da0 0x7f0a8c0a5260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 --2- 172.21.15.7:0/1391267076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0aa4101ac0 0x7f0aa419a5b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:01:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 -- 172.21.15.7:0/1391267076 >> 172.21.15.7:0/1391267076 conn(0x7f0aa40fd7e0 msgr2=0x7f0aa40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:01:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.711+0000 7f0aaaaae640 1 -- 172.21.15.7:0/1391267076 shutdown_connections 2024-09-10T11:01:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:01:54.712+0000 7f0aaaaae640 1 -- 172.21.15.7:0/1391267076 wait complete. 2024-09-10T11:01:54.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 127757 39.61 MB/sec execute 54 sec latency 236.881 ms 2024-09-10T11:01:55.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:55 smithi103 ceph-mon[35160]: pgmap v507: 97 pgs: 97 active+clean; 1.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 792 op/s 2024-09-10T11:01:55.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:55 smithi007 ceph-mon[30873]: pgmap v507: 97 pgs: 97 active+clean; 1.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 792 op/s 2024-09-10T11:01:55.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:55 smithi133 ceph-mon[35282]: pgmap v507: 97 pgs: 97 active+clean; 1.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 792 op/s 2024-09-10T11:01:55.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 128668 39.89 MB/sec execute 55 sec latency 103.426 ms 2024-09-10T11:01:56.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 129555 39.97 MB/sec execute 56 sec latency 97.007 ms 2024-09-10T11:01:57.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:57 smithi103 ceph-mon[35160]: pgmap v508: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-09-10T11:01:57.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:57 smithi007 ceph-mon[30873]: pgmap v508: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-09-10T11:01:57.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:57 smithi133 ceph-mon[35282]: pgmap v508: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-09-10T11:01:57.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 130019 39.83 MB/sec execute 57 sec latency 266.959 ms 2024-09-10T11:01:58.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 130600 39.67 MB/sec execute 58 sec latency 117.542 ms 2024-09-10T11:01:59.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:01:59 smithi103 ceph-mon[35160]: pgmap v509: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 779 op/s 2024-09-10T11:01:59.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:01:59 smithi007 ceph-mon[30873]: pgmap v509: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 779 op/s 2024-09-10T11:01:59.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:01:59 smithi133 ceph-mon[35282]: pgmap v509: 97 pgs: 97 active+clean; 1.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 779 op/s 2024-09-10T11:01:59.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 131367 39.48 MB/sec execute 59 sec latency 59.196 ms 2024-09-10T11:02:00.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 132410 39.97 MB/sec execute 60 sec latency 56.360 ms 2024-09-10T11:02:01.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:01 smithi103 ceph-mon[35160]: pgmap v510: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.10k op/s 2024-09-10T11:02:01.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:01 smithi007 ceph-mon[30873]: pgmap v510: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.10k op/s 2024-09-10T11:02:01.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:01 smithi133 ceph-mon[35282]: pgmap v510: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 18 MiB/s wr, 1.10k op/s 2024-09-10T11:02:01.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 132860 39.56 MB/sec execute 61 sec latency 160.231 ms 2024-09-10T11:02:02.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 133008 39.16 MB/sec execute 62 sec latency 377.016 ms 2024-09-10T11:02:03.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:03 smithi103 ceph-mon[35160]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 767 op/s 2024-09-10T11:02:03.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:03 smithi007 ceph-mon[30873]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 767 op/s 2024-09-10T11:02:03.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:03 smithi133 ceph-mon[35282]: pgmap v511: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 767 op/s 2024-09-10T11:02:03.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 133295 38.79 MB/sec execute 63 sec latency 489.511 ms 2024-09-10T11:02:04.376 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:04 smithi133 ceph-mon[35282]: osdmap e107: 12 total, 12 up, 12 in 2024-09-10T11:02:04.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:04 smithi103 ceph-mon[35160]: osdmap e107: 12 total, 12 up, 12 in 2024-09-10T11:02:04.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:04 smithi007 ceph-mon[30873]: osdmap e107: 12 total, 12 up, 12 in 2024-09-10T11:02:04.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 133641 38.51 MB/sec execute 64 sec latency 586.463 ms 2024-09-10T11:02:05.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:05 smithi103 ceph-mon[35160]: pgmap v513: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 920 op/s 2024-09-10T11:02:05.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:05 smithi007 ceph-mon[30873]: pgmap v513: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 920 op/s 2024-09-10T11:02:05.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:05 smithi133 ceph-mon[35282]: pgmap v513: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 920 op/s 2024-09-10T11:02:05.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 133941 38.15 MB/sec execute 65 sec latency 162.453 ms 2024-09-10T11:02:06.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 134537 38.03 MB/sec execute 66 sec latency 94.702 ms 2024-09-10T11:02:07.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:07 smithi103 ceph-mon[35160]: pgmap v514: 97 pgs: 36 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 41 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 910 op/s 2024-09-10T11:02:07.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:07 smithi007 ceph-mon[30873]: pgmap v514: 97 pgs: 36 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 41 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 910 op/s 2024-09-10T11:02:07.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:07 smithi133 ceph-mon[35282]: pgmap v514: 97 pgs: 36 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 41 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 910 op/s 2024-09-10T11:02:07.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.826+0000 7fea51e7b640 1 -- 172.21.15.7:0/3832264634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea4c1069e0 msgr2=0x7fea4c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.826+0000 7fea51e7b640 1 --2- 172.21.15.7:0/3832264634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea4c1069e0 0x7fea4c106dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009860 tx=0x7fea3c02a340 comp rx=0 tx=0).stop 2024-09-10T11:02:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.826+0000 7fea51e7b640 1 -- 172.21.15.7:0/3832264634 shutdown_connections 2024-09-10T11:02:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.826+0000 7fea51e7b640 1 --2- 172.21.15.7:0/3832264634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c101370 0x7fea4c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.826+0000 7fea51e7b640 1 --2- 172.21.15.7:0/3832264634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea4c1009d0 0x7fea4c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.826+0000 7fea51e7b640 1 --2- 172.21.15.7:0/3832264634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea4c1069e0 0x7fea4c106dc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.826+0000 7fea51e7b640 1 -- 172.21.15.7:0/3832264634 >> 172.21.15.7:0/3832264634 conn(0x7fea4c0fc6b0 msgr2=0x7fea4c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.826+0000 7fea51e7b640 1 -- 172.21.15.7:0/3832264634 shutdown_connections 2024-09-10T11:02:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.826+0000 7fea51e7b640 1 -- 172.21.15.7:0/3832264634 wait complete. 2024-09-10T11:02:07.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea51e7b640 1 Processor -- start 2024-09-10T11:02:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea51e7b640 1 -- start start 2024-09-10T11:02:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea51e7b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c1009d0 0x7fea4c0fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea51e7b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea4c101370 0x7fea4c0fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea51e7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea4c1069e0 0x7fea4c0fd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea51e7b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c110e20 con 0x7fea4c101370 2024-09-10T11:02:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea51e7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c110ca0 con 0x7fea4c1069e0 2024-09-10T11:02:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea51e7b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c110fa0 con 0x7fea4c1009d0 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea4affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea4c101370 0x7fea4c0fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea4affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea4c101370 0x7fea4c0fd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36268/0 (socket says 172.21.15.7:36268) 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea4affd640 1 -- 172.21.15.7:0/3098684100 learned_addr learned my addr 172.21.15.7:0/3098684100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.827+0000 7fea4bfff640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea4c1069e0 0x7fea4c0fd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea4affd640 1 -- 172.21.15.7:0/3098684100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c1009d0 msgr2=0x7fea4c0fcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea4affd640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c1009d0 0x7fea4c0fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea4affd640 1 -- 172.21.15.7:0/3098684100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea4c1069e0 msgr2=0x7fea4c0fd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea4affd640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea4c1069e0 0x7fea4c0fd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea4affd640 1 -- 172.21.15.7:0/3098684100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c1a6820 con 0x7fea4c101370 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea4affd640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea4c101370 0x7fea4c0fd180 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e9c0 tx=0x7fea4000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea48ff9640 1 -- 172.21.15.7:0/3098684100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4000cde0 con 0x7fea4c101370 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea51e7b640 1 -- 172.21.15.7:0/3098684100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c1a6b10 con 0x7fea4c101370 2024-09-10T11:02:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea51e7b640 1 -- 172.21.15.7:0/3098684100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c1a7050 con 0x7fea4c101370 2024-09-10T11:02:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea48ff9640 1 -- 172.21.15.7:0/3098684100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea40021910 con 0x7fea4c101370 2024-09-10T11:02:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.828+0000 7fea48ff9640 1 -- 172.21.15.7:0/3098684100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400109b0 con 0x7fea4c101370 2024-09-10T11:02:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea54c1a740 1 Processor -- start 2024-09-10T11:02:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea48ff9640 1 -- 172.21.15.7:0/3098684100 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fea40021ac0 con 0x7fea4c101370 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea54c1a740 1 -- start start 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff6eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6f410 0x55798ff79770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ff7c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea54c1a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55798fed32f0 con 0x55798ff6eaf0 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea54c1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55798fe7b270 con 0x55798ff79cb0 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea54c1a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55798fecf2b0 con 0x55798ff6f410 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff6eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff6eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36274/0 (socket says 172.21.15.7:36274) 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea4bfff640 1 -- 172.21.15.7:0/3154953325 learned_addr learned my addr 172.21.15.7:0/3154953325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.832+0000 7fea4affd640 1 --2- 172.21.15.7:0/3154953325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ff7c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.833+0000 7fea4bfff640 1 -- 172.21.15.7:0/3154953325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6f410 msgr2=0x55798ff79770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.833+0000 7fea4bfff640 1 --2- 172.21.15.7:0/3154953325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6f410 0x55798ff79770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.833+0000 7fea4bfff640 1 -- 172.21.15.7:0/3154953325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 msgr2=0x55798ff7c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.833+0000 7fea4bfff640 1 --2- 172.21.15.7:0/3154953325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ff7c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.833+0000 7fea4bfff640 1 -- 172.21.15.7:0/3154953325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798fe7a340 con 0x55798ff6eaf0 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.833+0000 7fea48ff9640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fea340a2e20 0x7fea340a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.833+0000 7fea4bfff640 1 --2- 172.21.15.7:0/3154953325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff6eed0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fea44009f50 tx=0x7fea440118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1397776a2be41863 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.833+0000 7fea4b7fe640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fea340a2e20 0x7fea340a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.833+0000 7fea31ffb640 1 -- 172.21.15.7:0/3154953325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44004d20 con 0x55798ff6eaf0 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.834+0000 7fea31ffb640 1 -- 172.21.15.7:0/3154953325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea44004050 con 0x55798ff6eaf0 2024-09-10T11:02:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.834+0000 7fea31ffb640 1 -- 172.21.15.7:0/3154953325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44005280 con 0x55798ff6eaf0 2024-09-10T11:02:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.834+0000 7fea48ff9640 1 -- 172.21.15.7:0/3098684100 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8608+0+0 (secure 0 0 0) 0x7fea400cf7f0 con 0x7fea4c101370 2024-09-10T11:02:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 -- 172.21.15.7:0/3154953325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 msgr2=0x55798ff6eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 --2- 172.21.15.7:0/3154953325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff6eed0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fea44009f50 tx=0x7fea440118d0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea4b7fe640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fea340a2e20 0x7fea340a52e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea3c005b60 tx=0x7fea3c005a90 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 -- 172.21.15.7:0/3154953325 shutdown_connections 2024-09-10T11:02:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 --2- 172.21.15.7:0/3154953325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ff7c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 --2- 172.21.15.7:0/3154953325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6f410 0x55798ff79770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 --2- 172.21.15.7:0/3154953325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff6eed0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 -- 172.21.15.7:0/3154953325 >> 172.21.15.7:0/3154953325 conn(0x55798ff6b060 msgr2=0x55798ff6b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 -- 172.21.15.7:0/3154953325 shutdown_connections 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 -- 172.21.15.7:0/3154953325 wait complete. 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 -- start start 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.835+0000 7fea54c1a740 1 Processor -- start 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff76b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6f410 0x55798ff77090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ff775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea54c1a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55798ff7c880 con 0x55798ff6eaf0 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea54c1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55798fe5c290 con 0x55798ff79cb0 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea54c1a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55798ff7ca00 con 0x55798ff6f410 2024-09-10T11:02:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea54c1a740 10 client.0 _ll_register_callbacks cb 0x55798fe8d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff76b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff76b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36278/0 (socket says 172.21.15.7:36278) 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4b7fe640 1 -- 172.21.15.7:0/291425850 learned_addr learned my addr 172.21.15.7:0/291425850 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4bfff640 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ff775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4b7fe640 1 -- 172.21.15.7:0/291425850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6f410 msgr2=0x55798ff77090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4b7fe640 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6f410 0x55798ff77090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4b7fe640 1 -- 172.21.15.7:0/291425850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 msgr2=0x55798ff775d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4b7fe640 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ff775d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4b7fe640 1 -- 172.21.15.7:0/291425850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798fbe6210 con 0x55798ff6eaf0 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4bfff640 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ff775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea4b7fe640 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff76b50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fea3c034000 tx=0x7fea3c030670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5431f2bfe5df1f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.836+0000 7fea2affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.837+0000 7fea2affd640 1 -- 172.21.15.7:0/291425850 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c04d060 con 0x55798ff6eaf0 2024-09-10T11:02:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.837+0000 7fea2affd640 1 -- 172.21.15.7:0/291425850 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8608+0+0 (secure 0 0 0) 0x7fea3c004040 con 0x55798ff6eaf0 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.837+0000 7fea54c1a740 1 -- 172.21.15.7:0/291425850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea44004050 con 0x55798ff6eaf0 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.837+0000 7fea2affd640 1 -- 172.21.15.7:0/291425850 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fea3c037040 con 0x55798ff6eaf0 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.837+0000 7fea54c1a740 10 client.16560 fetch_fsmap learned FSMap version 51 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.837+0000 7fea54c1a740 1 -- 172.21.15.7:0/291425850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55798fcb2620 con 0x55798ff6eaf0 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.837+0000 7fea2affd640 1 -- 172.21.15.7:0/291425850 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea44004050 con 0x55798ff6eaf0 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.838+0000 7fea2affd640 1 -- 172.21.15.7:0/291425850 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 51) ==== 8362+0+0 (secure 0 0 0) 0x7fea3c03a030 con 0x55798ff6eaf0 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.838+0000 7fea54c1a740 10 client.16560 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.838+0000 7fea54c1a740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.838+0000 7fea54c1a740 20 client.16560 populate_metadata read hostname 'smithi007' 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.838+0000 7fea54c1a740 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55798ff89c40 0x55798ff8c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.838+0000 7fea54c1a740 4 client.16560 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:02:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.838+0000 7fea54c1a740 1 -- 172.21.15.7:0/291425850 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55798fee0a20 con 0x55798ff89c40 2024-09-10T11:02:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.838+0000 7fea4affd640 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55798ff89c40 0x55798ff8c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.839+0000 7fea4affd640 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55798ff89c40 0x55798ff8c0b0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f5aeab316621434 server_cookie=c4158e4ab60e9aa8 in_seq=0 out_seq=0 2024-09-10T11:02:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.839+0000 7fea2affd640 10 client.16560 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:02:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea2affd640 1 -- 172.21.15.7:0/291425850 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55798fee0a20 con 0x55798ff89c40 2024-09-10T11:02:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea2affd640 10 client.16560 handle_command_reply: tid=0 2024-09-10T11:02:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea54c1a740 1 client.16560 shutdown 2024-09-10T11:02:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea54c1a740 1 -- 172.21.15.7:0/291425850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 msgr2=0x55798ff76b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea54c1a740 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff76b50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fea3c034000 tx=0x7fea3c030670 comp rx=0 tx=0).stop 2024-09-10T11:02:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea54c1a740 1 -- 172.21.15.7:0/291425850 shutdown_connections 2024-09-10T11:02:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea54c1a740 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55798ff89c40 0x55798ff8c0b0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea54c1a740 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ff775d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea54c1a740 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6f410 0x55798ff77090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea54c1a740 1 --2- 172.21.15.7:0/291425850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6eaf0 0x55798ff76b50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea54c1a740 1 -- 172.21.15.7:0/291425850 >> 172.21.15.7:0/291425850 conn(0x55798ff6b060 msgr2=0x55798ff70d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea2affd640 0 client.16560 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:02:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.845+0000 7fea54c1a740 1 -- 172.21.15.7:0/291425850 shutdown_connections 2024-09-10T11:02:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.846+0000 7fea54c1a740 1 -- 172.21.15.7:0/291425850 wait complete. 2024-09-10T11:02:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.846+0000 7fea54c1a740 20 client.16560 trim_cache size 0 max 16384 2024-09-10T11:02:07.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.868+0000 7fea54c1a740 1 Processor -- start 2024-09-10T11:02:07.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.868+0000 7fea54c1a740 1 -- start start 2024-09-10T11:02:07.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 0x55798ff71560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff6f410 0x55798ff78ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff79cb0 0x55798ff77400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea54c1a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea38011180 con 0x55798ff79cb0 2024-09-10T11:02:07.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea54c1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea3800e9f0 con 0x55798ff6f410 2024-09-10T11:02:07.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea54c1a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea3800c840 con 0x55798ff6eaf0 2024-09-10T11:02:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff6f410 0x55798ff78ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4affd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff6f410 0x55798ff78ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.7:36574/0 (socket says 172.21.15.7:36574) 2024-09-10T11:02:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4affd640 1 -- 172.21.15.7:0/1089171101 learned_addr learned my addr 172.21.15.7:0/1089171101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4bfff640 1 --2- 172.21.15.7:0/1089171101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff79cb0 0x55798ff77400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4b7fe640 1 --2- 172.21.15.7:0/1089171101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 0x55798ff71560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4affd640 1 -- 172.21.15.7:0/1089171101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 msgr2=0x55798ff71560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:02:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4affd640 1 --2- 172.21.15.7:0/1089171101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 0x55798ff71560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4affd640 1 -- 172.21.15.7:0/1089171101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff79cb0 msgr2=0x55798ff77400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:02:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4affd640 1 --2- 172.21.15.7:0/1089171101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff79cb0 0x55798ff77400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4affd640 1 -- 172.21.15.7:0/1089171101 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798ff6ae90 con 0x55798ff6f410 2024-09-10T11:02:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.869+0000 7fea4affd640 1 --2- 172.21.15.7:0/1089171101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff6f410 0x55798ff78ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c6a0 tx=0x7fea4000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec2635d3d0cb474 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.870+0000 7fea317fa640 1 -- 172.21.15.7:0/1089171101 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40021db0 con 0x55798ff6f410 2024-09-10T11:02:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.870+0000 7fea317fa640 1 -- 172.21.15.7:0/1089171101 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4009c380 con 0x55798ff6f410 2024-09-10T11:02:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.870+0000 7fea317fa640 1 -- 172.21.15.7:0/1089171101 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400982f0 con 0x55798ff6f410 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.871+0000 7fea54c1a740 1 -- 172.21.15.7:0/1089171101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff6f410 msgr2=0x55798ff78ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.871+0000 7fea54c1a740 1 --2- 172.21.15.7:0/1089171101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff6f410 0x55798ff78ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c6a0 tx=0x7fea4000c890 comp rx=0 tx=0).stop 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.871+0000 7fea54c1a740 1 -- 172.21.15.7:0/1089171101 shutdown_connections 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.871+0000 7fea54c1a740 1 --2- 172.21.15.7:0/1089171101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff79cb0 0x55798ff77400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.871+0000 7fea54c1a740 1 --2- 172.21.15.7:0/1089171101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff6f410 0x55798ff78ac0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.871+0000 7fea54c1a740 1 --2- 172.21.15.7:0/1089171101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 0x55798ff71560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.871+0000 7fea54c1a740 1 -- 172.21.15.7:0/1089171101 >> 172.21.15.7:0/1089171101 conn(0x55798ff6b060 msgr2=0x55798ff783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.871+0000 7fea54c1a740 1 -- 172.21.15.7:0/1089171101 shutdown_connections 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.871+0000 7fea54c1a740 1 -- 172.21.15.7:0/1089171101 wait complete. 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.871+0000 7fea54c1a740 1 -- start start 2024-09-10T11:02:07.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea54c1a740 1 Processor -- start 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 0x55798ff7d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6f410 0x55798ff7d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea54c1a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ffec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea54c1a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea3800ceb0 con 0x55798ff6f410 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea54c1a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea3800e320 con 0x55798ff79cb0 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea54c1a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea3800d580 con 0x55798ff6eaf0 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea54c1a740 10 client.0 _ll_register_callbacks cb 0x7fea3c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6f410 0x55798ff7d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6f410 0x55798ff7d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36310/0 (socket says 172.21.15.7:36310) 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 0x55798ff7d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4affd640 1 -- 172.21.15.7:0/3766762779 learned_addr learned my addr 172.21.15.7:0/3766762779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4bfff640 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ffec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4b7fe640 1 -- 172.21.15.7:0/3766762779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 msgr2=0x55798ffec9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4b7fe640 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ffec9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4b7fe640 1 -- 172.21.15.7:0/3766762779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6f410 msgr2=0x55798ff7d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4b7fe640 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6f410 0x55798ff7d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4b7fe640 1 -- 172.21.15.7:0/3766762779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798ff662d0 con 0x55798ff6eaf0 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.872+0000 7fea4affd640 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6f410 0x55798ff7d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.873+0000 7fea4b7fe640 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 0x55798ff7d460 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea3c02a8e0 tx=0x7fea3c048960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=975121d441c5ebaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.873+0000 7fea2affd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.873+0000 7fea2affd640 1 -- 172.21.15.7:0/3766762779 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c04d070 con 0x55798ff6eaf0 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.873+0000 7fea54c1a740 1 -- 172.21.15.7:0/3766762779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4009c380 con 0x55798ff6eaf0 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.873+0000 7fea2affd640 1 -- 172.21.15.7:0/3766762779 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8608+0+0 (secure 0 0 0) 0x7fea3c048d60 con 0x55798ff6eaf0 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.873+0000 7fea2affd640 1 -- 172.21.15.7:0/3766762779 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fea3c0044b0 con 0x55798ff6eaf0 2024-09-10T11:02:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.873+0000 7fea54c1a740 10 client.26194 fetch_fsmap learned FSMap version 51 2024-09-10T11:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.873+0000 7fea54c1a740 1 -- 172.21.15.7:0/3766762779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55798ff796e0 con 0x55798ff6eaf0 2024-09-10T11:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.874+0000 7fea2affd640 1 -- 172.21.15.7:0/3766762779 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea4009c380 con 0x55798ff6eaf0 2024-09-10T11:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.874+0000 7fea2affd640 1 -- 172.21.15.7:0/3766762779 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 51) ==== 8362+0+0 (secure 0 0 0) 0x7fea3c03a040 con 0x55798ff6eaf0 2024-09-10T11:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.874+0000 7fea54c1a740 10 client.26194 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T11:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.874+0000 7fea54c1a740 10 client.26194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-10T11:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.874+0000 7fea54c1a740 20 client.26194 populate_metadata read hostname 'smithi007' 2024-09-10T11:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.874+0000 7fea54c1a740 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55798fff4860 0x55798fff6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.874+0000 7fea54c1a740 4 client.26194 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T11:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.874+0000 7fea54c1a740 1 -- 172.21.15.7:0/3766762779 --> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55798fedad70 con 0x55798fff4860 2024-09-10T11:02:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.874+0000 7fea4affd640 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55798fff4860 0x55798fff6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.874+0000 7fea4affd640 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55798fff4860 0x55798fff6c80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59639fe9d1a259ee server_cookie=10e115ba8f2f0a60 in_seq=0 out_seq=0 2024-09-10T11:02:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:07.875+0000 7fea2affd640 10 client.26194 ms_handle_connect on v2:172.21.15.133:6838/1182528850 2024-09-10T11:02:07.888 INFO:tasks.workunit.client.0.smithi007.stdout: 10 135177 37.79 MB/sec execute 67 sec latency 408.533 ms 2024-09-10T11:02:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.335+0000 7fea2affd640 1 -- 172.21.15.7:0/3766762779 <== mds.0 v2:172.21.15.133:6838/1182528850 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55798fedad70 con 0x55798fff4860 2024-09-10T11:02:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.335+0000 7fea2affd640 10 client.26194 handle_command_reply: tid=0 2024-09-10T11:02:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.335+0000 7fea54c1a740 1 client.26194 shutdown 2024-09-10T11:02:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 1 -- 172.21.15.7:0/3766762779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 msgr2=0x55798ff7d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 0x55798ff7d460 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea3c02a8e0 tx=0x7fea3c048960 comp rx=0 tx=0).stop 2024-09-10T11:02:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 1 -- 172.21.15.7:0/3766762779 shutdown_connections 2024-09-10T11:02:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.133:6838/1182528850,v1:172.21.15.133:6839/1182528850] conn(0x55798fff4860 0x55798fff6c80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55798ff79cb0 0x55798ffec9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798ff6f410 0x55798ff7d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 1 --2- 172.21.15.7:0/3766762779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55798ff6eaf0 0x55798ff7d460 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 1 -- 172.21.15.7:0/3766762779 >> 172.21.15.7:0/3766762779 conn(0x55798ff6b060 msgr2=0x55798ff6a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea2affd640 0 client.26194 ms_handle_reset on v2:172.21.15.133:6838/1182528850 2024-09-10T11:02:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 1 -- 172.21.15.7:0/3766762779 shutdown_connections 2024-09-10T11:02:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 1 -- 172.21.15.7:0/3766762779 wait complete. 2024-09-10T11:02:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.336+0000 7fea54c1a740 20 client.26194 trim_cache size 0 max 16384 2024-09-10T11:02:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-09-10T11:02:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-09-10T11:02:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-09-10T11:02:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-09-10T11:02:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 -- 172.21.15.7:0/3098684100 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fea340a2e20 msgr2=0x7fea340a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fea340a2e20 0x7fea340a52e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea3c005b60 tx=0x7fea3c005a90 comp rx=0 tx=0).stop 2024-09-10T11:02:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 -- 172.21.15.7:0/3098684100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea4c101370 msgr2=0x7fea4c0fd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea4c101370 0x7fea4c0fd180 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e9c0 tx=0x7fea4000ee80 comp rx=0 tx=0).stop 2024-09-10T11:02:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 -- 172.21.15.7:0/3098684100 shutdown_connections 2024-09-10T11:02:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea4c1069e0 0x7fea4c0fd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fea340a2e20 0x7fea340a52e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea4c101370 0x7fea4c0fd180 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 --2- 172.21.15.7:0/3098684100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea4c1009d0 0x7fea4c0fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 -- 172.21.15.7:0/3098684100 >> 172.21.15.7:0/3098684100 conn(0x7fea4c0fc6b0 msgr2=0x7fea4c06a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.337+0000 7fea51e7b640 1 -- 172.21.15.7:0/3098684100 shutdown_connections 2024-09-10T11:02:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-09-10T11:02:08.340+0000 7fea51e7b640 1 -- 172.21.15.7:0/3098684100 wait complete. 2024-09-10T11:02:08.672 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:02:08 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e[65303]: ignoring --setuser ceph since I am not root 2024-09-10T11:02:08.672 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:02:08 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e[65303]: ignoring --setgroup ceph since I am not root 2024-09-10T11:02:08.672 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:02:08 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e[65303]: starting mds.e at 2024-09-10T11:02:08.889 INFO:tasks.workunit.client.0.smithi007.stdout: 10 135777 37.94 MB/sec execute 68 sec latency 244.396 ms 2024-09-10T11:02:08.968 INFO:tasks.workunit.client.0.smithi007.stdout:[132801] write failed on handle 12617 (Resource temporarily unavailable) 2024-09-10T11:02:08.968 INFO:tasks.workunit.client.0.smithi007.stdout:Child failed with status 1 2024-09-10T11:02:08.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:02:08.976 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-10T11:02:08.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T11:02:09.321 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:09 smithi007 ceph-mon[30873]: pgmap v515: 97 pgs: 36 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 41 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 910 op/s 2024-09-10T11:02:09.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:09 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:02:09.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:09 smithi007 ceph-mon[30873]: daemon mds.e restarted 2024-09-10T11:02:09.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:09 smithi007 ceph-mon[30873]: osdmap e108: 12 total, 12 up, 12 in 2024-09-10T11:02:09.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:09 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.103:6834/1460675138,v1:172.21.15.103:6835/1460675138] up:boot 2024-09-10T11:02:09.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:09 smithi007 ceph-mon[30873]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:02:09.322 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:09 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T11:02:09.323 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi007 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-10T11:02:09.567 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=0974d9d0be4c4dc3b6304d41fbd461a8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi007 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-10T11:02:09.570 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-10T11:02:09.581 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T11:02:09.586 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:09 smithi133 ceph-mon[35282]: pgmap v515: 97 pgs: 36 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 41 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 910 op/s 2024-09-10T11:02:09.586 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:09 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:02:09.586 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:09 smithi133 ceph-mon[35282]: daemon mds.e restarted 2024-09-10T11:02:09.586 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:09 smithi133 ceph-mon[35282]: osdmap e108: 12 total, 12 up, 12 in 2024-09-10T11:02:09.586 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:09 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.103:6834/1460675138,v1:172.21.15.103:6835/1460675138] up:boot 2024-09-10T11:02:09.587 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:09 smithi133 ceph-mon[35282]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:02:09.587 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:09 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T11:02:09.587 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:09 smithi103 ceph-mon[35160]: pgmap v515: 97 pgs: 36 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 41 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 910 op/s 2024-09-10T11:02:09.587 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:09 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:02:09.587 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:09 smithi103 ceph-mon[35160]: daemon mds.e restarted 2024-09-10T11:02:09.587 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:09 smithi103 ceph-mon[35160]: osdmap e108: 12 total, 12 up, 12 in 2024-09-10T11:02:09.587 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:09 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.103:6834/1460675138,v1:172.21.15.103:6835/1460675138] up:boot 2024-09-10T11:02:09.588 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:09 smithi103 ceph-mon[35160]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T11:02:09.588 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:09 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T11:02:09.596 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-10T11:02:09.615 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-10T11:02:09.615 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T11:02:09.615 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T11:02:09.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-10T11:02:09.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-10T11:02:09.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-10T11:02:09.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.879+0000 7f88fb253640 1 -- 172.21.15.7:0/2287262188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f4074ec0 msgr2=0x7f88f410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:09.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.879+0000 7f88fb253640 1 --2- 172.21.15.7:0/2287262188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f4074ec0 0x7f88f410de60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009900 tx=0x7f88e002a480 comp rx=0 tx=0).stop 2024-09-10T11:02:09.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.879+0000 7f88fb253640 1 -- 172.21.15.7:0/2287262188 shutdown_connections 2024-09-10T11:02:09.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.879+0000 7f88fb253640 1 --2- 172.21.15.7:0/2287262188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f410e3a0 0x7f88f4110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:09.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.879+0000 7f88fb253640 1 --2- 172.21.15.7:0/2287262188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f4074ec0 0x7f88f410de60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:09.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.879+0000 7f88fb253640 1 --2- 172.21.15.7:0/2287262188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88f40745a0 0x7f88f4074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:09.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.879+0000 7f88fb253640 1 -- 172.21.15.7:0/2287262188 >> 172.21.15.7:0/2287262188 conn(0x7f88f40fd7e0 msgr2=0x7f88f40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.879+0000 7f88fb253640 1 -- 172.21.15.7:0/2287262188 shutdown_connections 2024-09-10T11:02:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.879+0000 7f88fb253640 1 -- 172.21.15.7:0/2287262188 wait complete. 2024-09-10T11:02:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88fb253640 1 Processor -- start 2024-09-10T11:02:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88fb253640 1 -- start start 2024-09-10T11:02:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88fb253640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88f40745a0 0x7f88f419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:09.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88fb253640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f4074ec0 0x7f88f419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88fb253640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f410e3a0 0x7f88f419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88fb253640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88f4114130 con 0x7f88f4074ec0 2024-09-10T11:02:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88fb253640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88f4113fb0 con 0x7f88f40745a0 2024-09-10T11:02:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88fb253640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88f41142b0 con 0x7f88f410e3a0 2024-09-10T11:02:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88f3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f4074ec0 0x7f88f419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88f3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f4074ec0 0x7f88f419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36332/0 (socket says 172.21.15.7:36332) 2024-09-10T11:02:09.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88f3fff640 1 -- 172.21.15.7:0/155776649 learned_addr learned my addr 172.21.15.7:0/155776649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88f97f0640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f410e3a0 0x7f88f419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.880+0000 7f88f8fef640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88f40745a0 0x7f88f419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88f3fff640 1 -- 172.21.15.7:0/155776649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f410e3a0 msgr2=0x7f88f419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88f3fff640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f410e3a0 0x7f88f419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88f3fff640 1 -- 172.21.15.7:0/155776649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88f40745a0 msgr2=0x7f88f419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88f3fff640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88f40745a0 0x7f88f419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:09.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88f3fff640 1 -- 172.21.15.7:0/155776649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f419f5f0 con 0x7f88f4074ec0 2024-09-10T11:02:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88f8fef640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88f40745a0 0x7f88f419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:02:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88f3fff640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f4074ec0 0x7f88f419ab40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f88e0006010 tx=0x7f88e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88f1ffb640 1 -- 172.21.15.7:0/155776649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e003f030 con 0x7f88f4074ec0 2024-09-10T11:02:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88fb253640 1 -- 172.21.15.7:0/155776649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f419f880 con 0x7f88f4074ec0 2024-09-10T11:02:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88f1ffb640 1 -- 172.21.15.7:0/155776649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88e0003ce0 con 0x7f88f4074ec0 2024-09-10T11:02:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88f1ffb640 1 -- 172.21.15.7:0/155776649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0035070 con 0x7f88f4074ec0 2024-09-10T11:02:09.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.881+0000 7f88fb253640 1 -- 172.21.15.7:0/155776649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f41a7170 con 0x7f88f4074ec0 2024-09-10T11:02:09.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.882+0000 7f88fb253640 1 -- 172.21.15.7:0/155776649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f4074980 con 0x7f88f4074ec0 2024-09-10T11:02:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.883+0000 7f88f1ffb640 1 -- 172.21.15.7:0/155776649 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f88e0042030 con 0x7f88f4074ec0 2024-09-10T11:02:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.884+0000 7f88f1ffb640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f88e40a2da0 0x7f88e40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.884+0000 7f88f1ffb640 1 -- 172.21.15.7:0/155776649 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8564+0+0 (secure 0 0 0) 0x7f88e00efa20 con 0x7f88f4074ec0 2024-09-10T11:02:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.884+0000 7f88f8fef640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f88e40a2da0 0x7f88e40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.885+0000 7f88f8fef640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f88e40a2da0 0x7f88e40a5260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009930 tx=0x7f88e8008040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:09.888+0000 7f88f1ffb640 1 -- 172.21.15.7:0/155776649 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88e00bc670 con 0x7f88f4074ec0 2024-09-10T11:02:10.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.086+0000 7f88fb253640 1 -- 172.21.15.7:0/155776649 --> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f88f4062630 con 0x7f88e40a2da0 2024-09-10T11:02:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.097+0000 7f88f1ffb640 1 -- 172.21.15.7:0/155776649 <== mgr.15132 v2:172.21.15.7:6800/354580929 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f88f4062630 con 0x7f88e40a2da0 2024-09-10T11:02:10.098 INFO:teuthology.orchestra.run.smithi007.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-10T00:00:00", "created": "2024-09-10T10:45:56", "first": "2024-09-10T10:46:00", "last": "2024-09-10T11:02:00", "last_pruned": "2024-09-10T11:02:01", "created_count": 17, "pruned_count": 11, "active": true} 2024-09-10T11:02:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 -- 172.21.15.7:0/155776649 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f88e40a2da0 msgr2=0x7f88e40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f88e40a2da0 0x7f88e40a5260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009930 tx=0x7f88e8008040 comp rx=0 tx=0).stop 2024-09-10T11:02:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 -- 172.21.15.7:0/155776649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f4074ec0 msgr2=0x7f88f419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f4074ec0 0x7f88f419ab40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f88e0006010 tx=0x7f88e00042f0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 -- 172.21.15.7:0/155776649 shutdown_connections 2024-09-10T11:02:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f410e3a0 0x7f88f419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f88e40a2da0 0x7f88e40a5260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f4074ec0 0x7f88f419ab40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 --2- 172.21.15.7:0/155776649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88f40745a0 0x7f88f419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 -- 172.21.15.7:0/155776649 >> 172.21.15.7:0/155776649 conn(0x7f88f40fd7e0 msgr2=0x7f88f40fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 -- 172.21.15.7:0/155776649 shutdown_connections 2024-09-10T11:02:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.099+0000 7f88fb253640 1 -- 172.21.15.7:0/155776649 wait complete. 2024-09-10T11:02:10.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-10T11:02:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.215+0000 7f28eb8dd640 1 -- 172.21.15.7:0/501520694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28e41008d0 msgr2=0x7f28e4100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.215+0000 7f28eb8dd640 1 --2- 172.21.15.7:0/501520694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28e41008d0 0x7f28e4100cb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009900 tx=0x7f28e002a410 comp rx=0 tx=0).stop 2024-09-10T11:02:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 -- 172.21.15.7:0/501520694 shutdown_connections 2024-09-10T11:02:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 --2- 172.21.15.7:0/501520694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e410e480 0x7f28e4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 --2- 172.21.15.7:0/501520694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28e4101280 0x7f28e410deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 --2- 172.21.15.7:0/501520694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28e41008d0 0x7f28e4100cb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 -- 172.21.15.7:0/501520694 >> 172.21.15.7:0/501520694 conn(0x7f28e40fc6b0 msgr2=0x7f28e40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 -- 172.21.15.7:0/501520694 shutdown_connections 2024-09-10T11:02:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 -- 172.21.15.7:0/501520694 wait complete. 2024-09-10T11:02:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 Processor -- start 2024-09-10T11:02:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 -- start start 2024-09-10T11:02:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28e41008d0 0x7f28e419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101280 0x7f28e419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28e410e480 0x7f28e419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f28e41140e0 con 0x7f28e41008d0 2024-09-10T11:02:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28e4113f60 con 0x7f28e410e480 2024-09-10T11:02:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28eb8dd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f28e4114260 con 0x7f28e4101280 2024-09-10T11:02:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.216+0000 7f28e9679640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28e41008d0 0x7f28e419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e9679640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28e41008d0 0x7f28e419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53126/0 (socket says 172.21.15.7:53126) 2024-09-10T11:02:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e9679640 1 -- 172.21.15.7:0/2241360326 learned_addr learned my addr 172.21.15.7:0/2241360326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e8e78640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101280 0x7f28e419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e9e7a640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28e410e480 0x7f28e419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e9679640 1 -- 172.21.15.7:0/2241360326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101280 msgr2=0x7f28e419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e9679640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101280 0x7f28e419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e9679640 1 -- 172.21.15.7:0/2241360326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28e410e480 msgr2=0x7f28e419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e9679640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28e410e480 0x7f28e419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e9679640 1 -- 172.21.15.7:0/2241360326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e419f580 con 0x7f28e41008d0 2024-09-10T11:02:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e8e78640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101280 0x7f28e419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:02:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28e9679640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28e41008d0 0x7f28e419a590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f28e0006010 tx=0x7f28e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28da7fc640 1 -- 172.21.15.7:0/2241360326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e003f030 con 0x7f28e41008d0 2024-09-10T11:02:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28da7fc640 1 -- 172.21.15.7:0/2241360326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e0003ce0 con 0x7f28e41008d0 2024-09-10T11:02:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28eb8dd640 1 -- 172.21.15.7:0/2241360326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e419f810 con 0x7f28e41008d0 2024-09-10T11:02:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28da7fc640 1 -- 172.21.15.7:0/2241360326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0035070 con 0x7f28e41008d0 2024-09-10T11:02:10.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.217+0000 7f28eb8dd640 1 -- 172.21.15.7:0/2241360326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e41a7100 con 0x7f28e41008d0 2024-09-10T11:02:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.218+0000 7f28eb8dd640 1 -- 172.21.15.7:0/2241360326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e4100cb0 con 0x7f28e41008d0 2024-09-10T11:02:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.219+0000 7f28da7fc640 1 -- 172.21.15.7:0/2241360326 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f28e0042030 con 0x7f28e41008d0 2024-09-10T11:02:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.220+0000 7f28da7fc640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f28d00a2e20 0x7f28d00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.220+0000 7f28e8e78640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f28d00a2e20 0x7f28d00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.221+0000 7f28da7fc640 1 -- 172.21.15.7:0/2241360326 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8564+0+0 (secure 0 0 0) 0x7f28e00efd70 con 0x7f28e41008d0 2024-09-10T11:02:10.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.221+0000 7f28e8e78640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f28d00a2e20 0x7f28d00a52e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f28d40058c0 tx=0x7f28d4005850 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:10.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.224+0000 7f28da7fc640 1 -- 172.21.15.7:0/2241360326 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28e00bc9c0 con 0x7f28e41008d0 2024-09-10T11:02:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.422+0000 7f28eb8dd640 1 -- 172.21.15.7:0/2241360326 --> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f28e410f760 con 0x7f28d00a2e20 2024-09-10T11:02:10.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.432+0000 7f28da7fc640 1 -- 172.21.15.7:0/2241360326 <== mgr.15132 v2:172.21.15.7:6800/354580929 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f28e410f760 con 0x7f28d00a2e20 2024-09-10T11:02:10.433 INFO:teuthology.orchestra.run.smithi007.stdout:/ 1m 6m3h 2024-09-10T11:02:10.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 -- 172.21.15.7:0/2241360326 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f28d00a2e20 msgr2=0x7f28d00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f28d00a2e20 0x7f28d00a52e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f28d40058c0 tx=0x7f28d4005850 comp rx=0 tx=0).stop 2024-09-10T11:02:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 -- 172.21.15.7:0/2241360326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28e41008d0 msgr2=0x7f28e419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28e41008d0 0x7f28e419a590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f28e0006010 tx=0x7f28e00042f0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 -- 172.21.15.7:0/2241360326 shutdown_connections 2024-09-10T11:02:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28e410e480 0x7f28e419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101280 0x7f28e419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f28d00a2e20 0x7f28d00a52e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 --2- 172.21.15.7:0/2241360326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28e41008d0 0x7f28e419a590 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 -- 172.21.15.7:0/2241360326 >> 172.21.15.7:0/2241360326 conn(0x7f28e40fc6b0 msgr2=0x7f28e40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 -- 172.21.15.7:0/2241360326 shutdown_connections 2024-09-10T11:02:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.434+0000 7f28eb8dd640 1 -- 172.21.15.7:0/2241360326 wait complete. 2024-09-10T11:02:10.446 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T11:02:10.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:02:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.557+0000 7f63e3f0a640 1 -- 172.21.15.7:0/1189833827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63dc10e3a0 msgr2=0x7f63dc110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.557+0000 7f63e3f0a640 1 --2- 172.21.15.7:0/1189833827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63dc10e3a0 0x7f63dc110790 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009ca0 tx=0x7f63d402a8c0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.557+0000 7f63e3f0a640 1 -- 172.21.15.7:0/1189833827 shutdown_connections 2024-09-10T11:02:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.557+0000 7f63e3f0a640 1 --2- 172.21.15.7:0/1189833827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63dc10e3a0 0x7f63dc110790 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.557+0000 7f63e3f0a640 1 --2- 172.21.15.7:0/1189833827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc1011a0 0x7f63dc10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.557+0000 7f63e3f0a640 1 --2- 172.21.15.7:0/1189833827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63dc100880 0x7f63dc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.557+0000 7f63e3f0a640 1 -- 172.21.15.7:0/1189833827 >> 172.21.15.7:0/1189833827 conn(0x7f63dc0fc6c0 msgr2=0x7f63dc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.557+0000 7f63e3f0a640 1 -- 172.21.15.7:0/1189833827 shutdown_connections 2024-09-10T11:02:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.557+0000 7f63e3f0a640 1 -- 172.21.15.7:0/1189833827 wait complete. 2024-09-10T11:02:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e3f0a640 1 Processor -- start 2024-09-10T11:02:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e3f0a640 1 -- start start 2024-09-10T11:02:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e3f0a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63dc100880 0x7f63dc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e3f0a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc1011a0 0x7f63dc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e3f0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63dc10e3a0 0x7f63dc19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e3f0a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc1140b0 con 0x7f63dc100880 2024-09-10T11:02:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e3f0a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc113f30 con 0x7f63dc10e3a0 2024-09-10T11:02:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e3f0a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63dc114230 con 0x7f63dc1011a0 2024-09-10T11:02:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e1ca6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63dc100880 0x7f63dc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e1ca6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63dc100880 0x7f63dc19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53144/0 (socket says 172.21.15.7:53144) 2024-09-10T11:02:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.558+0000 7f63e1ca6640 1 -- 172.21.15.7:0/2023076619 learned_addr learned my addr 172.21.15.7:0/2023076619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63e24a7640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63dc10e3a0 0x7f63dc19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63e1ca6640 1 -- 172.21.15.7:0/2023076619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc1011a0 msgr2=0x7f63dc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63e1ca6640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc1011a0 0x7f63dc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63e1ca6640 1 -- 172.21.15.7:0/2023076619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63dc10e3a0 msgr2=0x7f63dc19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63e1ca6640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63dc10e3a0 0x7f63dc19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63e1ca6640 1 -- 172.21.15.7:0/2023076619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63dc19f510 con 0x7f63dc100880 2024-09-10T11:02:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63e24a7640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63dc10e3a0 0x7f63dc19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:02:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63e1ca6640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63dc100880 0x7f63dc19a520 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c920 tx=0x7f63d800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63d2ffd640 1 -- 172.21.15.7:0/2023076619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8007c70 con 0x7f63dc100880 2024-09-10T11:02:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63d2ffd640 1 -- 172.21.15.7:0/2023076619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d8007e10 con 0x7f63dc100880 2024-09-10T11:02:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63e3f0a640 1 -- 172.21.15.7:0/2023076619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63dc19f800 con 0x7f63dc100880 2024-09-10T11:02:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63d2ffd640 1 -- 172.21.15.7:0/2023076619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8018890 con 0x7f63dc100880 2024-09-10T11:02:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.559+0000 7f63e3f0a640 1 -- 172.21.15.7:0/2023076619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63dc1a7090 con 0x7f63dc100880 2024-09-10T11:02:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.560+0000 7f63e3f0a640 1 -- 172.21.15.7:0/2023076619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63dc100c60 con 0x7f63dc100880 2024-09-10T11:02:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.561+0000 7f63d2ffd640 1 -- 172.21.15.7:0/2023076619 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f63d8018a70 con 0x7f63dc100880 2024-09-10T11:02:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.562+0000 7f63d2ffd640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63c80a2d00 0x7f63c80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.562+0000 7f63e14a5640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63c80a2d00 0x7f63c80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.562+0000 7f63d2ffd640 1 -- 172.21.15.7:0/2023076619 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8564+0+0 (secure 0 0 0) 0x7f63d80cf6e0 con 0x7f63dc100880 2024-09-10T11:02:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.563+0000 7f63e14a5640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63c80a2d00 0x7f63c80a51c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f63dc110c10 tx=0x7f63cc005b60 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.566+0000 7f63d2ffd640 1 -- 172.21.15.7:0/2023076619 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63d8014030 con 0x7f63dc100880 2024-09-10T11:02:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.744+0000 7f63e3f0a640 1 -- 172.21.15.7:0/2023076619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63dc1a73d0 con 0x7f63dc100880 2024-09-10T11:02:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.746+0000 7f63d2ffd640 1 -- 172.21.15.7:0/2023076619 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+23172 (secure 0 0 0) 0x7f63d800de20 con 0x7f63dc100880 2024-09-10T11:02:10.747 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T11:02:10.747 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":108,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:36:33.117443+0000","modified":"2024-09-10T11:02:08.429941+0000","last_up_change":"2024-09-10T10:43:23.913978+0000","last_in_change":"2024-09-10T10:42:59.733844+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:39:37.104114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:07.441548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:09.970273+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9239e99-3a62-4883-b2f1-8964ebb54492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6803","nonce":704452390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6805","nonce":704452390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6809","nonce":704452390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6807","nonce":704452390}]},"public_addr":"172.21.15.7:6803/704452390","cluster_addr":"172.21.15.7:6805/704452390","heartbeat_back_addr":"172.21.15.7:6809/704452390","heartbeat_front_addr":"172.21.15.7:6807/704452390","state":["exists","up"]},{"osd":1,"uuid":"d750ff16-48c8-421d-b46b-49f0d2b7f4e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6801","nonce":2409158189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6803","nonce":2409158189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6807","nonce":2409158189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6805","nonce":2409158189}]},"public_addr":"172.21.15.103:6801/2409158189","cluster_addr":"172.21.15.103:6803/2409158189","heartbeat_back_addr":"172.21.15.103:6807/2409158189","heartbeat_front_addr":"172.21.15.103:6805/2409158189","state":["exists","up"]},{"osd":2,"uuid":"1ecb070a-a2f1-499a-99e7-f1967ebb52ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6801","nonce":1657349858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6803","nonce":1657349858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6807","nonce":1657349858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6805","nonce":1657349858}]},"public_addr":"172.21.15.133:6801/1657349858","cluster_addr":"172.21.15.133:6803/1657349858","heartbeat_back_addr":"172.21.15.133:6807/1657349858","heartbeat_front_addr":"172.21.15.133:6805/1657349858","state":["exists","up"]},{"osd":3,"uuid":"8e53d79e-038f-4a7f-9ac9-16b60f47bb4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6811","nonce":871781144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6813","nonce":871781144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6817","nonce":871781144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6815","nonce":871781144}]},"public_addr":"172.21.15.7:6811/871781144","cluster_addr":"172.21.15.7:6813/871781144","heartbeat_back_addr":"172.21.15.7:6817/871781144","heartbeat_front_addr":"172.21.15.7:6815/871781144","state":["exists","up"]},{"osd":4,"uuid":"e4f04c13-594c-43dd-9a98-d2138061b1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6809","nonce":1937243043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6811","nonce":1937243043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6815","nonce":1937243043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6813","nonce":1937243043}]},"public_addr":"172.21.15.103:6809/1937243043","cluster_addr":"172.21.15.103:6811/1937243043","heartbeat_back_addr":"172.21.15.103:6815/1937243043","heartbeat_front_addr":"172.21.15.103:6813/1937243043","state":["exists","up"]},{"osd":5,"uuid":"73143170-5816-407d-ae03-dcf896057681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6809","nonce":577882774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6811","nonce":577882774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6815","nonce":577882774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6813","nonce":577882774}]},"public_addr":"172.21.15.133:6809/577882774","cluster_addr":"172.21.15.133:6811/577882774","heartbeat_back_addr":"172.21.15.133:6815/577882774","heartbeat_front_addr":"172.21.15.133:6813/577882774","state":["exists","up"]},{"osd":6,"uuid":"3a5904d5-36de-47b0-97fa-5a10b24d0c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6819","nonce":2646192734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6821","nonce":2646192734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6825","nonce":2646192734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6823","nonce":2646192734}]},"public_addr":"172.21.15.7:6819/2646192734","cluster_addr":"172.21.15.7:6821/2646192734","heartbeat_back_addr":"172.21.15.7:6825/2646192734","heartbeat_front_addr":"172.21.15.7:6823/2646192734","state":["exists","up"]},{"osd":7,"uuid":"91e79d42-decd-4405-b9f2-9480f7edfca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6817","nonce":3858867236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6819","nonce":3858867236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6823","nonce":3858867236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6821","nonce":3858867236}]},"public_addr":"172.21.15.103:6817/3858867236","cluster_addr":"172.21.15.103:6819/3858867236","heartbeat_back_addr":"172.21.15.103:6823/3858867236","heartbeat_front_addr":"172.21.15.103:6821/3858867236","state":["exists","up"]},{"osd":8,"uuid":"64373754-e81a-430c-967d-4b4f289f3473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6817","nonce":131428321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6819","nonce":131428321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6823","nonce":131428321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6821","nonce":131428321}]},"public_addr":"172.21.15.133:6817/131428321","cluster_addr":"172.21.15.133:6819/131428321","heartbeat_back_addr":"172.21.15.133:6823/131428321","heartbeat_front_addr":"172.21.15.133:6821/131428321","state":["exists","up"]},{"osd":9,"uuid":"91ebe2c9-bda3-4596-ac76-5678c71f850b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6827","nonce":945633240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6829","nonce":945633240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6833","nonce":945633240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6831","nonce":945633240}]},"public_addr":"172.21.15.7:6827/945633240","cluster_addr":"172.21.15.7:6829/945633240","heartbeat_back_addr":"172.21.15.7:6833/945633240","heartbeat_front_addr":"172.21.15.7:6831/945633240","state":["exists","up"]},{"osd":10,"uuid":"2f65568f-89d1-4a67-a222-8e263d34c710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6825","nonce":907835659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6827","nonce":907835659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6831","nonce":907835659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6829","nonce":907835659}]},"public_addr":"172.21.15.103:6825/907835659","cluster_addr":"172.21.15.103:6827/907835659","heartbeat_back_addr":"172.21.15.103:6831/907835659","heartbeat_front_addr":"172.21.15.103:6829/907835659","state":["exists","up"]},{"osd":11,"uuid":"4b49839b-32e9-4105-abee-a253f1aa3f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6825","nonce":2538184038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6827","nonce":2538184038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6831","nonce":2538184038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6829","nonce":2538184038}]},"public_addr":"172.21.15.133:6825/2538184038","cluster_addr":"172.21.15.133:6827/2538184038","heartbeat_back_addr":"172.21.15.133:6831/2538184038","heartbeat_front_addr":"172.21.15.133:6829/2538184038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:44.541279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:10.103733+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:35.290268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:01.070763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:25.666941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:50.925733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:16.486537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:41.464155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:06.610089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:31.994080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:57.000696+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:21.398669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1740947437":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/3214781379":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/3099326751":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/959611916":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/11850255":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3768226338":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6801/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/262340568":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/258851660":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6800/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/3411487770":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/3525096194":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/3394895925":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/3525096194":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/1846259971":"2024-09-11T10:45:53.785414+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:02:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 -- 172.21.15.7:0/2023076619 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63c80a2d00 msgr2=0x7f63c80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63c80a2d00 0x7f63c80a51c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f63dc110c10 tx=0x7f63cc005b60 comp rx=0 tx=0).stop 2024-09-10T11:02:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 -- 172.21.15.7:0/2023076619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63dc100880 msgr2=0x7f63dc19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63dc100880 0x7f63dc19a520 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c920 tx=0x7f63d800cde0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 -- 172.21.15.7:0/2023076619 shutdown_connections 2024-09-10T11:02:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f63dc10e3a0 0x7f63dc19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63dc1011a0 0x7f63dc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f63c80a2d00 0x7f63c80a51c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 --2- 172.21.15.7:0/2023076619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63dc100880 0x7f63dc19a520 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 -- 172.21.15.7:0/2023076619 >> 172.21.15.7:0/2023076619 conn(0x7f63dc0fc6c0 msgr2=0x7f63dc0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 -- 172.21.15.7:0/2023076619 shutdown_connections 2024-09-10T11:02:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.748+0000 7f63e3f0a640 1 -- 172.21.15.7:0/2023076619 wait complete. 2024-09-10T11:02:10.761 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:39:37.104114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:44:07.441548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:44:09.970273+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:02:10.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:02:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.867+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1716940988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dc0102c80 msgr2=0x7f2dc0103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.867+0000 7f2dc6f68640 1 --2- 172.21.15.7:0/1716940988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dc0102c80 0x7f2dc0103060 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009900 tx=0x7f2db402a480 comp rx=0 tx=0).stop 2024-09-10T11:02:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.867+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1716940988 shutdown_connections 2024-09-10T11:02:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.867+0000 7f2dc6f68640 1 --2- 172.21.15.7:0/1716940988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc010c0d0 0x7f2dc010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.867+0000 7f2dc6f68640 1 --2- 172.21.15.7:0/1716940988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dc01035a0 0x7f2dc010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.867+0000 7f2dc6f68640 1 --2- 172.21.15.7:0/1716940988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dc0102c80 0x7f2dc0103060 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.867+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1716940988 >> 172.21.15.7:0/1716940988 conn(0x7f2dc00fc690 msgr2=0x7f2dc00feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.867+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1716940988 shutdown_connections 2024-09-10T11:02:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.867+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1716940988 wait complete. 2024-09-10T11:02:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dc6f68640 1 Processor -- start 2024-09-10T11:02:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dc6f68640 1 -- start start 2024-09-10T11:02:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dc6f68640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dc0102c80 0x7f2dc0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dc6f68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dc01035a0 0x7f2dc0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dc6f68640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc010c0d0 0x7f2dc019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dc6f68640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc0111eb0 con 0x7f2dc01035a0 2024-09-10T11:02:10.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dc6f68640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc0111d30 con 0x7f2dc0102c80 2024-09-10T11:02:10.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dc6f68640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc0112030 con 0x7f2dc010c0d0 2024-09-10T11:02:10.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dc01035a0 0x7f2dc0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dc01035a0 0x7f2dc0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53160/0 (socket says 172.21.15.7:53160) 2024-09-10T11:02:10.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dbffff640 1 -- 172.21.15.7:0/1111836169 learned_addr learned my addr 172.21.15.7:0/1111836169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dc4d04640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dc0102c80 0x7f2dc0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.868+0000 7f2dc5505640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc010c0d0 0x7f2dc019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dbffff640 1 -- 172.21.15.7:0/1111836169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc010c0d0 msgr2=0x7f2dc019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dbffff640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc010c0d0 0x7f2dc019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dbffff640 1 -- 172.21.15.7:0/1111836169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dc0102c80 msgr2=0x7f2dc0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dbffff640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dc0102c80 0x7f2dc0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dbffff640 1 -- 172.21.15.7:0/1111836169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc019d430 con 0x7f2dc01035a0 2024-09-10T11:02:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dc5505640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc010c0d0 0x7f2dc019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:02:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dbffff640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dc01035a0 0x7f2dc0198980 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00d740 tx=0x7f2dac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dbdffb640 1 -- 172.21.15.7:0/1111836169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac017070 con 0x7f2dc01035a0 2024-09-10T11:02:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dbdffb640 1 -- 172.21.15.7:0/1111836169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dac00de50 con 0x7f2dc01035a0 2024-09-10T11:02:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1111836169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc019d720 con 0x7f2dc01035a0 2024-09-10T11:02:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dbdffb640 1 -- 172.21.15.7:0/1111836169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00f620 con 0x7f2dc01035a0 2024-09-10T11:02:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.869+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1111836169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc01a4fb0 con 0x7f2dc01035a0 2024-09-10T11:02:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.870+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1111836169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc0103060 con 0x7f2dc01035a0 2024-09-10T11:02:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.871+0000 7f2dbdffb640 1 -- 172.21.15.7:0/1111836169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2dac009c30 con 0x7f2dc01035a0 2024-09-10T11:02:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.872+0000 7f2dbdffb640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2db00a2d80 0x7f2db00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.872+0000 7f2dbdffb640 1 -- 172.21.15.7:0/1111836169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8564+0+0 (secure 0 0 0) 0x7f2dac0cefe0 con 0x7f2dc01035a0 2024-09-10T11:02:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.872+0000 7f2dc4d04640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2db00a2d80 0x7f2db00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.873+0000 7f2dc4d04640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2db00a2d80 0x7f2db00a5240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009230 tx=0x7f2db40058d0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:10.876+0000 7f2dbdffb640 1 -- 172.21.15.7:0/1111836169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2dac09bbb0 con 0x7f2dc01035a0 2024-09-10T11:02:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.053+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1111836169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2dc002d7c0 con 0x7f2dc01035a0 2024-09-10T11:02:11.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.054+0000 7f2dbdffb640 1 -- 172.21.15.7:0/1111836169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7f2dac09b4b0 con 0x7f2dc01035a0 2024-09-10T11:02:11.055 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-10T11:02:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1111836169 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2db00a2d80 msgr2=0x7f2db00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2db00a2d80 0x7f2db00a5240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009230 tx=0x7f2db40058d0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1111836169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dc01035a0 msgr2=0x7f2dc0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dc01035a0 0x7f2dc0198980 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00d740 tx=0x7f2dac00dc00 comp rx=0 tx=0).stop 2024-09-10T11:02:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1111836169 shutdown_connections 2024-09-10T11:02:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dc010c0d0 0x7f2dc019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f2db00a2d80 0x7f2db00a5240 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dc01035a0 0x7f2dc0198980 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 --2- 172.21.15.7:0/1111836169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2dc0102c80 0x7f2dc0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1111836169 >> 172.21.15.7:0/1111836169 conn(0x7f2dc00fc690 msgr2=0x7f2dc00fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1111836169 shutdown_connections 2024-09-10T11:02:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.056+0000 7f2dc6f68640 1 -- 172.21.15.7:0/1111836169 wait complete. 2024-09-10T11:02:11.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T11:02:11.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.175+0000 7f7761f11640 1 -- 172.21.15.7:0/4286679866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f775c102d00 msgr2=0x7f775c1030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.175+0000 7f7761f11640 1 --2- 172.21.15.7:0/4286679866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f775c102d00 0x7f775c1030e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f774c009900 tx=0x7f774c02a480 comp rx=0 tx=0).stop 2024-09-10T11:02:11.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.175+0000 7f7761f11640 1 -- 172.21.15.7:0/4286679866 shutdown_connections 2024-09-10T11:02:11.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.175+0000 7f7761f11640 1 --2- 172.21.15.7:0/4286679866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f775c10c0d0 0x7f775c10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.175+0000 7f7761f11640 1 --2- 172.21.15.7:0/4286679866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c103620 0x7f775c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.175+0000 7f7761f11640 1 --2- 172.21.15.7:0/4286679866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f775c102d00 0x7f775c1030e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.175+0000 7f7761f11640 1 -- 172.21.15.7:0/4286679866 >> 172.21.15.7:0/4286679866 conn(0x7f775c0fc690 msgr2=0x7f775c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:11.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.175+0000 7f7761f11640 1 -- 172.21.15.7:0/4286679866 shutdown_connections 2024-09-10T11:02:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.175+0000 7f7761f11640 1 -- 172.21.15.7:0/4286679866 wait complete. 2024-09-10T11:02:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.176+0000 7f7761f11640 1 Processor -- start 2024-09-10T11:02:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.176+0000 7f7761f11640 1 -- start start 2024-09-10T11:02:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.176+0000 7f7761f11640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c102d00 0x7f775c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.176+0000 7f7761f11640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f775c103620 0x7f775c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.176+0000 7f7761f11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f775c10c0d0 0x7f775c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.176+0000 7f7761f11640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f775c111eb0 con 0x7f775c103620 2024-09-10T11:02:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.176+0000 7f7761f11640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f775c111d30 con 0x7f775c102d00 2024-09-10T11:02:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.176+0000 7f7761f11640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f775c112030 con 0x7f775c10c0d0 2024-09-10T11:02:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.176+0000 7f775affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f775c103620 0x7f775c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.176+0000 7f775affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f775c103620 0x7f775c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53178/0 (socket says 172.21.15.7:53178) 2024-09-10T11:02:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f775affd640 1 -- 172.21.15.7:0/539124230 learned_addr learned my addr 172.21.15.7:0/539124230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f775bfff640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f775c10c0d0 0x7f775c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f775b7fe640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c102d00 0x7f775c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f775affd640 1 -- 172.21.15.7:0/539124230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f775c10c0d0 msgr2=0x7f775c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f775affd640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f775c10c0d0 0x7f775c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f775affd640 1 -- 172.21.15.7:0/539124230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c102d00 msgr2=0x7f775c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f775affd640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c102d00 0x7f775c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f775affd640 1 -- 172.21.15.7:0/539124230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c19d440 con 0x7f775c103620 2024-09-10T11:02:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f775affd640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f775c103620 0x7f775c198990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f775000d950 tx=0x7f775000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f7758ff9640 1 -- 172.21.15.7:0/539124230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750017030 con 0x7f775c103620 2024-09-10T11:02:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f7758ff9640 1 -- 172.21.15.7:0/539124230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7750004020 con 0x7f775c103620 2024-09-10T11:02:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f7761f11640 1 -- 172.21.15.7:0/539124230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c19d730 con 0x7f775c103620 2024-09-10T11:02:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f7758ff9640 1 -- 172.21.15.7:0/539124230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77500047a0 con 0x7f775c103620 2024-09-10T11:02:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.177+0000 7f7761f11640 1 -- 172.21.15.7:0/539124230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c1a4fc0 con 0x7f775c103620 2024-09-10T11:02:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.178+0000 7f7761f11640 1 -- 172.21.15.7:0/539124230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775c1030e0 con 0x7f775c103620 2024-09-10T11:02:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.179+0000 7f7758ff9640 1 -- 172.21.15.7:0/539124230 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f77500041d0 con 0x7f775c103620 2024-09-10T11:02:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.180+0000 7f7758ff9640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77440a2d00 0x7f77440a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.180+0000 7f775b7fe640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77440a2d00 0x7f77440a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.180+0000 7f7758ff9640 1 -- 172.21.15.7:0/539124230 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8564+0+0 (secure 0 0 0) 0x7f77500cee40 con 0x7f775c103620 2024-09-10T11:02:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.181+0000 7f775b7fe640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77440a2d00 0x7f77440a51c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f774c005ab0 tx=0x7f774c005a20 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.185+0000 7f7758ff9640 1 -- 172.21.15.7:0/539124230 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f775009ba30 con 0x7f775c103620 2024-09-10T11:02:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.364+0000 7f7761f11640 1 -- 172.21.15.7:0/539124230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f775c02d7c0 con 0x7f775c103620 2024-09-10T11:02:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.364+0000 7f7758ff9640 1 -- 172.21.15.7:0/539124230 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) ==== 104+0+11 (secure 0 0 0) 0x7f775009b330 con 0x7f775c103620 2024-09-10T11:02:11.366 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-10T11:02:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 -- 172.21.15.7:0/539124230 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77440a2d00 msgr2=0x7f77440a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77440a2d00 0x7f77440a51c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f774c005ab0 tx=0x7f774c005a20 comp rx=0 tx=0).stop 2024-09-10T11:02:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 -- 172.21.15.7:0/539124230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f775c103620 msgr2=0x7f775c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f775c103620 0x7f775c198990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f775000d950 tx=0x7f775000de10 comp rx=0 tx=0).stop 2024-09-10T11:02:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 -- 172.21.15.7:0/539124230 shutdown_connections 2024-09-10T11:02:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f775c10c0d0 0x7f775c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f77440a2d00 0x7f77440a51c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f775c103620 0x7f775c198990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 --2- 172.21.15.7:0/539124230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f775c102d00 0x7f775c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 -- 172.21.15.7:0/539124230 >> 172.21.15.7:0/539124230 conn(0x7f775c0fc690 msgr2=0x7f775c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:11.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 -- 172.21.15.7:0/539124230 shutdown_connections 2024-09-10T11:02:11.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.366+0000 7f7761f11640 1 -- 172.21.15.7:0/539124230 wait complete. 2024-09-10T11:02:11.377 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:11 smithi007 ceph-mon[30873]: pgmap v517: 97 pgs: 4 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 89 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-10T11:02:11.377 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:11 smithi007 ceph-mon[30873]: from='client.16572 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:02:11.378 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:11 smithi007 ceph-mon[30873]: from='client.16578 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:02:11.378 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:11 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2023076619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:02:11.378 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:11 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1111836169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T11:02:11.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T11:02:11.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:11 smithi103 ceph-mon[35160]: pgmap v517: 97 pgs: 4 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 89 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-10T11:02:11.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:11 smithi103 ceph-mon[35160]: from='client.16572 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:02:11.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:11 smithi103 ceph-mon[35160]: from='client.16578 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:02:11.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:11 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2023076619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:02:11.423 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:11 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1111836169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T11:02:11.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:11 smithi133 ceph-mon[35282]: pgmap v517: 97 pgs: 4 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 89 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 1.07k op/s 2024-09-10T11:02:11.475 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:11 smithi133 ceph-mon[35282]: from='client.16572 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:02:11.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:11 smithi133 ceph-mon[35282]: from='client.16578 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:02:11.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:11 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2023076619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:02:11.476 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:11 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1111836169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T11:02:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.484+0000 7f9e811fd640 1 -- 172.21.15.7:0/1504206366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e7c1008e0 msgr2=0x7f9e7c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.484+0000 7f9e811fd640 1 --2- 172.21.15.7:0/1504206366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e7c1008e0 0x7f9e7c100cc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009930 tx=0x7f9e6c02a4d0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.484+0000 7f9e811fd640 1 -- 172.21.15.7:0/1504206366 shutdown_connections 2024-09-10T11:02:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.484+0000 7f9e811fd640 1 --2- 172.21.15.7:0/1504206366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7c108130 0x7f9e7c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.484+0000 7f9e811fd640 1 --2- 172.21.15.7:0/1504206366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e7c101200 0x7f9e7c1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.484+0000 7f9e811fd640 1 --2- 172.21.15.7:0/1504206366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e7c1008e0 0x7f9e7c100cc0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.484+0000 7f9e811fd640 1 -- 172.21.15.7:0/1504206366 >> 172.21.15.7:0/1504206366 conn(0x7f9e7c075ea0 msgr2=0x7f9e7c0762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.484+0000 7f9e811fd640 1 -- 172.21.15.7:0/1504206366 shutdown_connections 2024-09-10T11:02:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.484+0000 7f9e811fd640 1 -- 172.21.15.7:0/1504206366 wait complete. 2024-09-10T11:02:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e811fd640 1 Processor -- start 2024-09-10T11:02:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e811fd640 1 -- start start 2024-09-10T11:02:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e811fd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e7c1008e0 0x7f9e7c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e811fd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e7c101200 0x7f9e7c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e811fd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7c108130 0x7f9e7c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e811fd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c111f80 con 0x7f9e7c1008e0 2024-09-10T11:02:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e811fd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c111e00 con 0x7f9e7c101200 2024-09-10T11:02:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e811fd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c112100 con 0x7f9e7c108130 2024-09-10T11:02:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e7ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e7c1008e0 0x7f9e7c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e7ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e7c1008e0 0x7f9e7c198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53202/0 (socket says 172.21.15.7:53202) 2024-09-10T11:02:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.485+0000 7f9e7ad9d640 1 -- 172.21.15.7:0/3830994735 learned_addr learned my addr 172.21.15.7:0/3830994735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e7a59c640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e7c101200 0x7f9e7c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e7b59e640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7c108130 0x7f9e7c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e7ad9d640 1 -- 172.21.15.7:0/3830994735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7c108130 msgr2=0x7f9e7c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e7ad9d640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7c108130 0x7f9e7c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e7ad9d640 1 -- 172.21.15.7:0/3830994735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e7c101200 msgr2=0x7f9e7c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e7ad9d640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e7c101200 0x7f9e7c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e7ad9d640 1 -- 172.21.15.7:0/3830994735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c19d540 con 0x7f9e7c1008e0 2024-09-10T11:02:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e7ad9d640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e7c1008e0 0x7f9e7c198550 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009230 tx=0x7f9e6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e63fff640 1 -- 172.21.15.7:0/3830994735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c03f030 con 0x7f9e7c1008e0 2024-09-10T11:02:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e811fd640 1 -- 172.21.15.7:0/3830994735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c19d7d0 con 0x7f9e7c1008e0 2024-09-10T11:02:11.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e63fff640 1 -- 172.21.15.7:0/3830994735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c003ad0 con 0x7f9e7c1008e0 2024-09-10T11:02:11.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e63fff640 1 -- 172.21.15.7:0/3830994735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c035070 con 0x7f9e7c1008e0 2024-09-10T11:02:11.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.486+0000 7f9e811fd640 1 -- 172.21.15.7:0/3830994735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c1a50c0 con 0x7f9e7c1008e0 2024-09-10T11:02:11.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.487+0000 7f9e811fd640 1 -- 172.21.15.7:0/3830994735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7c100cc0 con 0x7f9e7c1008e0 2024-09-10T11:02:11.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.488+0000 7f9e63fff640 1 -- 172.21.15.7:0/3830994735 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9e6c042030 con 0x7f9e7c1008e0 2024-09-10T11:02:11.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.489+0000 7f9e63fff640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9e640a2d00 0x7f9e640a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.489+0000 7f9e7a59c640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9e640a2d00 0x7f9e640a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.489+0000 7f9e63fff640 1 -- 172.21.15.7:0/3830994735 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8564+0+0 (secure 0 0 0) 0x7f9e6c0ef8a0 con 0x7f9e7c1008e0 2024-09-10T11:02:11.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.490+0000 7f9e7a59c640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9e640a2d00 0x7f9e640a51c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c100210 tx=0x7f9e70009110 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:11.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.492+0000 7f9e63fff640 1 -- 172.21.15.7:0/3830994735 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e6c0bc490 con 0x7f9e7c1008e0 2024-09-10T11:02:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.668+0000 7f9e811fd640 1 -- 172.21.15.7:0/3830994735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e7c10a0c0 con 0x7f9e7c1008e0 2024-09-10T11:02:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.669+0000 7f9e63fff640 1 -- 172.21.15.7:0/3830994735 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) ==== 100+0+11 (secure 0 0 0) 0x7f9e6c0bbd90 con 0x7f9e7c1008e0 2024-09-10T11:02:11.670 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-10T11:02:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 -- 172.21.15.7:0/3830994735 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9e640a2d00 msgr2=0x7f9e640a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9e640a2d00 0x7f9e640a51c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c100210 tx=0x7f9e70009110 comp rx=0 tx=0).stop 2024-09-10T11:02:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 -- 172.21.15.7:0/3830994735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e7c1008e0 msgr2=0x7f9e7c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e7c1008e0 0x7f9e7c198550 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009230 tx=0x7f9e6c004270 comp rx=0 tx=0).stop 2024-09-10T11:02:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 -- 172.21.15.7:0/3830994735 shutdown_connections 2024-09-10T11:02:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7c108130 0x7f9e7c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e7c101200 0x7f9e7c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f9e640a2d00 0x7f9e640a51c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 --2- 172.21.15.7:0/3830994735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e7c1008e0 0x7f9e7c198550 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 -- 172.21.15.7:0/3830994735 >> 172.21.15.7:0/3830994735 conn(0x7f9e7c075ea0 msgr2=0x7f9e7c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 -- 172.21.15.7:0/3830994735 shutdown_connections 2024-09-10T11:02:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.671+0000 7f9e811fd640 1 -- 172.21.15.7:0/3830994735 wait complete. 2024-09-10T11:02:11.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T11:02:11.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.785+0000 7f34e9333640 1 -- 172.21.15.7:0/787900861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34e4102cf0 msgr2=0x7f34e41030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.785+0000 7f34e9333640 1 --2- 172.21.15.7:0/787900861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34e4102cf0 0x7f34e41030d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009900 tx=0x7f34d402a480 comp rx=0 tx=0).stop 2024-09-10T11:02:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.785+0000 7f34e9333640 1 -- 172.21.15.7:0/787900861 shutdown_connections 2024-09-10T11:02:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.785+0000 7f34e9333640 1 --2- 172.21.15.7:0/787900861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e410c0c0 0x7f34e410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.785+0000 7f34e9333640 1 --2- 172.21.15.7:0/787900861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e4103610 0x7f34e410baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.785+0000 7f34e9333640 1 --2- 172.21.15.7:0/787900861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34e4102cf0 0x7f34e41030d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.785+0000 7f34e9333640 1 -- 172.21.15.7:0/787900861 >> 172.21.15.7:0/787900861 conn(0x7f34e4077a10 msgr2=0x7f34e40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.785+0000 7f34e9333640 1 -- 172.21.15.7:0/787900861 shutdown_connections 2024-09-10T11:02:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.785+0000 7f34e9333640 1 -- 172.21.15.7:0/787900861 wait complete. 2024-09-10T11:02:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.785+0000 7f34e9333640 1 Processor -- start 2024-09-10T11:02:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e9333640 1 -- start start 2024-09-10T11:02:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e9333640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4102cf0 0x7f34e41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e9333640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34e4103610 0x7f34e4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e9333640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e410c0c0 0x7f34e419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e9333640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4111ee0 con 0x7f34e4103610 2024-09-10T11:02:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e9333640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4111d60 con 0x7f34e4102cf0 2024-09-10T11:02:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e9333640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4112060 con 0x7f34e410c0c0 2024-09-10T11:02:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34e4103610 0x7f34e4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34e4103610 0x7f34e4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53226/0 (socket says 172.21.15.7:53226) 2024-09-10T11:02:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e27fc640 1 -- 172.21.15.7:0/382346088 learned_addr learned my addr 172.21.15.7:0/382346088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e37fe640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e410c0c0 0x7f34e419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e2ffd640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4102cf0 0x7f34e41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e27fc640 1 -- 172.21.15.7:0/382346088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e410c0c0 msgr2=0x7f34e419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e27fc640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e410c0c0 0x7f34e419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e27fc640 1 -- 172.21.15.7:0/382346088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4102cf0 msgr2=0x7f34e41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e27fc640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4102cf0 0x7f34e41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e27fc640 1 -- 172.21.15.7:0/382346088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e419d3c0 con 0x7f34e4103610 2024-09-10T11:02:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.786+0000 7f34e2ffd640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4102cf0 0x7f34e41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:02:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.787+0000 7f34e27fc640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34e4103610 0x7f34e4198910 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f34d800e9f0 tx=0x7f34d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.787+0000 7f34cbfff640 1 -- 172.21.15.7:0/382346088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d800cd80 con 0x7f34e4103610 2024-09-10T11:02:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.787+0000 7f34cbfff640 1 -- 172.21.15.7:0/382346088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d8010910 con 0x7f34e4103610 2024-09-10T11:02:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.787+0000 7f34e9333640 1 -- 172.21.15.7:0/382346088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e419d6b0 con 0x7f34e4103610 2024-09-10T11:02:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.787+0000 7f34cbfff640 1 -- 172.21.15.7:0/382346088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8018990 con 0x7f34e4103610 2024-09-10T11:02:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.787+0000 7f34e9333640 1 -- 172.21.15.7:0/382346088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e41a4f40 con 0x7f34e4103610 2024-09-10T11:02:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.788+0000 7f34e9333640 1 -- 172.21.15.7:0/382346088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e41030d0 con 0x7f34e4103610 2024-09-10T11:02:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.790+0000 7f34cbfff640 1 -- 172.21.15.7:0/382346088 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f34d8010ac0 con 0x7f34e4103610 2024-09-10T11:02:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.791+0000 7f34cbfff640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f34cc0a2d00 0x7f34cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.791+0000 7f34cbfff640 1 -- 172.21.15.7:0/382346088 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8564+0+0 (secure 0 0 0) 0x7f34d80cfd30 con 0x7f34e4103610 2024-09-10T11:02:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.791+0000 7f34e2ffd640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f34cc0a2d00 0x7f34cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.791+0000 7f34e2ffd640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f34cc0a2d00 0x7f34cc0a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f34d4005ab0 tx=0x7f34d4000bc0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:11.794+0000 7f34cbfff640 1 -- 172.21.15.7:0/382346088 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34d809c8c0 con 0x7f34e4103610 2024-09-10T11:02:12.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.022+0000 7f34e9333640 1 -- 172.21.15.7:0/382346088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f34e4199720 con 0x7f34e4103610 2024-09-10T11:02:12.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.032+0000 7f34cbfff640 1 -- 172.21.15.7:0/382346088 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740888 (secure 0 0 0) 0x7f34e4199720 con 0x7f34e4103610 2024-09-10T11:02:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T11:02:12.043 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T11:02:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 25, 2024-09-10T11:02:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "active_gid": 15132, 2024-09-10T11:02:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "active_name": "x", 2024-09-10T11:02:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addrs": { 2024-09-10T11:02:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-10T11:02:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-10T11:02:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-10T11:02:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 354580929 2024-09-10T11:02:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-10T11:02:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-10T11:02:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 354580929 2024-09-10T11:02:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addr": "172.21.15.7:6801/354580929", 2024-09-10T11:02:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "active_change": "2024-09-10T10:45:53.785666+0000", 2024-09-10T11:02:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "active_mgr_features": 4540701547738038271, 2024-09-10T11:02:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "available": true, 2024-09-10T11:02:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "standbys": [ 2024-09-10T11:02:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24841, 2024-09-10T11:02:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "z", 2024-09-10T11:02:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-09-10T11:02:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-09-10T11:02:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-09-10T11:02:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T11:02:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-09-10T11:02:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-09-10T11:02:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-09-10T11:02:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-09-10T11:02:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-09-10T11:02:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-09-10T11:02:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-09-10T11:02:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-09-10T11:02:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-09-10T11:02:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-09-10T11:02:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-09-10T11:02:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-09-10T11:02:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-09-10T11:02:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-09-10T11:02:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-09-10T11:02:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-09-10T11:02:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-09-10T11:02:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-09-10T11:02:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-09-10T11:02:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-09-10T11:02:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-09-10T11:02:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-09-10T11:02:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T11:02:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-09-10T11:02:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-09-10T11:02:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-09-10T11:02:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-09-10T11:02:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-09-10T11:02:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-09-10T11:02:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T11:02:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-09-10T11:02:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-09-10T11:02:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-09-10T11:02:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T11:02:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T11:02:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-09-10T11:02:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-09-10T11:02:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-09-10T11:02:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-09-10T11:02:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T11:02:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T11:02:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-09-10T11:02:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-09-10T11:02:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-09-10T11:02:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-09-10T11:02:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T11:02:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-09-10T11:02:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-09-10T11:02:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T11:02:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T11:02:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T11:02:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-09-10T11:02:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-09-10T11:02:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-09-10T11:02:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-09-10T11:02:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-09-10T11:02:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-09-10T11:02:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T11:02:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-09-10T11:02:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-09-10T11:02:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-09-10T11:02:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T11:02:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T11:02:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-09-10T11:02:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-09-10T11:02:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-09-10T11:02:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-09-10T11:02:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T11:02:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T11:02:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-09-10T11:02:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-09-10T11:02:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T11:02:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-09-10T11:02:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-09-10T11:02:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-09-10T11:02:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-09-10T11:02:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-09-10T11:02:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-09-10T11:02:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-09-10T11:02:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-09-10T11:02:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-09-10T11:02:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-09-10T11:02:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-09-10T11:02:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T11:02:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T11:02:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-09-10T11:02:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-09-10T11:02:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T11:02:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T11:02:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-09-10T11:02:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-09-10T11:02:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T11:02:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-09-10T11:02:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-09-10T11:02:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-09-10T11:02:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-09-10T11:02:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T11:02:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-09-10T11:02:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-09-10T11:02:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-09-10T11:02:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T11:02:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-09-10T11:02:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-09-10T11:02:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-09-10T11:02:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T11:02:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-09-10T11:02:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-09-10T11:02:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T11:02:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T11:02:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-09-10T11:02:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-09-10T11:02:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T11:02:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-09-10T11:02:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T11:02:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-09-10T11:02:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T11:02:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-09-10T11:02:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-09-10T11:02:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-09-10T11:02:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-09-10T11:02:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-09-10T11:02:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T11:02:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-09-10T11:02:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-09-10T11:02:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T11:02:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-09-10T11:02:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-09-10T11:02:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T11:02:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-09-10T11:02:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-09-10T11:02:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T11:02:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-09-10T11:02:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-09-10T11:02:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T11:02:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-09-10T11:02:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-09-10T11:02:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T11:02:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-09-10T11:02:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-09-10T11:02:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-09-10T11:02:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T11:02:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-09-10T11:02:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-09-10T11:02:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-09-10T11:02:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T11:02:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-09-10T11:02:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-09-10T11:02:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T11:02:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-09-10T11:02:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-09-10T11:02:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-09-10T11:02:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T11:02:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-09-10T11:02:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-09-10T11:02:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-09-10T11:02:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T11:02:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-09-10T11:02:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-09-10T11:02:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-09-10T11:02:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T11:02:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-09-10T11:02:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-09-10T11:02:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-09-10T11:02:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T11:02:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-09-10T11:02:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-09-10T11:02:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-09-10T11:02:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T11:02:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-09-10T11:02:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-09-10T11:02:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-09-10T11:02:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T11:02:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-09-10T11:02:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-09-10T11:02:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T11:02:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-09-10T11:02:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-09-10T11:02:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-09-10T11:02:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T11:02:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-09-10T11:02:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-09-10T11:02:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T11:02:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-09-10T11:02:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-09-10T11:02:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-09-10T11:02:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T11:02:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-09-10T11:02:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-09-10T11:02:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-09-10T11:02:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T11:02:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-09-10T11:02:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-09-10T11:02:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-09-10T11:02:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T11:02:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-09-10T11:02:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-09-10T11:02:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T11:02:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-09-10T11:02:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T11:02:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-09-10T11:02:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T11:02:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-09-10T11:02:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-09-10T11:02:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-09-10T11:02:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T11:02:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-09-10T11:02:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-09-10T11:02:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-09-10T11:02:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-09-10T11:02:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-09-10T11:02:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-09-10T11:02:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T11:02:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-09-10T11:02:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-09-10T11:02:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-09-10T11:02:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-09-10T11:02:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-09-10T11:02:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T11:02:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T11:02:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-09-10T11:02:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-09-10T11:02:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-09-10T11:02:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-09-10T11:02:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-09-10T11:02:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T11:02:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-09-10T11:02:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-09-10T11:02:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T11:02:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-09-10T11:02:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-09-10T11:02:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T11:02:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T11:02:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T11:02:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T11:02:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T11:02:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T11:02:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T11:02:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-09-10T11:02:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T11:02:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-09-10T11:02:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-09-10T11:02:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-09-10T11:02:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-09-10T11:02:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-09-10T11:02:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T11:02:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-09-10T11:02:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-09-10T11:02:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-09-10T11:02:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-09-10T11:02:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-09-10T11:02:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-09-10T11:02:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-09-10T11:02:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-09-10T11:02:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T11:02:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-09-10T11:02:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-09-10T11:02:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-09-10T11:02:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T11:02:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-09-10T11:02:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-09-10T11:02:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T11:02:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T11:02:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-09-10T11:02:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-09-10T11:02:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T11:02:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-09-10T11:02:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-09-10T11:02:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T11:02:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-09-10T11:02:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-09-10T11:02:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T11:02:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-09-10T11:02:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-09-10T11:02:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T11:02:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-09-10T11:02:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-09-10T11:02:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T11:02:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-09-10T11:02:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-09-10T11:02:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-09-10T11:02:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-09-10T11:02:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-09-10T11:02:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-09-10T11:02:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T11:02:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-09-10T11:02:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-09-10T11:02:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T11:02:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-09-10T11:02:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-09-10T11:02:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T11:02:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-09-10T11:02:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-09-10T11:02:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T11:02:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-09-10T11:02:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-09-10T11:02:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T11:02:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-09-10T11:02:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-09-10T11:02:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T11:02:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-09-10T11:02:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-09-10T11:02:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-09-10T11:02:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-09-10T11:02:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T11:02:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-09-10T11:02:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-09-10T11:02:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-09-10T11:02:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T11:02:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-09-10T11:02:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T11:02:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T11:02:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T11:02:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T11:02:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T11:02:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T11:02:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-09-10T11:02:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T11:02:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T11:02:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T11:02:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T11:02:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T11:02:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T11:02:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T11:02:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T11:02:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T11:02:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T11:02:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T11:02:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T11:02:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T11:02:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T11:02:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T11:02:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T11:02:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T11:02:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T11:02:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T11:02:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T11:02:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T11:02:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T11:02:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T11:02:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-09-10T11:02:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T11:02:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T11:02:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-09-10T11:02:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-09-10T11:02:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T11:02:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T11:02:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-09-10T11:02:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T11:02:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T11:02:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T11:02:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T11:02:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T11:02:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T11:02:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T11:02:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T11:02:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T11:02:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T11:02:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T11:02:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T11:02:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T11:02:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T11:02:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T11:02:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T11:02:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T11:02:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T11:02:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T11:02:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T11:02:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T11:02:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T11:02:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T11:02:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T11:02:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T11:02:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T11:02:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T11:02:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T11:02:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T11:02:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T11:02:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T11:02:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T11:02:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T11:02:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T11:02:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T11:02:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-09-10T11:02:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T11:02:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T11:02:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-09-10T11:02:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T11:02:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T11:02:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T11:02:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T11:02:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-09-10T11:02:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T11:02:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T11:02:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T11:02:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T11:02:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T11:02:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T11:02:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T11:02:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T11:02:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T11:02:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T11:02:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-09-10T11:02:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-09-10T11:02:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-09-10T11:02:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-09-10T11:02:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-09-10T11:02:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-09-10T11:02:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-09-10T11:02:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-09-10T11:02:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-09-10T11:02:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-09-10T11:02:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-09-10T11:02:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-09-10T11:02:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-09-10T11:02:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-09-10T11:02:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-09-10T11:02:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-09-10T11:02:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T11:02:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-09-10T11:02:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-09-10T11:02:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-09-10T11:02:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-09-10T11:02:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-09-10T11:02:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-09-10T11:02:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-09-10T11:02:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-09-10T11:02:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-09-10T11:02:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-09-10T11:02:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-09-10T11:02:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-09-10T11:02:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-09-10T11:02:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-09-10T11:02:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-09-10T11:02:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-09-10T11:02:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-09-10T11:02:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-09-10T11:02:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-09-10T11:02:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-09-10T11:02:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-09-10T11:02:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-09-10T11:02:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-09-10T11:02:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-09-10T11:02:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-09-10T11:02:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-09-10T11:02:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-09-10T11:02:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-09-10T11:02:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-09-10T11:02:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T11:02:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-09-10T11:02:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-09-10T11:02:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-09-10T11:02:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T11:02:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-09-10T11:02:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-09-10T11:02:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-09-10T11:02:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-09-10T11:02:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-09-10T11:02:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-09-10T11:02:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-09-10T11:02:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T11:02:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-09-10T11:02:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-09-10T11:02:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T11:02:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T11:02:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-09-10T11:02:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-09-10T11:02:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-09-10T11:02:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T11:02:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-09-10T11:02:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-09-10T11:02:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-09-10T11:02:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-09-10T11:02:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-09-10T11:02:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-09-10T11:02:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-09-10T11:02:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-09-10T11:02:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-09-10T11:02:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-09-10T11:02:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-09-10T11:02:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-09-10T11:02:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-09-10T11:02:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T11:02:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-09-10T11:02:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-09-10T11:02:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-09-10T11:02:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T11:02:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-09-10T11:02:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-09-10T11:02:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-09-10T11:02:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-09-10T11:02:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-09-10T11:02:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T11:02:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-09-10T11:02:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-09-10T11:02:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-09-10T11:02:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-09-10T11:02:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-09-10T11:02:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-09-10T11:02:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-09-10T11:02:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-09-10T11:02:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-09-10T11:02:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-09-10T11:02:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T11:02:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-09-10T11:02:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-09-10T11:02:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-09-10T11:02:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T11:02:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-09-10T11:02:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-09-10T11:02:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-09-10T11:02:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-09-10T11:02:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-09-10T11:02:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-09-10T11:02:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T11:02:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-09-10T11:02:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-09-10T11:02:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-09-10T11:02:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-09-10T11:02:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-09-10T11:02:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-09-10T11:02:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T11:02:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-09-10T11:02:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-09-10T11:02:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-09-10T11:02:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T11:02:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-09-10T11:02:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-09-10T11:02:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-09-10T11:02:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-09-10T11:02:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-09-10T11:02:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T11:02:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-09-10T11:02:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-09-10T11:02:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-09-10T11:02:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T11:02:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-09-10T11:02:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-09-10T11:02:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-09-10T11:02:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-09-10T11:02:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-09-10T11:02:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-09-10T11:02:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-09-10T11:02:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T11:02:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-09-10T11:02:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-09-10T11:02:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-09-10T11:02:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-09-10T11:02:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-09-10T11:02:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-09-10T11:02:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T11:02:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T11:02:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-09-10T11:02:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.488 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.488 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.488 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.488 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.488 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.488 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-09-10T11:02:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-09-10T11:02:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-09-10T11:02:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-09-10T11:02:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-09-10T11:02:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T11:02:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-09-10T11:02:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-09-10T11:02:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-09-10T11:02:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-09-10T11:02:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-09-10T11:02:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-09-10T11:02:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-09-10T11:02:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-09-10T11:02:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-09-10T11:02:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-09-10T11:02:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-09-10T11:02:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-09-10T11:02:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-09-10T11:02:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-09-10T11:02:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T11:02:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-09-10T11:02:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-09-10T11:02:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-09-10T11:02:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T11:02:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-09-10T11:02:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-09-10T11:02:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-09-10T11:02:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-09-10T11:02:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-09-10T11:02:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-09-10T11:02:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-09-10T11:02:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-09-10T11:02:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-09-10T11:02:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-09-10T11:02:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-09-10T11:02:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-09-10T11:02:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T11:02:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T11:02:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-09-10T11:02:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-09-10T11:02:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T11:02:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T11:02:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-09-10T11:02:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-09-10T11:02:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-09-10T11:02:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-09-10T11:02:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-09-10T11:02:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-09-10T11:02:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-09-10T11:02:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T11:02:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-09-10T11:02:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-09-10T11:02:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-09-10T11:02:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T11:02:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-09-10T11:02:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-09-10T11:02:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-09-10T11:02:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-09-10T11:02:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-09-10T11:02:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-09-10T11:02:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-09-10T11:02:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-09-10T11:02:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-09-10T11:02:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-09-10T11:02:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-09-10T11:02:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-09-10T11:02:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-09-10T11:02:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-09-10T11:02:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-09-10T11:02:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-09-10T11:02:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-09-10T11:02:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T11:02:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-09-10T11:02:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-09-10T11:02:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-09-10T11:02:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-09-10T11:02:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-09-10T11:02:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-09-10T11:02:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-09-10T11:02:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-09-10T11:02:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-09-10T11:02:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-09-10T11:02:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-09-10T11:02:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-09-10T11:02:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-09-10T11:02:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-09-10T11:02:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-09-10T11:02:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-09-10T11:02:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T11:02:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-09-10T11:02:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-09-10T11:02:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-09-10T11:02:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-09-10T11:02:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-09-10T11:02:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-09-10T11:02:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-09-10T11:02:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-09-10T11:02:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-09-10T11:02:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-09-10T11:02:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-09-10T11:02:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-09-10T11:02:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-09-10T11:02:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-09-10T11:02:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-09-10T11:02:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-09-10T11:02:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-09-10T11:02:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-09-10T11:02:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-09-10T11:02:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-09-10T11:02:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-09-10T11:02:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-09-10T11:02:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-09-10T11:02:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-09-10T11:02:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-09-10T11:02:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-09-10T11:02:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-09-10T11:02:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-09-10T11:02:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-09-10T11:02:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-09-10T11:02:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T11:02:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-09-10T11:02:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-09-10T11:02:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-09-10T11:02:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-09-10T11:02:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-09-10T11:02:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-09-10T11:02:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-09-10T11:02:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-09-10T11:02:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T11:02:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-09-10T11:02:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-09-10T11:02:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-09-10T11:02:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-09-10T11:02:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T11:02:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-09-10T11:02:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-09-10T11:02:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T11:02:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-09-10T11:02:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-09-10T11:02:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T11:02:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-09-10T11:02:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-09-10T11:02:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T11:02:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-09-10T11:02:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-09-10T11:02:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T11:02:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-09-10T11:02:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-09-10T11:02:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-09-10T11:02:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-09-10T11:02:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-09-10T11:02:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-09-10T11:02:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T11:02:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-09-10T11:02:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-09-10T11:02:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-09-10T11:02:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-09-10T11:02:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-09-10T11:02:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-09-10T11:02:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-09-10T11:02:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-09-10T11:02:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-09-10T11:02:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-09-10T11:02:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-09-10T11:02:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-09-10T11:02:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-09-10T11:02:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-09-10T11:02:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-09-10T11:02:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-09-10T11:02:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-09-10T11:02:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T11:02:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-09-10T11:02:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-09-10T11:02:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-09-10T11:02:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-09-10T11:02:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-09-10T11:02:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T11:02:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-09-10T11:02:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-09-10T11:02:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-09-10T11:02:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-09-10T11:02:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-09-10T11:02:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T11:02:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-09-10T11:02:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T11:02:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T11:02:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-09-10T11:02:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-09-10T11:02:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-09-10T11:02:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-09-10T11:02:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-09-10T11:02:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-09-10T11:02:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-09-10T11:02:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-09-10T11:02:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-09-10T11:02:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-09-10T11:02:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-09-10T11:02:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-09-10T11:02:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-09-10T11:02:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T11:02:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24842, 2024-09-10T11:02:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "y", 2024-09-10T11:02:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-09-10T11:02:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-09-10T11:02:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-09-10T11:02:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T11:02:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-09-10T11:02:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-09-10T11:02:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-09-10T11:02:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-09-10T11:02:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-09-10T11:02:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-09-10T11:02:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-09-10T11:02:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-09-10T11:02:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-09-10T11:02:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-09-10T11:02:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-09-10T11:02:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-09-10T11:02:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-09-10T11:02:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-09-10T11:02:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-09-10T11:02:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-09-10T11:02:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-09-10T11:02:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-09-10T11:02:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-09-10T11:02:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-09-10T11:02:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-09-10T11:02:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-09-10T11:02:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T11:02:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-09-10T11:02:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-09-10T11:02:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-09-10T11:02:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-09-10T11:02:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-09-10T11:02:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-09-10T11:02:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T11:02:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-09-10T11:02:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-09-10T11:02:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-09-10T11:02:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T11:02:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T11:02:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-09-10T11:02:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-09-10T11:02:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-09-10T11:02:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-09-10T11:02:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T11:02:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T11:02:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-09-10T11:02:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-09-10T11:02:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-09-10T11:02:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-09-10T11:02:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T11:02:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-09-10T11:02:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-09-10T11:02:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T11:02:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T11:02:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T11:02:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-09-10T11:02:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-09-10T11:02:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-09-10T11:02:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-09-10T11:02:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-09-10T11:02:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-09-10T11:02:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T11:02:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-09-10T11:02:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-09-10T11:02:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-09-10T11:02:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T11:02:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T11:02:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-09-10T11:02:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-09-10T11:02:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-09-10T11:02:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-09-10T11:02:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T11:02:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T11:02:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-09-10T11:02:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-09-10T11:02:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T11:02:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-09-10T11:02:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-09-10T11:02:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-09-10T11:02:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-09-10T11:02:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-09-10T11:02:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-09-10T11:02:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-09-10T11:02:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-09-10T11:02:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-09-10T11:02:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-09-10T11:02:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-09-10T11:02:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T11:02:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T11:02:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-09-10T11:02:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-09-10T11:02:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T11:02:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T11:02:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-09-10T11:02:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-09-10T11:02:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T11:02:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-09-10T11:02:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-09-10T11:02:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-09-10T11:02:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-09-10T11:02:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T11:02:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-09-10T11:02:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-09-10T11:02:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-09-10T11:02:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T11:02:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-09-10T11:02:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-09-10T11:02:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-09-10T11:02:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T11:02:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-09-10T11:02:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-09-10T11:02:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T11:02:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T11:02:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-09-10T11:02:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-09-10T11:02:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T11:02:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-09-10T11:02:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T11:02:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-09-10T11:02:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T11:02:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-09-10T11:02:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-09-10T11:02:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-09-10T11:02:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-09-10T11:02:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-09-10T11:02:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T11:02:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-09-10T11:02:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-09-10T11:02:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T11:02:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-09-10T11:02:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-09-10T11:02:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T11:02:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-09-10T11:02:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-09-10T11:02:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T11:02:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-09-10T11:02:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-09-10T11:02:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T11:02:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-09-10T11:02:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-09-10T11:02:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T11:02:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-09-10T11:02:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-09-10T11:02:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-09-10T11:02:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T11:02:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-09-10T11:02:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-09-10T11:02:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-09-10T11:02:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T11:02:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-09-10T11:02:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-09-10T11:02:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T11:02:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-09-10T11:02:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-09-10T11:02:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-09-10T11:02:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T11:02:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-09-10T11:02:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-09-10T11:02:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-09-10T11:02:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T11:02:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-09-10T11:02:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-09-10T11:02:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-09-10T11:02:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T11:02:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-09-10T11:02:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-09-10T11:02:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-09-10T11:02:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T11:02:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-09-10T11:02:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-09-10T11:02:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-09-10T11:02:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T11:02:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-09-10T11:02:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-09-10T11:02:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-09-10T11:02:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T11:02:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-09-10T11:02:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-09-10T11:02:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T11:02:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-09-10T11:02:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-09-10T11:02:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-09-10T11:02:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T11:02:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-09-10T11:02:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-09-10T11:02:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T11:02:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-09-10T11:02:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-09-10T11:02:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-09-10T11:02:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T11:02:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-09-10T11:02:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-09-10T11:02:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-09-10T11:02:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T11:02:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-09-10T11:02:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-09-10T11:02:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-09-10T11:02:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T11:02:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-09-10T11:02:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-09-10T11:02:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T11:02:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-09-10T11:02:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T11:02:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-09-10T11:02:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T11:02:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-09-10T11:02:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-09-10T11:02:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-09-10T11:02:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T11:02:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-09-10T11:02:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-09-10T11:02:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-09-10T11:02:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-09-10T11:02:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-09-10T11:02:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-09-10T11:02:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T11:02:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-09-10T11:02:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-09-10T11:02:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-09-10T11:02:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-09-10T11:02:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-09-10T11:02:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T11:02:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T11:02:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-09-10T11:02:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-09-10T11:02:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-09-10T11:02:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-09-10T11:02:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-09-10T11:02:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T11:02:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-09-10T11:02:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-09-10T11:02:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T11:02:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-09-10T11:02:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-09-10T11:02:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T11:02:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T11:02:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T11:02:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T11:02:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T11:02:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T11:02:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T11:02:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-09-10T11:02:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T11:02:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-09-10T11:02:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-09-10T11:02:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-09-10T11:02:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-09-10T11:02:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-09-10T11:02:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T11:02:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-09-10T11:02:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-09-10T11:02:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-09-10T11:02:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-09-10T11:02:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-09-10T11:02:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-09-10T11:02:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-09-10T11:02:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-09-10T11:02:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T11:02:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-09-10T11:02:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-09-10T11:02:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-09-10T11:02:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T11:02:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-09-10T11:02:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-09-10T11:02:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T11:02:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T11:02:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-09-10T11:02:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-09-10T11:02:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T11:02:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-09-10T11:02:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-09-10T11:02:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T11:02:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-09-10T11:02:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-09-10T11:02:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T11:02:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-09-10T11:02:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-09-10T11:02:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T11:02:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-09-10T11:02:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-09-10T11:02:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T11:02:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-09-10T11:02:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-09-10T11:02:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-09-10T11:02:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-09-10T11:02:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-09-10T11:02:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-09-10T11:02:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T11:02:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-09-10T11:02:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-09-10T11:02:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T11:02:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-09-10T11:02:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-09-10T11:02:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T11:02:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-09-10T11:02:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-09-10T11:02:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T11:02:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-09-10T11:02:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-09-10T11:02:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T11:02:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-09-10T11:02:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-09-10T11:02:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T11:02:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-09-10T11:02:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-09-10T11:02:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-09-10T11:02:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-09-10T11:02:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T11:02:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-09-10T11:02:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-09-10T11:02:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-09-10T11:02:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T11:02:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-09-10T11:02:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T11:02:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T11:02:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T11:02:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T11:02:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T11:02:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T11:02:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-09-10T11:02:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T11:02:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T11:02:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T11:02:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T11:02:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T11:02:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T11:02:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T11:02:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T11:02:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T11:02:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T11:02:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T11:02:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T11:02:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T11:02:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T11:02:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T11:02:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T11:02:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T11:02:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T11:02:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T11:02:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T11:02:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T11:02:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T11:02:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T11:02:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-09-10T11:02:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T11:02:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T11:02:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-09-10T11:02:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-09-10T11:02:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T11:02:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T11:02:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-09-10T11:02:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T11:02:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T11:02:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T11:02:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T11:02:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T11:02:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T11:02:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T11:02:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T11:02:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T11:02:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T11:02:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T11:02:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T11:02:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T11:02:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T11:02:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T11:02:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T11:02:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T11:02:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T11:02:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T11:02:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T11:02:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T11:02:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T11:02:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T11:02:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T11:02:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T11:02:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T11:02:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T11:02:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T11:02:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T11:02:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T11:02:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T11:02:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T11:02:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T11:02:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T11:02:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T11:02:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-09-10T11:02:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T11:02:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T11:02:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-09-10T11:02:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T11:02:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T11:02:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.100 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.100 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T11:02:13.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T11:02:13.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.101 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-09-10T11:02:13.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.101 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.101 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T11:02:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T11:02:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T11:02:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T11:02:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T11:02:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T11:02:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T11:02:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T11:02:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T11:02:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T11:02:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-09-10T11:02:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-09-10T11:02:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-09-10T11:02:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-09-10T11:02:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-09-10T11:02:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-09-10T11:02:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-09-10T11:02:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-09-10T11:02:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-09-10T11:02:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-09-10T11:02:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-09-10T11:02:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-09-10T11:02:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-09-10T11:02:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-09-10T11:02:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-09-10T11:02:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-09-10T11:02:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T11:02:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-09-10T11:02:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-09-10T11:02:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-09-10T11:02:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-09-10T11:02:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-09-10T11:02:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-09-10T11:02:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-09-10T11:02:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-09-10T11:02:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-09-10T11:02:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-09-10T11:02:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-09-10T11:02:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-09-10T11:02:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-09-10T11:02:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-09-10T11:02:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-09-10T11:02:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-09-10T11:02:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-09-10T11:02:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-09-10T11:02:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-09-10T11:02:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-09-10T11:02:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-09-10T11:02:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-09-10T11:02:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-09-10T11:02:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-09-10T11:02:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-09-10T11:02:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-09-10T11:02:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-09-10T11:02:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-09-10T11:02:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-09-10T11:02:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T11:02:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-09-10T11:02:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-09-10T11:02:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-09-10T11:02:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T11:02:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-09-10T11:02:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-09-10T11:02:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-09-10T11:02:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-09-10T11:02:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-09-10T11:02:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-09-10T11:02:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-09-10T11:02:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T11:02:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-09-10T11:02:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-09-10T11:02:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T11:02:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T11:02:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-09-10T11:02:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-09-10T11:02:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-09-10T11:02:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T11:02:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-09-10T11:02:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-09-10T11:02:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-09-10T11:02:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-09-10T11:02:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-09-10T11:02:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-09-10T11:02:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-09-10T11:02:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-09-10T11:02:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-09-10T11:02:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-09-10T11:02:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-09-10T11:02:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-09-10T11:02:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-09-10T11:02:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T11:02:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-09-10T11:02:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-09-10T11:02:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-09-10T11:02:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T11:02:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-09-10T11:02:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-09-10T11:02:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-09-10T11:02:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-09-10T11:02:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-09-10T11:02:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T11:02:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-09-10T11:02:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-09-10T11:02:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-09-10T11:02:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-09-10T11:02:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-09-10T11:02:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-09-10T11:02:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-09-10T11:02:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-09-10T11:02:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-09-10T11:02:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-09-10T11:02:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T11:02:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-09-10T11:02:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-09-10T11:02:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-09-10T11:02:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T11:02:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-09-10T11:02:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-09-10T11:02:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-09-10T11:02:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-09-10T11:02:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-09-10T11:02:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-09-10T11:02:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T11:02:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-09-10T11:02:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-09-10T11:02:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-09-10T11:02:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-09-10T11:02:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-09-10T11:02:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-09-10T11:02:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T11:02:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-09-10T11:02:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-09-10T11:02:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-09-10T11:02:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T11:02:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-09-10T11:02:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-09-10T11:02:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-09-10T11:02:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-09-10T11:02:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-09-10T11:02:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T11:02:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-09-10T11:02:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-09-10T11:02:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-09-10T11:02:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T11:02:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-09-10T11:02:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-09-10T11:02:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-09-10T11:02:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-09-10T11:02:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-09-10T11:02:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-09-10T11:02:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-09-10T11:02:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T11:02:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-09-10T11:02:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-09-10T11:02:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-09-10T11:02:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-09-10T11:02:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-09-10T11:02:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-09-10T11:02:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T11:02:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T11:02:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-09-10T11:02:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-09-10T11:02:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-09-10T11:02:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-09-10T11:02:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-09-10T11:02:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-09-10T11:02:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T11:02:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-09-10T11:02:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-09-10T11:02:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-09-10T11:02:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-09-10T11:02:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-09-10T11:02:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-09-10T11:02:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-09-10T11:02:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-09-10T11:02:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-09-10T11:02:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-09-10T11:02:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-09-10T11:02:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-09-10T11:02:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-09-10T11:02:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-09-10T11:02:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T11:02:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-09-10T11:02:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-09-10T11:02:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-09-10T11:02:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T11:02:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-09-10T11:02:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-09-10T11:02:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.362 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.362 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.362 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.362 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-09-10T11:02:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-09-10T11:02:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-09-10T11:02:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.363 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-09-10T11:02:13.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.364 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:13.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:13.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.365 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:13.365 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.365 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.365 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-09-10T11:02:13.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.366 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-09-10T11:02:13.367 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.367 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.367 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.367 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-09-10T11:02:13.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-09-10T11:02:13.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.368 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.368 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.368 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.369 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.369 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.369 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-09-10T11:02:13.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-09-10T11:02:13.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.370 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.370 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.370 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.370 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.370 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T11:02:13.370 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T11:02:13.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.371 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.371 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.372 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.372 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.372 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.372 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.373 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.373 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.373 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.374 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.374 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.374 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.374 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.376 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.376 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.377 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.377 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.377 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.377 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.377 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.377 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.377 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.378 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.378 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.378 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.378 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.378 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.379 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.379 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.379 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.379 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.379 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.379 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.380 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-09-10T11:02:13.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-09-10T11:02:13.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.381 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.381 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.382 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.382 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T11:02:13.382 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T11:02:13.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.382 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-09-10T11:02:13.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.383 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.383 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.383 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.383 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.384 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-09-10T11:02:13.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-09-10T11:02:13.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:13.384 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-09-10T11:02:13.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.385 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.385 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.386 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-09-10T11:02:13.386 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-09-10T11:02:13.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.386 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.386 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-09-10T11:02:13.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T11:02:13.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.388 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-09-10T11:02:13.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-09-10T11:02:13.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-09-10T11:02:13.388 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.388 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.389 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.389 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T11:02:13.389 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.389 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.390 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-09-10T11:02:13.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-09-10T11:02:13.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.390 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.390 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-09-10T11:02:13.390 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.390 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.391 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.391 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-09-10T11:02:13.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-09-10T11:02:13.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.392 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-09-10T11:02:13.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.393 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-09-10T11:02:13.393 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-09-10T11:02:13.393 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.393 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.393 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.393 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.393 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.394 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-09-10T11:02:13.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-09-10T11:02:13.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.394 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-09-10T11:02:13.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-09-10T11:02:13.395 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-09-10T11:02:13.395 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.395 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.395 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.395 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-09-10T11:02:13.396 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.396 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.396 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.396 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.397 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.397 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.398 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.398 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.398 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.398 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.398 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.398 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.398 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.398 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.399 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.399 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.399 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.399 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.399 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.400 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.400 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.400 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.400 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.400 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.401 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.401 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.402 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.402 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.402 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.403 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.403 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.403 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.403 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.403 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.403 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.403 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.404 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.404 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.404 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.406 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-09-10T11:02:13.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-09-10T11:02:13.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.406 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.407 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.407 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-09-10T11:02:13.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T11:02:13.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.408 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.408 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.409 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.409 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-09-10T11:02:13.409 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-09-10T11:02:13.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.410 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.410 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.410 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.410 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.410 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.410 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.411 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.411 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-09-10T11:02:13.411 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.412 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.412 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-09-10T11:02:13.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-09-10T11:02:13.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.413 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.413 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.413 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-09-10T11:02:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-09-10T11:02:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.415 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.415 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.416 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.416 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.416 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.416 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.419 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.419 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.420 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.420 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.421 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.421 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.421 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.422 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.422 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.422 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.422 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.422 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.422 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.423 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.423 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.423 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.423 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.424 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.424 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.424 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.424 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.424 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.424 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.424 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.425 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-09-10T11:02:13.425 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-09-10T11:02:13.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.425 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.426 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.426 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.426 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.426 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.426 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.427 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-09-10T11:02:13.427 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-09-10T11:02:13.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.427 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.427 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.428 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.428 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.428 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.428 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.428 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.428 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-09-10T11:02:13.429 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.429 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.429 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.430 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.430 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.430 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.431 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.431 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.431 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.431 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.431 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.431 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.431 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.431 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.432 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.432 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.432 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.432 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.432 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.433 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.433 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.433 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.433 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.434 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.434 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.434 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.434 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.434 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.434 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.435 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.435 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.435 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.435 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.435 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.436 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.436 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.436 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.436 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.436 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.436 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.436 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.437 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.437 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.437 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.437 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.437 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.438 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.438 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.438 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.438 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.438 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.438 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.438 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.439 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.439 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-09-10T11:02:13.440 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.440 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.440 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.440 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.440 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.440 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.441 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.441 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.441 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.441 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.441 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.441 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.441 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.441 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.442 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.442 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.442 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.442 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.442 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.442 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.443 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.444 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.444 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.444 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.444 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.444 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.444 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.445 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.445 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.445 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.445 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.446 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.446 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.446 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.446 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.446 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.446 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.446 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.448 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.448 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.449 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.449 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.449 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-09-10T11:02:13.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-09-10T11:02:13.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.449 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.450 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-09-10T11:02:13.450 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.450 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.450 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.450 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T11:02:13.450 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.451 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.451 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-09-10T11:02:13.451 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.452 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.452 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.452 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.452 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.452 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.453 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.453 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.453 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.453 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.453 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.453 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.453 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.454 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.454 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.454 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.454 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.455 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.455 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.455 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.456 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.456 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.456 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.456 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.456 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.456 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.457 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.457 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.457 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.457 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.457 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.457 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.458 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.458 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.458 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.458 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.458 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.458 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.458 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.459 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.459 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.459 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.460 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.460 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.460 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.460 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.460 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-09-10T11:02:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-09-10T11:02:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.462 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.462 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.462 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.462 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.463 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-09-10T11:02:13.463 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-09-10T11:02:13.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.463 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-09-10T11:02:13.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.463 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.464 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.464 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.464 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.464 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-09-10T11:02:13.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-09-10T11:02:13.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.465 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.465 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.466 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-09-10T11:02:13.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-09-10T11:02:13.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.466 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.467 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.467 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.467 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.467 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.467 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.467 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.468 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-09-10T11:02:13.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-09-10T11:02:13.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:13.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.469 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.469 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.469 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.469 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.469 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.470 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.470 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-09-10T11:02:13.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-09-10T11:02:13.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.470 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.471 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.471 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.471 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.471 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.471 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.471 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-09-10T11:02:13.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-09-10T11:02:13.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.472 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.472 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.472 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.473 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.473 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.473 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-09-10T11:02:13.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-09-10T11:02:13.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.474 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.474 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.474 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.475 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.475 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.475 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-09-10T11:02:13.475 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-09-10T11:02:13.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.476 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.476 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.476 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:13.476 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-09-10T11:02:13.476 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.476 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.476 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.477 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.477 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-09-10T11:02:13.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-09-10T11:02:13.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.477 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.477 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.478 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.478 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.478 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.478 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.478 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.479 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.479 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-09-10T11:02:13.479 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-09-10T11:02:13.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.479 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.480 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.480 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.480 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.480 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.480 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.480 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.480 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.481 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-09-10T11:02:13.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-09-10T11:02:13.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.481 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.481 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.482 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.482 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.482 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.482 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.482 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.482 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-09-10T11:02:13.483 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.483 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.483 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.484 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-09-10T11:02:13.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-09-10T11:02:13.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.484 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.485 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.485 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T11:02:13.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.485 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-09-10T11:02:13.486 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-09-10T11:02:13.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.486 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.486 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.487 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-09-10T11:02:13.487 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.487 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.487 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.488 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.488 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.488 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.488 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.488 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.488 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.489 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.489 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.489 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.489 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.489 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.489 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.489 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.490 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.490 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.490 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.490 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.491 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.491 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.491 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.492 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.492 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.492 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.493 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.493 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.493 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.493 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.493 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.493 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.493 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.494 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.494 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.494 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.494 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.494 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.494 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.495 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.495 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.495 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.495 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.495 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.495 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.496 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.496 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-09-10T11:02:13.497 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.497 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.497 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.497 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.498 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.498 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.498 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.498 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.498 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.498 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.499 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.499 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.499 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.499 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.499 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.499 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.499 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.500 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.500 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.500 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.500 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.501 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.501 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.502 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.502 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.502 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.503 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.503 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.503 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.503 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.504 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.504 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.504 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.504 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.504 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.504 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.504 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.505 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.505 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.505 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.505 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.506 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.506 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.506 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.507 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.508 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.508 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-09-10T11:02:13.509 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.509 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.509 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.509 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.510 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.510 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.510 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.510 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.510 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.511 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.511 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.511 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.511 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.511 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.511 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.511 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.511 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.512 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.512 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.513 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.513 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.513 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.514 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.514 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.514 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.515 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.515 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.515 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.515 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.515 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.515 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.515 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.515 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.516 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.516 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.516 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.516 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.516 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.517 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.517 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.517 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.517 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.517 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.518 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.518 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-09-10T11:02:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-09-10T11:02:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-09-10T11:02:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T11:02:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-09-10T11:02:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.526 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.526 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.526 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.526 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.526 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.526 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.527 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.527 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.527 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.528 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.528 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.528 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.528 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.528 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.528 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.529 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.529 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.529 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.529 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.529 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.529 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.529 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.530 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.530 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.530 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.530 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.531 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.531 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.532 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-09-10T11:02:13.532 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.532 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.532 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.532 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-09-10T11:02:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-09-10T11:02:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.534 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T11:02:13.534 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.534 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.534 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.534 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-09-10T11:02:13.534 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-09-10T11:02:13.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.535 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.535 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.535 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.535 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.535 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T11:02:13.536 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.536 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.536 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-09-10T11:02:13.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-09-10T11:02:13.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.536 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.537 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.537 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.537 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.537 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.537 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T11:02:13.537 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.538 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.538 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-09-10T11:02:13.538 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-09-10T11:02:13.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.538 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.539 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.539 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.539 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.539 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T11:02:13.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.539 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.540 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-09-10T11:02:13.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-09-10T11:02:13.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.540 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.541 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.541 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.541 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T11:02:13.541 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.541 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.541 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-09-10T11:02:13.541 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-09-10T11:02:13.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.542 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.542 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.542 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.542 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.542 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.542 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.542 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.543 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.543 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.543 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-09-10T11:02:13.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-09-10T11:02:13.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.544 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.544 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.544 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.545 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.545 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-09-10T11:02:13.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-09-10T11:02:13.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.545 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.545 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T11:02:13.546 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.546 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.546 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.546 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.546 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.546 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-09-10T11:02:13.547 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-09-10T11:02:13.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.547 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.547 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.547 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.547 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.547 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.548 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.548 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.548 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.548 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:13.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:13.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.549 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.549 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-09-10T11:02:13.549 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-09-10T11:02:13.549 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.549 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.549 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.549 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-09-10T11:02:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-09-10T11:02:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.551 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-09-10T11:02:13.551 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.551 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.551 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.551 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.551 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.551 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.552 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-09-10T11:02:13.552 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-09-10T11:02:13.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.552 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.552 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.552 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.553 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.553 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.553 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.553 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.553 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-09-10T11:02:13.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-09-10T11:02:13.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.554 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.554 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.554 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.554 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.555 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.555 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.555 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.555 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.556 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.556 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.556 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.556 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.556 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.557 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.557 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.557 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.557 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.557 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.557 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.557 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.558 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.558 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.558 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.558 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.558 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.559 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.559 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.559 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.559 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.559 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.559 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.559 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.560 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.560 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.560 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.560 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.560 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.561 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.561 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.561 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.561 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.561 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.561 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.561 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.562 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.562 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.562 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.562 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.562 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.562 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.563 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.563 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.563 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.563 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.563 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.563 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.564 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.564 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.564 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.564 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-09-10T11:02:13.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-09-10T11:02:13.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.565 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.565 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.565 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.565 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.565 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.565 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.566 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.566 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-09-10T11:02:13.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-09-10T11:02:13.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.567 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.567 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.567 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.567 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.567 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.568 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-09-10T11:02:13.568 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-09-10T11:02:13.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.568 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.568 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.568 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T11:02:13.569 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.569 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.569 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.569 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.569 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.569 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.569 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-09-10T11:02:13.570 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.570 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.570 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.571 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.571 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.571 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.571 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.572 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.572 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.572 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.572 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.572 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.572 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.572 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.572 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.572 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.573 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.573 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.573 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.573 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.577 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.577 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.577 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.577 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.577 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.578 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.578 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.578 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.578 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.578 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.579 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.579 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-09-10T11:02:13.580 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.580 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.580 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.580 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.581 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.581 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.581 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.581 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.581 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.581 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.582 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.582 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.582 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.582 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.582 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.582 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.582 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.583 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.583 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.583 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.584 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.584 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.584 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.584 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.584 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.585 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.585 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.585 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.585 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.585 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.586 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.586 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.586 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.586 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.586 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.586 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.586 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.587 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.587 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.587 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.587 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.587 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.588 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.588 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.588 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.588 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.588 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.588 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.589 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.589 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.589 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.589 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-09-10T11:02:13.589 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-09-10T11:02:13.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.590 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-09-10T11:02:13.590 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.590 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.590 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T11:02:13.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.591 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.591 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-09-10T11:02:13.591 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-09-10T11:02:13.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.591 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.591 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.592 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.592 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.592 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.592 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-09-10T11:02:13.592 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.592 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.593 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-09-10T11:02:13.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-09-10T11:02:13.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.593 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.593 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:13.593 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.594 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.594 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T11:02:13.594 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.594 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.595 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-09-10T11:02:13.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T11:02:13.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.595 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.595 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.595 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.595 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.596 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T11:02:13.596 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.596 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.596 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.596 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-09-10T11:02:13.597 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.597 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.597 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.597 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-09-10T11:02:13.598 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-09-10T11:02:13.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:13.598 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.598 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-09-10T11:02:13.598 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.598 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.598 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.599 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.599 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.599 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-09-10T11:02:13.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-09-10T11:02:13.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.600 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.600 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.600 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.600 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.600 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.600 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.600 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.601 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:13.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:13.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.601 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.601 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:13.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.602 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.602 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.602 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.602 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.602 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.602 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.603 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.603 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.603 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.603 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.604 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.604 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.604 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.604 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.604 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.604 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.604 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.605 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.605 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.605 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.605 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.606 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.606 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.606 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.606 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.606 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.606 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.607 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.607 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.607 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.607 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.608 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.608 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.608 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.608 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.608 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.608 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.608 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.609 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.609 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.609 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.609 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.610 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.610 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.610 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.610 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.610 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.610 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.611 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.611 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-09-10T11:02:13.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-09-10T11:02:13.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.612 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.612 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.612 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.612 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.612 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.613 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.613 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-09-10T11:02:13.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-09-10T11:02:13.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.613 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.614 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-09-10T11:02:13.614 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.614 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.614 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.614 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.614 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.614 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.615 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-09-10T11:02:13.615 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-09-10T11:02:13.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T11:02:13.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.616 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.616 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.616 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.616 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.616 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.617 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:13.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.617 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.617 INFO:teuthology.orchestra.run.smithi007.stdout: "modules": [ 2024-09-10T11:02:13.617 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-09-10T11:02:13.617 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard", 2024-09-10T11:02:13.617 INFO:teuthology.orchestra.run.smithi007.stdout: "iostat", 2024-09-10T11:02:13.617 INFO:teuthology.orchestra.run.smithi007.stdout: "nfs", 2024-09-10T11:02:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: "restful", 2024-09-10T11:02:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_schedule" 2024-09-10T11:02:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-09-10T11:02:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-09-10T11:02:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.618 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.619 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.619 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:13.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:13.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.619 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.619 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:13.619 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.620 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.620 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.620 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T11:02:13.620 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.620 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.620 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.621 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.621 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.621 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.621 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.621 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.621 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.622 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.622 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.622 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.622 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.622 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.622 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.622 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.623 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.623 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.623 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.623 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.624 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.624 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.624 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.624 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.624 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.625 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.625 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.625 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.625 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.625 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.626 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.626 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.626 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.626 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.626 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.626 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.626 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.627 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.628 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.629 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.629 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.629 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.629 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.629 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-09-10T11:02:13.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-09-10T11:02:13.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.630 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.630 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.630 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.630 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.631 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.631 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-09-10T11:02:13.631 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.631 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.631 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-09-10T11:02:13.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-09-10T11:02:13.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.632 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.632 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-09-10T11:02:13.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.632 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.632 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-09-10T11:02:13.633 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.633 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.633 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-09-10T11:02:13.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-09-10T11:02:13.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.633 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.634 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.634 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.634 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.634 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.634 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-09-10T11:02:13.634 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.634 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.635 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-09-10T11:02:13.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-09-10T11:02:13.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.635 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.635 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.635 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.636 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.636 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.636 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-09-10T11:02:13.636 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.636 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.636 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-09-10T11:02:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-09-10T11:02:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-09-10T11:02:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-09-10T11:02:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-09-10T11:02:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-09-10T11:02:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.639 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.639 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.639 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.639 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.639 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.639 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-09-10T11:02:13.640 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.640 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.640 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-09-10T11:02:13.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-09-10T11:02:13.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.640 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.640 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.641 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.641 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.641 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.641 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.641 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T11:02:13.641 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.642 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.642 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-09-10T11:02:13.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-09-10T11:02:13.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.642 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.642 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.643 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.643 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.643 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-09-10T11:02:13.643 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.643 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-09-10T11:02:13.644 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.644 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.644 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.644 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-09-10T11:02:13.645 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-09-10T11:02:13.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.645 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.645 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.645 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.646 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T11:02:13.646 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.646 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.646 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-09-10T11:02:13.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-09-10T11:02:13.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.647 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.647 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-09-10T11:02:13.647 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.647 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.647 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.647 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T11:02:13.647 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T11:02:13.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.648 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.648 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-09-10T11:02:13.648 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-09-10T11:02:13.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:13.648 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.648 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:13.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-09-10T11:02:13.649 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-09-10T11:02:13.649 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.649 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T11:02:13.649 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T11:02:13.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.649 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.650 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-09-10T11:02:13.650 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-09-10T11:02:13.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:13.650 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.650 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-09-10T11:02:13.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:13.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-09-10T11:02:13.651 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.651 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T11:02:13.651 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.651 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.651 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-09-10T11:02:13.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-09-10T11:02:13.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.652 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.652 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T11:02:13.652 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.652 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.652 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T11:02:13.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T11:02:13.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.653 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-09-10T11:02:13.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-09-10T11:02:13.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:13.654 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-09-10T11:02:13.654 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-09-10T11:02:13.654 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-09-10T11:02:13.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.654 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-09-10T11:02:13.654 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T11:02:13.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.655 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.655 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-09-10T11:02:13.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-09-10T11:02:13.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.655 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.656 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-09-10T11:02:13.656 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.656 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.656 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.656 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T11:02:13.656 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T11:02:13.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.656 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.657 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-09-10T11:02:13.657 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-09-10T11:02:13.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:13.657 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.657 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:13.657 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-09-10T11:02:13.658 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-09-10T11:02:13.658 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.658 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T11:02:13.658 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T11:02:13.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.658 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.658 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.659 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.659 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.659 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.659 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.659 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.660 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.660 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.660 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.660 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.660 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.660 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.660 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.660 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.661 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.661 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.661 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.661 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.662 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.662 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.662 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.662 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.662 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.662 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.663 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.663 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.663 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.664 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.664 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.664 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.664 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.664 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.665 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.665 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.665 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.665 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.665 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.665 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.665 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.666 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.666 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.666 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.666 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.666 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.667 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.667 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.667 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.667 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.667 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.667 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.668 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-09-10T11:02:13.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-09-10T11:02:13.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:13.668 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.668 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:13.668 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.669 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.669 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T11:02:13.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.669 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.669 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-09-10T11:02:13.670 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-09-10T11:02:13.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.670 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-09-10T11:02:13.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.670 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.670 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.670 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-09-10T11:02:13.671 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-09-10T11:02:13.671 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-09-10T11:02:13.671 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-09-10T11:02:13.671 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-09-10T11:02:13.671 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.671 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-09-10T11:02:13.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.672 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.672 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-09-10T11:02:13.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-09-10T11:02:13.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.672 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.672 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.672 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.673 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.673 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.673 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T11:02:13.673 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.673 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.673 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:13.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:13.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.674 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.674 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:13.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.674 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.674 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.675 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T11:02:13.675 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.675 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.675 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-09-10T11:02:13.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-09-10T11:02:13.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.676 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:13.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:13.676 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.676 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.676 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T11:02:13.676 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T11:02:13.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.677 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.677 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-09-10T11:02:13.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-09-10T11:02:13.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:13.677 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.678 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.678 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.678 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.678 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T11:02:13.678 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.678 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-09-10T11:02:13.679 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.679 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.679 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.679 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-09-10T11:02:13.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-09-10T11:02:13.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:13.680 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.680 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-09-10T11:02:13.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.680 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.680 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.681 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T11:02:13.681 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.681 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.681 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-09-10T11:02:13.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-09-10T11:02:13.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.682 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.682 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.682 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-09-10T11:02:13.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.682 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.682 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.682 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T11:02:13.682 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.683 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.683 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-09-10T11:02:13.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-09-10T11:02:13.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-09-10T11:02:13.684 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.684 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.684 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.684 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T11:02:13.684 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.685 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-09-10T11:02:13.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-09-10T11:02:13.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.686 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.686 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T11:02:13.686 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T11:02:13.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.686 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.686 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-09-10T11:02:13.687 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-09-10T11:02:13.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.687 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.687 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:13.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.687 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T11:02:13.688 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.688 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.688 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-09-10T11:02:13.688 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T11:02:13.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:13.689 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.689 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-09-10T11:02:13.689 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.689 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.689 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.689 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T11:02:13.690 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.690 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.690 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-09-10T11:02:13.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-09-10T11:02:13.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.690 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.691 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.691 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.691 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.691 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-09-10T11:02:13.691 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-09-10T11:02:13.691 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-09-10T11:02:13.692 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.692 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T11:02:13.692 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.692 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.692 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.692 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-09-10T11:02:13.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-09-10T11:02:13.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.693 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.693 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.693 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.693 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.693 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.693 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.694 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T11:02:13.694 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.694 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.694 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-09-10T11:02:13.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-09-10T11:02:13.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.695 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.695 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.695 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.695 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.695 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.695 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T11:02:13.695 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.696 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.696 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-09-10T11:02:13.696 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-09-10T11:02:13.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.696 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.697 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.697 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.697 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.697 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.697 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T11:02:13.697 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.698 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.698 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-09-10T11:02:13.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-09-10T11:02:13.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.698 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.698 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T11:02:13.698 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.699 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.699 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.699 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:13.699 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.699 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.699 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-09-10T11:02:13.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-09-10T11:02:13.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.700 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T11:02:13.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.700 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.701 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.701 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-09-10T11:02:13.701 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.701 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.701 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.701 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-09-10T11:02:13.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-09-10T11:02:13.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.702 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.702 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T11:02:13.702 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.702 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.702 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-09-10T11:02:13.703 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.703 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.703 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-09-10T11:02:13.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-09-10T11:02:13.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.704 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.704 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T11:02:13.704 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.704 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.704 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.704 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:13.705 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.705 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-09-10T11:02:13.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-09-10T11:02:13.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.705 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.706 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T11:02:13.706 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.706 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-09-10T11:02:13.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.706 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.707 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-09-10T11:02:13.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-09-10T11:02:13.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.707 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.707 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T11:02:13.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.707 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.708 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.708 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-09-10T11:02:13.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.708 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.708 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-09-10T11:02:13.708 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-09-10T11:02:13.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.709 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.709 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T11:02:13.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.709 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.709 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.709 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-09-10T11:02:13.710 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.710 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.711 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-09-10T11:02:13.712 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-09-10T11:02:13.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.712 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T11:02:13.712 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.712 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.712 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.713 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-09-10T11:02:13.713 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.713 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.713 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.713 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-09-10T11:02:13.713 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-09-10T11:02:13.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.714 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.714 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T11:02:13.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.714 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.714 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.714 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-09-10T11:02:13.714 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.715 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.715 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-09-10T11:02:13.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-09-10T11:02:13.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.715 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.716 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T11:02:13.716 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.716 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.716 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.716 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-09-10T11:02:13.716 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.716 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.716 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.717 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-09-10T11:02:13.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-09-10T11:02:13.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.717 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.717 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T11:02:13.717 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.718 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.718 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.718 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:13.718 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.718 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.718 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-09-10T11:02:13.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-09-10T11:02:13.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.719 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.719 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.719 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T11:02:13.719 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.719 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.719 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.719 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-09-10T11:02:13.719 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.720 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.720 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-09-10T11:02:13.720 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-09-10T11:02:13.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.720 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.721 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T11:02:13.721 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.721 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.721 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.721 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-09-10T11:02:13.721 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.721 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.722 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-09-10T11:02:13.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-09-10T11:02:13.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.722 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.722 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T11:02:13.722 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.722 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.723 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.723 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-09-10T11:02:13.723 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.723 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-09-10T11:02:13.723 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-09-10T11:02:13.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.724 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.724 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T11:02:13.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.724 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.724 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.724 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-09-10T11:02:13.725 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.725 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.725 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-09-10T11:02:13.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-09-10T11:02:13.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.726 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.726 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T11:02:13.726 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.726 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.726 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.726 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-09-10T11:02:13.726 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.727 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.727 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-09-10T11:02:13.727 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-09-10T11:02:13.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.727 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.728 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.728 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T11:02:13.728 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.729 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.729 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-09-10T11:02:13.729 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-09-10T11:02:13.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.729 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.729 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.730 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:13.730 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.730 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.730 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.730 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T11:02:13.730 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-09-10T11:02:13.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T11:02:13.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.732 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-09-10T11:02:13.732 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.732 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.732 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.732 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T11:02:13.732 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.732 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-09-10T11:02:13.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-09-10T11:02:13.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-09-10T11:02:13.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.734 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T11:02:13.734 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.734 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.735 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-09-10T11:02:13.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-09-10T11:02:13.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.735 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-09-10T11:02:13.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.735 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.736 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.736 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-09-10T11:02:13.736 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.736 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.736 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-09-10T11:02:13.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-09-10T11:02:13.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.737 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.737 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.737 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.738 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T11:02:13.738 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.738 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.738 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-09-10T11:02:13.738 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-09-10T11:02:13.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.739 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.739 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:13.739 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.739 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.739 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.739 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-09-10T11:02:13.739 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-09-10T11:02:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-09-10T11:02:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T11:02:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T11:02:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-09-10T11:02:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-09-10T11:02:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-09-10T11:02:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.744 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-09-10T11:02:13.744 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-09-10T11:02:13.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.744 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.744 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.744 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.745 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.745 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.745 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T11:02:13.745 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.745 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.745 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-09-10T11:02:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-09-10T11:02:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.746 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.747 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T11:02:13.747 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.747 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.747 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.747 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.747 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.747 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.748 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.748 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.748 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.748 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.748 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.748 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.748 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.748 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.749 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.749 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.749 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.749 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.749 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.749 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.750 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.750 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-09-10T11:02:13.750 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-09-10T11:02:13.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.750 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.750 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.750 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.750 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.751 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.751 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T11:02:13.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.751 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.752 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.752 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.752 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.752 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.752 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.752 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T11:02:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.753 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.754 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.754 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.754 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.755 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.755 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.755 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.755 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.755 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.755 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.755 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.755 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.756 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.756 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.756 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.756 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.756 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.756 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.757 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.757 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.757 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.757 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.757 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.757 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.758 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.758 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T11:02:13.758 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T11:02:13.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.758 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.758 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.759 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.759 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.759 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.759 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T11:02:13.759 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.760 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T11:02:13.760 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T11:02:13.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.760 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.760 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-09-10T11:02:13.760 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.760 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.760 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.761 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T11:02:13.761 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.761 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.761 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-09-10T11:02:13.761 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-09-10T11:02:13.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.762 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.762 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.762 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.762 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.762 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.762 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-09-10T11:02:13.762 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-09-10T11:02:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-09-10T11:02:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.763 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T11:02:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.765 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-09-10T11:02:13.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-09-10T11:02:13.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.765 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.765 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.765 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.766 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.766 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.766 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-09-10T11:02:13.766 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.766 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.766 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-09-10T11:02:13.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-09-10T11:02:13.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.767 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.767 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.767 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-09-10T11:02:13.767 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.767 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.767 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.768 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-09-10T11:02:13.768 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-09-10T11:02:13.768 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.768 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T11:02:13.768 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.768 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.769 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-09-10T11:02:13.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-09-10T11:02:13.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.769 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.769 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-09-10T11:02:13.769 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.770 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.770 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.770 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T11:02:13.770 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.770 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.770 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-09-10T11:02:13.771 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-09-10T11:02:13.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.771 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T11:02:13.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.771 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.771 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.771 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T11:02:13.772 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.772 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.772 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-09-10T11:02:13.772 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-09-10T11:02:13.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.772 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.773 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.773 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.773 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.773 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.773 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T11:02:13.773 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.773 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.774 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-09-10T11:02:13.774 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-09-10T11:02:13.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.774 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.774 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.775 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.775 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.775 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.775 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T11:02:13.775 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.775 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.776 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-09-10T11:02:13.776 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-09-10T11:02:13.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.776 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.776 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.776 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.776 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.777 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.777 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T11:02:13.777 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.777 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.777 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-09-10T11:02:13.778 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-09-10T11:02:13.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.778 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.778 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.778 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.778 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.778 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.779 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T11:02:13.779 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.779 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.779 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-09-10T11:02:13.779 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-09-10T11:02:13.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.780 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.780 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.780 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.780 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.780 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.780 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T11:02:13.781 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.781 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.781 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-09-10T11:02:13.781 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-09-10T11:02:13.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.781 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.782 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-09-10T11:02:13.782 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.782 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.782 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.782 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-09-10T11:02:13.782 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.783 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.783 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-09-10T11:02:13.783 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-09-10T11:02:13.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.783 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.783 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.783 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.784 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.784 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.784 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T11:02:13.784 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.784 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.784 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.784 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-09-10T11:02:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-09-10T11:02:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.785 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T11:02:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-09-10T11:02:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-09-10T11:02:13.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T11:02:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-09-10T11:02:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-09-10T11:02:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.788 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T11:02:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-09-10T11:02:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-09-10T11:02:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.790 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T11:02:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-09-10T11:02:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-09-10T11:02:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.791 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T11:02:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-09-10T11:02:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.793 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.794 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.795 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.796 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.797 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.798 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.799 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.801 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-09-10T11:02:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-09-10T11:02:13.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-09-10T11:02:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T11:02:13.803 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-09-10T11:02:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-09-10T11:02:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-09-10T11:02:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T11:02:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.805 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-09-10T11:02:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.806 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T11:02:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T11:02:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.807 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T11:02:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T11:02:13.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.809 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T11:02:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T11:02:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.810 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-09-10T11:02:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T11:02:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.813 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T11:02:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T11:02:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.814 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T11:02:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T11:02:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T11:02:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T11:02:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T11:02:13.818 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T11:02:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.819 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T11:02:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T11:02:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.821 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T11:02:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T11:02:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.822 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T11:02:13.823 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T11:02:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.824 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T11:02:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T11:02:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.826 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T11:02:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T11:02:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.828 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T11:02:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T11:02:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.829 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T11:02:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T11:02:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.830 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-09-10T11:02:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T11:02:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T11:02:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.832 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.833 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-09-10T11:02:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-09-10T11:02:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.834 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T11:02:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T11:02:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-09-10T11:02:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.836 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T11:02:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T11:02:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.837 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.838 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T11:02:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T11:02:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.839 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.840 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T11:02:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T11:02:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.841 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T11:02:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T11:02:13.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.843 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T11:02:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T11:02:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.844 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.845 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T11:02:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T11:02:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.846 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.847 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T11:02:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T11:02:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.848 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T11:02:13.849 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T11:02:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.850 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T11:02:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T11:02:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.851 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T11:02:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T11:02:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.853 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T11:02:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T11:02:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T11:02:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T11:02:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T11:02:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T11:02:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T11:02:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T11:02:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T11:02:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T11:02:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T11:02:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T11:02:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T11:02:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T11:02:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T11:02:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-09-10T11:02:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T11:02:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T11:02:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-09-10T11:02:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T11:02:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T11:02:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T11:02:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T11:02:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-09-10T11:02:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T11:02:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T11:02:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T11:02:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T11:02:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T11:02:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T11:02:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T11:02:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T11:02:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T11:02:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T11:02:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-09-10T11:02:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-09-10T11:02:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-09-10T11:02:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-09-10T11:02:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-09-10T11:02:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-09-10T11:02:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-09-10T11:02:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-09-10T11:02:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-09-10T11:02:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-09-10T11:02:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-09-10T11:02:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-09-10T11:02:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-09-10T11:02:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-09-10T11:02:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-09-10T11:02:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-09-10T11:02:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T11:02:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-09-10T11:02:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-09-10T11:02:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-09-10T11:02:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-09-10T11:02:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-09-10T11:02:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-09-10T11:02:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-09-10T11:02:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-09-10T11:02:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-09-10T11:02:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-09-10T11:02:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-09-10T11:02:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-09-10T11:02:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-09-10T11:02:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-09-10T11:02:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-09-10T11:02:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-09-10T11:02:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-09-10T11:02:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-09-10T11:02:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-09-10T11:02:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-09-10T11:02:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-09-10T11:02:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-09-10T11:02:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-09-10T11:02:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-09-10T11:02:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-09-10T11:02:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-09-10T11:02:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-09-10T11:02:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-09-10T11:02:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-09-10T11:02:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T11:02:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-09-10T11:02:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-09-10T11:02:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-09-10T11:02:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T11:02:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-09-10T11:02:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-09-10T11:02:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-09-10T11:02:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-09-10T11:02:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-09-10T11:02:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-09-10T11:02:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-09-10T11:02:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T11:02:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-09-10T11:02:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-09-10T11:02:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T11:02:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T11:02:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-09-10T11:02:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-09-10T11:02:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-09-10T11:02:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T11:02:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-09-10T11:02:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-09-10T11:02:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-09-10T11:02:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-09-10T11:02:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-09-10T11:02:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-09-10T11:02:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-09-10T11:02:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-09-10T11:02:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-09-10T11:02:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-09-10T11:02:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-09-10T11:02:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-09-10T11:02:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-09-10T11:02:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-09-10T11:02:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T11:02:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-09-10T11:02:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-09-10T11:02:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-09-10T11:02:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T11:02:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-09-10T11:02:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-09-10T11:02:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-09-10T11:02:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-09-10T11:02:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-09-10T11:02:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T11:02:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-09-10T11:02:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-09-10T11:02:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-09-10T11:02:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-09-10T11:02:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-09-10T11:02:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-09-10T11:02:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-09-10T11:02:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-09-10T11:02:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-09-10T11:02:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-09-10T11:02:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T11:02:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-09-10T11:02:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-09-10T11:02:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-09-10T11:02:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T11:02:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-09-10T11:02:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-09-10T11:02:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-09-10T11:02:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-09-10T11:02:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-09-10T11:02:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-09-10T11:02:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T11:02:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-09-10T11:02:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-09-10T11:02:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-09-10T11:02:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-09-10T11:02:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-09-10T11:02:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-09-10T11:02:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T11:02:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-09-10T11:02:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-09-10T11:02:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-09-10T11:02:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T11:02:14.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.010 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.010 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.011 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.011 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.011 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.011 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.012 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.012 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.012 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.012 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.013 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.014 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.014 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.016 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.017 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.017 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.017 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.018 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.018 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.018 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.018 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.019 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-09-10T11:02:14.020 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.020 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.020 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.020 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-09-10T11:02:14.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-09-10T11:02:14.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.021 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-09-10T11:02:14.021 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.021 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.021 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.021 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-09-10T11:02:14.021 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T11:02:14.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.022 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.022 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.023 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.023 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.023 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.023 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.023 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.023 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.024 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.024 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.024 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.024 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.024 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.024 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.025 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.025 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.025 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.025 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.025 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.026 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.026 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.027 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.028 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.028 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.028 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.028 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.028 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.028 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.028 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.029 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.029 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.029 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.029 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.030 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.031 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.031 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-09-10T11:02:14.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-09-10T11:02:14.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.032 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.032 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.032 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.032 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-09-10T11:02:14.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T11:02:14.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.033 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.033 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-09-10T11:02:14.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-09-10T11:02:14.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.033 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.034 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-09-10T11:02:14.034 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.034 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.034 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.034 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-09-10T11:02:14.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.035 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.035 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-09-10T11:02:14.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-09-10T11:02:14.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-09-10T11:02:14.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.036 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.036 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.036 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T11:02:14.036 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.036 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.037 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-09-10T11:02:14.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-09-10T11:02:14.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.037 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.037 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.038 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.038 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.038 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-09-10T11:02:14.038 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.038 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.038 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-09-10T11:02:14.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-09-10T11:02:14.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.039 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-09-10T11:02:14.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.039 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.040 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T11:02:14.040 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T11:02:14.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.040 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-09-10T11:02:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.041 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.042 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.042 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.042 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.043 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.043 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.043 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.043 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.043 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.043 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.044 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.044 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.044 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.045 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.045 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.046 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.046 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.046 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.047 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.047 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.047 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.047 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.047 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.048 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.048 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.048 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.048 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.049 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.049 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.049 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.049 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.049 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.050 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.050 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.050 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-09-10T11:02:14.051 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.051 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.051 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.051 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.052 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.052 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.053 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.053 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.053 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.053 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.053 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.053 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.054 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.054 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.055 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.056 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.056 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.057 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.057 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.057 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.057 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.057 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.057 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.058 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.058 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.058 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.058 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.059 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-09-10T11:02:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-09-10T11:02:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-09-10T11:02:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-09-10T11:02:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T11:02:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.082 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-09-10T11:02:14.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-09-10T11:02:14.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.083 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.083 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.083 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-09-10T11:02:14.083 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-09-10T11:02:14.084 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-09-10T11:02:14.084 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-09-10T11:02:14.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-09-10T11:02:14.085 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.085 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.085 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.085 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.086 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.086 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.086 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.087 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.087 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.087 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.087 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.087 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.087 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.087 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.087 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.088 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.088 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.088 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.089 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.089 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.089 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.090 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.090 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.090 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.090 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.090 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.091 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.091 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.091 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.091 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.091 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.091 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.092 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.092 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.092 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.092 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.093 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.094 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.094 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.094 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-09-10T11:02:14.095 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.095 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.095 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.095 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.096 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.096 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.096 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.097 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.097 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.097 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.097 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.097 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.097 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.098 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.099 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.099 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.099 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.099 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.099 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.100 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.101 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.101 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.101 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.101 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.101 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.101 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.102 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.102 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.102 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.102 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.103 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.103 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.104 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-09-10T11:02:14.105 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.105 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.105 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.105 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.106 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.106 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.106 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.106 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.107 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.107 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.107 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.107 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.107 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.108 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.108 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.109 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.109 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.109 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.113 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.114 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.114 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.114 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:14.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:14.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:14.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:14.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.115 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-09-10T11:02:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-09-10T11:02:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:14.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.117 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-09-10T11:02:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-09-10T11:02:14.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-09-10T11:02:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T11:02:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-09-10T11:02:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-09-10T11:02:14.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-09-10T11:02:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.122 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T11:02:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-09-10T11:02:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-09-10T11:02:14.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.125 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.126 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.128 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.129 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.131 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.132 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-09-10T11:02:14.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-09-10T11:02:14.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.135 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-09-10T11:02:14.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.135 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-09-10T11:02:14.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.136 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.136 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-09-10T11:02:14.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-09-10T11:02:14.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:14.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-09-10T11:02:14.137 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.137 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-09-10T11:02:14.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-09-10T11:02:14.138 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.138 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.139 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.139 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-09-10T11:02:14.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-09-10T11:02:14.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:14.140 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.140 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.140 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.141 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-09-10T11:02:14.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-09-10T11:02:14.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:14.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.142 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.142 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.142 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T11:02:14.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T11:02:14.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.142 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.143 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.143 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.144 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.144 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.144 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.144 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.144 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.144 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.144 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.144 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.145 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.145 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.146 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.146 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.146 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.146 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.147 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.148 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.148 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.148 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.148 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.148 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.149 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.149 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.149 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.149 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.149 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.149 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.150 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.150 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.150 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.150 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.150 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.151 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.151 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.151 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-09-10T11:02:14.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-09-10T11:02:14.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.152 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.152 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.152 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T11:02:14.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T11:02:14.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.154 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-09-10T11:02:14.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.155 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.155 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-09-10T11:02:14.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-09-10T11:02:14.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:14.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-09-10T11:02:14.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.157 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.157 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.157 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-09-10T11:02:14.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-09-10T11:02:14.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.158 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-09-10T11:02:14.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.158 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T11:02:14.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.159 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-09-10T11:02:14.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-09-10T11:02:14.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-09-10T11:02:14.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.160 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T11:02:14.160 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.161 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-09-10T11:02:14.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-09-10T11:02:14.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-09-10T11:02:14.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.162 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.162 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.163 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-09-10T11:02:14.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-09-10T11:02:14.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.163 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-09-10T11:02:14.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-09-10T11:02:14.164 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-09-10T11:02:14.164 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.165 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-09-10T11:02:14.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-09-10T11:02:14.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.166 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-09-10T11:02:14.166 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-09-10T11:02:14.166 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-09-10T11:02:14.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.166 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-09-10T11:02:14.167 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.167 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.167 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.167 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.168 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.168 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.169 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.169 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.169 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.169 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.169 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.169 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.170 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.170 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.170 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.171 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.171 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.171 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.171 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.172 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.173 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.173 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.173 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.173 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.174 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.174 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.174 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.174 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.174 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.175 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.175 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.176 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.176 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-09-10T11:02:14.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-09-10T11:02:14.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-09-10T11:02:14.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.178 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.178 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.178 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.179 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-09-10T11:02:14.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T11:02:14.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.179 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.180 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.180 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.180 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.180 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-09-10T11:02:14.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-09-10T11:02:14.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.181 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.181 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.181 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.181 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.181 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.182 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-09-10T11:02:14.183 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.183 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.183 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-09-10T11:02:14.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-09-10T11:02:14.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.183 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:14.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.184 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.185 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-09-10T11:02:14.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-09-10T11:02:14.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.186 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.186 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.186 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.187 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.187 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.188 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.188 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.188 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.188 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.188 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.189 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.189 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.189 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.189 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.190 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.191 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.192 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.192 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.192 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.193 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.193 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.193 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.193 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.193 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.193 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.194 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.194 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.194 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.196 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-09-10T11:02:14.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-09-10T11:02:14.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.197 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-09-10T11:02:14.197 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-09-10T11:02:14.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.198 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.198 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.198 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.198 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-09-10T11:02:14.200 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.200 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.200 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.200 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.201 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.201 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.201 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.201 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.202 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.202 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.202 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.202 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.202 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.202 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.205 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.205 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.205 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.205 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.206 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.206 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.206 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.206 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.206 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.206 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.207 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.207 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.207 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.207 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.207 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.207 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.208 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.208 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.208 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.208 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-09-10T11:02:14.210 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.210 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.210 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.210 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.211 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.211 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.211 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.211 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.211 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.212 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.212 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.212 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.212 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.212 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.212 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.212 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.213 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.213 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.213 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.213 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.214 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.214 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.214 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.215 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.215 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.216 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.216 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.216 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.216 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.216 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.216 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.217 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.217 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.217 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.217 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.217 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.217 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.218 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.218 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.218 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.219 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.219 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.219 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.219 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-09-10T11:02:14.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-09-10T11:02:14.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.220 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.220 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-09-10T11:02:14.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.220 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T11:02:14.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-09-10T11:02:14.222 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.222 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.222 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.222 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.222 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.223 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.223 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.223 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.224 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.224 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.224 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.224 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.224 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.224 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.225 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.225 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.226 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.226 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.226 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.226 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.227 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.227 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-09-10T11:02:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-09-10T11:02:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-09-10T11:02:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-09-10T11:02:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-09-10T11:02:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-09-10T11:02:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-09-10T11:02:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-09-10T11:02:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-09-10T11:02:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-09-10T11:02:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-09-10T11:02:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-09-10T11:02:14.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.239 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.239 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.239 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.240 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-09-10T11:02:14.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-09-10T11:02:14.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.240 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.241 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.241 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-09-10T11:02:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-09-10T11:02:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-09-10T11:02:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-09-10T11:02:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-09-10T11:02:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-09-10T11:02:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-09-10T11:02:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-09-10T11:02:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-09-10T11:02:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-09-10T11:02:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-09-10T11:02:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-09-10T11:02:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.251 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.251 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-09-10T11:02:14.251 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-09-10T11:02:14.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.251 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.252 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.252 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.252 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-09-10T11:02:14.253 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.253 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.254 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.254 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-09-10T11:02:14.254 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-09-10T11:02:14.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.255 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.255 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T11:02:14.255 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.255 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.256 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-09-10T11:02:14.256 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-09-10T11:02:14.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.256 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.256 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.256 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.256 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-09-10T11:02:14.257 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.257 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.258 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.258 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.258 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.258 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.258 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.258 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.259 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.259 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.259 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.259 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.259 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.259 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.259 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.259 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.260 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.260 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.261 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.261 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.261 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.261 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.262 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.262 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.263 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.263 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.263 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.263 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.263 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.263 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.263 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.264 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.264 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.264 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.264 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.264 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.265 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.265 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.265 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.265 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.266 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.266 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.266 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-09-10T11:02:14.267 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.267 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.267 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.267 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.268 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.268 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.268 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.268 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.269 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.269 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.269 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.269 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.269 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.269 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.269 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.270 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.270 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.271 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.271 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.271 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.271 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.271 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.271 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.272 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.272 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.272 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.273 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.273 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.273 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.273 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.273 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.273 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.273 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.274 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.274 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.274 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.274 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.274 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.274 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.274 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.275 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.275 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.275 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.276 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-09-10T11:02:14.277 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.277 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.277 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.277 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.277 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.278 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.278 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.278 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.278 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.279 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.279 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.279 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.279 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.279 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.280 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.280 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.280 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.280 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.280 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.281 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.281 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.281 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.281 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.281 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.282 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.282 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.282 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.282 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.282 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.283 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.283 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.283 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.283 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.283 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.283 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.283 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.284 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.284 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.285 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.285 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.285 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.286 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.286 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.286 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-09-10T11:02:14.287 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.287 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.287 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.287 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-09-10T11:02:14.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-09-10T11:02:14.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T11:02:14.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.288 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.289 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:14.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:14.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:14.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.290 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-09-10T11:02:14.290 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.290 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.291 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.291 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.291 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.292 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.292 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.292 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.292 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.292 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.292 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.292 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.293 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.293 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.293 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.293 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.293 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.293 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.295 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.295 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.295 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.295 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.296 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.296 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.296 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.296 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.296 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.296 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.297 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.297 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.297 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.297 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.297 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.297 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.297 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.297 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.298 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.298 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.298 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.299 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.299 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.300 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.300 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-09-10T11:02:14.300 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.300 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.301 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.301 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-09-10T11:02:14.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-09-10T11:02:14.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:14.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.302 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.302 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.302 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T11:02:14.302 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.302 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.303 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-09-10T11:02:14.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-09-10T11:02:14.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.303 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:14.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.304 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T11:02:14.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.304 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.304 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-09-10T11:02:14.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-09-10T11:02:14.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:14.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.305 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T11:02:14.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.306 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-09-10T11:02:14.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-09-10T11:02:14.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.307 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T11:02:14.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.308 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-09-10T11:02:14.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-09-10T11:02:14.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.309 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.309 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.309 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.309 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T11:02:14.309 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.310 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.310 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-09-10T11:02:14.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-09-10T11:02:14.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.310 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.310 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.311 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.311 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.311 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.312 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-09-10T11:02:14.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-09-10T11:02:14.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.312 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.312 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.312 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.312 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.313 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.313 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.313 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.313 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-09-10T11:02:14.313 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-09-10T11:02:14.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.314 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.314 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T11:02:14.314 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.314 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.314 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.314 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.315 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.315 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.315 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-09-10T11:02:14.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-09-10T11:02:14.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.316 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.316 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.316 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.316 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.317 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:14.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:14.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.317 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-09-10T11:02:14.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-09-10T11:02:14.318 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.318 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.318 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.319 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-09-10T11:02:14.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-09-10T11:02:14.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.319 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.319 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-09-10T11:02:14.320 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.320 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.320 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.320 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.320 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.320 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.321 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-09-10T11:02:14.321 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-09-10T11:02:14.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.322 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.322 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.322 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.322 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.323 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-09-10T11:02:14.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-09-10T11:02:14.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.323 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.323 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.324 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.324 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.324 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.324 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.324 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.325 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.325 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.325 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.325 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.326 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.326 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.326 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.326 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.326 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.326 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.326 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.327 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.327 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.327 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.328 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.328 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.328 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.330 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.330 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.330 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.330 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.330 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.331 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.331 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.331 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.331 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.332 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.332 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.332 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.333 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.333 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.334 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-09-10T11:02:14.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-09-10T11:02:14.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.334 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.334 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.335 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.335 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-09-10T11:02:14.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-09-10T11:02:14.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.337 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.337 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.337 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-09-10T11:02:14.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-09-10T11:02:14.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.338 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.338 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T11:02:14.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-09-10T11:02:14.340 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.340 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.340 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.340 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.341 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.341 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.341 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.341 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.341 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.341 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.341 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.341 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.342 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.342 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.342 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.342 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.342 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.342 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.343 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.343 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.343 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.343 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.344 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.344 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.344 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.345 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.345 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.345 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.345 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.346 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.346 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.346 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.346 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.346 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.346 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.346 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.347 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.347 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.347 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.347 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.347 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.348 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.348 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.348 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.348 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.348 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-09-10T11:02:14.349 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.350 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.350 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.350 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.350 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.351 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.351 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.351 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.351 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.352 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.352 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.352 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.352 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.352 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.352 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.352 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.353 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.353 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.354 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.354 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.354 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.354 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.354 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.355 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.357 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.357 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.357 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.357 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.358 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.358 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.358 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.358 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.358 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.358 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.358 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.359 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-09-10T11:02:14.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-09-10T11:02:14.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.359 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.360 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-09-10T11:02:14.360 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.360 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T11:02:14.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.361 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-09-10T11:02:14.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-09-10T11:02:14.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.361 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.362 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.362 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.362 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-09-10T11:02:14.362 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.363 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.363 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-09-10T11:02:14.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-09-10T11:02:14.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.363 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.364 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-09-10T11:02:14.364 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.364 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T11:02:14.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.365 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.365 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-09-10T11:02:14.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T11:02:14.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.366 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-09-10T11:02:14.366 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.366 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T11:02:14.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-09-10T11:02:14.367 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-09-10T11:02:14.368 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-09-10T11:02:14.368 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-09-10T11:02:14.368 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-09-10T11:02:14.368 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-09-10T11:02:14.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-09-10T11:02:14.368 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-09-10T11:02:14.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.369 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.369 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.369 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.369 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.370 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-09-10T11:02:14.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-09-10T11:02:14.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.370 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.370 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.370 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.371 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.371 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-09-10T11:02:14.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-09-10T11:02:14.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-09-10T11:02:14.372 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-09-10T11:02:14.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.372 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.373 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-09-10T11:02:14.373 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-09-10T11:02:14.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.374 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.374 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.374 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.374 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.374 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.374 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.374 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.374 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.375 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.375 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.375 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.375 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.375 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.375 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.376 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-09-10T11:02:14.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-09-10T11:02:14.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.377 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.377 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.377 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.377 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.378 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-09-10T11:02:14.378 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-09-10T11:02:14.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.378 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-09-10T11:02:14.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.379 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-09-10T11:02:14.379 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-09-10T11:02:14.379 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-09-10T11:02:14.379 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-09-10T11:02:14.379 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-09-10T11:02:14.379 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-09-10T11:02:14.379 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-09-10T11:02:14.380 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.380 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.380 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.380 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-09-10T11:02:14.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-09-10T11:02:14.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-09-10T11:02:14.381 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-10T11:02:14.381 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-09-10T11:02:14.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.381 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.382 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.382 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-09-10T11:02:14.382 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-09-10T11:02:14.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-09-10T11:02:14.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.383 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.383 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.383 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.383 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.384 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-09-10T11:02:14.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-09-10T11:02:14.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-09-10T11:02:14.384 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-09-10T11:02:14.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.385 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.385 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.386 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-09-10T11:02:14.386 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-09-10T11:02:14.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-09-10T11:02:14.386 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-09-10T11:02:14.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-09-10T11:02:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T11:02:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-09-10T11:02:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-09-10T11:02:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-09-10T11:02:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-09-10T11:02:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-09-10T11:02:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-09-10T11:02:14.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-09-10T11:02:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: "services": { 2024-09-10T11:02:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard": "https://172.21.15.7:8443/" 2024-09-10T11:02:14.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.389 INFO:teuthology.orchestra.run.smithi007.stdout: "always_on_modules": { 2024-09-10T11:02:14.389 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus": [ 2024-09-10T11:02:14.389 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-09-10T11:02:14.389 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-10T11:02:14.389 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-09-10T11:02:14.389 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-09-10T11:02:14.389 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-09-10T11:02:14.389 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-09-10T11:02:14.390 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-09-10T11:02:14.390 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-09-10T11:02:14.390 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-09-10T11:02:14.390 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-09-10T11:02:14.390 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.390 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific": [ 2024-09-10T11:02:14.390 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-09-10T11:02:14.391 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-10T11:02:14.391 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-09-10T11:02:14.391 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-09-10T11:02:14.391 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-09-10T11:02:14.391 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-09-10T11:02:14.391 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-09-10T11:02:14.391 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-09-10T11:02:14.391 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-09-10T11:02:14.392 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-09-10T11:02:14.392 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.392 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy": [ 2024-09-10T11:02:14.392 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-09-10T11:02:14.392 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-10T11:02:14.392 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-09-10T11:02:14.392 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-09-10T11:02:14.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-09-10T11:02:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-09-10T11:02:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-09-10T11:02:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-09-10T11:02:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-09-10T11:02:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-09-10T11:02:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: "reef": [ 2024-09-10T11:02:14.394 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-09-10T11:02:14.394 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-10T11:02:14.394 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-09-10T11:02:14.394 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-09-10T11:02:14.394 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-09-10T11:02:14.394 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-09-10T11:02:14.394 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-09-10T11:02:14.394 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-09-10T11:02:14.395 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-09-10T11:02:14.395 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-09-10T11:02:14.395 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T11:02:14.395 INFO:teuthology.orchestra.run.smithi007.stdout: "squid": [ 2024-09-10T11:02:14.395 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-09-10T11:02:14.395 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-09-10T11:02:14.395 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-09-10T11:02:14.395 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-09-10T11:02:14.396 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-09-10T11:02:14.396 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-09-10T11:02:14.396 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-09-10T11:02:14.396 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-09-10T11:02:14.396 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-09-10T11:02:14.396 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-09-10T11:02:14.396 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:14.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.397 INFO:teuthology.orchestra.run.smithi007.stdout: "last_failure_osd_epoch": 82, 2024-09-10T11:02:14.397 INFO:teuthology.orchestra.run.smithi007.stdout: "active_clients": [ 2024-09-10T11:02:14.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.397 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-09-10T11:02:14.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-10T11:02:14.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-10T11:02:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-09-10T11:02:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 540136581 2024-09-10T11:02:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "libcephsqlite", 2024-09-10T11:02:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-10T11:02:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-10T11:02:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-09-10T11:02:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1302454085 2024-09-10T11:02:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-09-10T11:02:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-10T11:02:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-10T11:02:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-09-10T11:02:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2466563482 2024-09-10T11:02:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-09-10T11:02:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-10T11:02:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-10T11:02:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-09-10T11:02:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 733492144 2024-09-10T11:02:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-09-10T11:02:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-10T11:02:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-10T11:02:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-09-10T11:02:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 4189157918 2024-09-10T11:02:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-09-10T11:02:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-10T11:02:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T11:02:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-10T11:02:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-09-10T11:02:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1801082971 2024-09-10T11:02:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T11:02:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T11:02:14.408 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:12 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/539124230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T11:02:14.408 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:12 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/3830994735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T11:02:14.408 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:12 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:02:14.409 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:12 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/382346088' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T11:02:14.409 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:13 smithi133 ceph-mon[35282]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-09-10T11:02:14.409 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:13 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:02:14.409 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:13 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:02:14.409 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:13 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T11:02:14.409 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:12 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/539124230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T11:02:14.409 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:12 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/3830994735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T11:02:14.410 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:12 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:02:14.410 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:12 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/382346088' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T11:02:14.410 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:13 smithi103 ceph-mon[35160]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-09-10T11:02:14.410 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:13 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:02:14.410 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:13 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:02:14.410 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:13 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T11:02:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 -- 172.21.15.7:0/382346088 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f34cc0a2d00 msgr2=0x7f34cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f34cc0a2d00 0x7f34cc0a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f34d4005ab0 tx=0x7f34d4000bc0 comp rx=0 tx=0).stop 2024-09-10T11:02:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 -- 172.21.15.7:0/382346088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34e4103610 msgr2=0x7f34e4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34e4103610 0x7f34e4198910 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f34d800e9f0 tx=0x7f34d800eeb0 comp rx=0 tx=0).stop 2024-09-10T11:02:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 -- 172.21.15.7:0/382346088 shutdown_connections 2024-09-10T11:02:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34e410c0c0 0x7f34e419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f34cc0a2d00 0x7f34cc0a51c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34e4103610 0x7f34e4198910 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 --2- 172.21.15.7:0/382346088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34e4102cf0 0x7f34e41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 -- 172.21.15.7:0/382346088 >> 172.21.15.7:0/382346088 conn(0x7f34e4077a10 msgr2=0x7f34e40fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 -- 172.21.15.7:0/382346088 shutdown_connections 2024-09-10T11:02:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:12.038+0000 7f34e9333640 1 -- 172.21.15.7:0/382346088 wait complete. 2024-09-10T11:02:14.419 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T11:02:14.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T11:02:14.419 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:12 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/539124230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T11:02:14.420 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:12 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/3830994735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T11:02:14.420 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:12 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:02:14.420 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:12 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/382346088' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T11:02:14.420 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:13 smithi007 ceph-mon[30873]: pgmap v518: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-09-10T11:02:14.420 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:13 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:02:14.420 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:13 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:02:14.420 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:13 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' 2024-09-10T11:02:14.678 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:14.800 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:14 smithi007 ceph-mon[30873]: osdmap e109: 12 total, 12 up, 12 in 2024-09-10T11:02:14.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:14 smithi103 ceph-mon[35160]: osdmap e109: 12 total, 12 up, 12 in 2024-09-10T11:02:14.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:14 smithi133 ceph-mon[35282]: osdmap e109: 12 total, 12 up, 12 in 2024-09-10T11:02:15.149 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T11:02:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-09-10T11:02:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-09-10T11:02:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-09-10T11:02:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-09-10T11:02:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-09-10T11:02:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-09-10T11:02:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.001529544, 2024-09-10T11:02:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T11:02:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000227589, 2024-09-10T11:02:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T11:02:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T11:02:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 304, 2024-09-10T11:02:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 294, 2024-09-10T11:02:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1466912, 2024-09-10T11:02:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 181040, 2024-09-10T11:02:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-09-10T11:02:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-09-10T11:02:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.043879866, 2024-09-10T11:02:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.017141177, 2024-09-10T11:02:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.038178032, 2024-09-10T11:02:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 293, 2024-09-10T11:02:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.013261839, 2024-09-10T11:02:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000045262 2024-09-10T11:02:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 1466912, 2024-09-10T11:02:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 181040 2024-09-10T11:02:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 7, 2024-09-10T11:02:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 227, 2024-09-10T11:02:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3472, 2024-09-10T11:02:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 276016, 2024-09-10T11:02:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 6, 2024-09-10T11:02:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 3, 2024-09-10T11:02:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.017720714, 2024-09-10T11:02:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.010877979, 2024-09-10T11:02:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.004268517, 2024-09-10T11:02:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 225, 2024-09-10T11:02:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.018362670, 2024-09-10T11:02:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000081611 2024-09-10T11:02:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-09-10T11:02:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 276016 2024-09-10T11:02:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-09-10T11:02:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-09-10T11:02:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-09-10T11:02:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-09-10T11:02:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-09-10T11:02:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148748, 2024-09-10T11:02:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 32, 2024-09-10T11:02:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-09-10T11:02:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-09-10T11:02:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-09-10T11:02:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-09-10T11:02:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-09-10T11:02:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-09-10T11:02:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-09-10T11:02:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-09-10T11:02:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-09-10T11:02:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-09-10T11:02:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-09-10T11:02:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-09-10T11:02:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-09-10T11:02:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-09-10T11:02:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 307, 2024-09-10T11:02:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 1424039, 2024-09-10T11:02:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 307, 2024-09-10T11:02:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-09-10T11:02:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-09-10T11:02:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 307, 2024-09-10T11:02:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 1424039, 2024-09-10T11:02:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-09-10T11:02:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:15.171 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T11:02:15.683 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:15 smithi007 ceph-mon[30873]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 537 op/s 2024-09-10T11:02:15.885 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-10T11:02:15.885 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-10T11:02:15.885 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-10T11:02:15.885 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-10T11:02:15.885 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-10T11:02:15.885 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T11:02:15.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T11:02:15.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:15 smithi103 ceph-mon[35160]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 537 op/s 2024-09-10T11:02:15.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:15 smithi133 ceph-mon[35282]: pgmap v520: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 8.2 MiB/s wr, 537 op/s 2024-09-10T11:02:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:16.658 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T11:02:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:16.658 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 309, 2024-09-10T11:02:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 299, 2024-09-10T11:02:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1468272, 2024-09-10T11:02:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 184240, 2024-09-10T11:02:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-09-10T11:02:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-09-10T11:02:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.046900244, 2024-09-10T11:02:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.017549321, 2024-09-10T11:02:16.659 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.039760134, 2024-09-10T11:02:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 298, 2024-09-10T11:02:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.013936298, 2024-09-10T11:02:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000046766 2024-09-10T11:02:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:16.660 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 1468272, 2024-09-10T11:02:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 184240 2024-09-10T11:02:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:16.661 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 3, 2024-09-10T11:02:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 229, 2024-09-10T11:02:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 480, 2024-09-10T11:02:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 276272, 2024-09-10T11:02:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 6, 2024-09-10T11:02:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 3, 2024-09-10T11:02:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.017074921, 2024-09-10T11:02:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.012177232, 2024-09-10T11:02:16.662 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.002313730, 2024-09-10T11:02:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 226, 2024-09-10T11:02:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.013767571, 2024-09-10T11:02:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000060918 2024-09-10T11:02:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:16.663 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T11:02:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 276272 2024-09-10T11:02:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 4, 2024-09-10T11:02:16.664 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 4, 2024-09-10T11:02:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 2992, 2024-09-10T11:02:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 448, 2024-09-10T11:02:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-09-10T11:02:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-09-10T11:02:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.002331857, 2024-09-10T11:02:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000043991, 2024-09-10T11:02:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.002284441, 2024-09-10T11:02:16.665 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 3, 2024-09-10T11:02:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.004169370, 2024-09-10T11:02:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.001389790 2024-09-10T11:02:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:16.666 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T11:02:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T11:02:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-09-10T11:02:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-09-10T11:02:16.667 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-09-10T11:02:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-09-10T11:02:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:16.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:16.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:16.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:16.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-09-10T11:02:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148714, 2024-09-10T11:02:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-09-10T11:02:16.672 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-09-10T11:02:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-09-10T11:02:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-09-10T11:02:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-09-10T11:02:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:16.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-09-10T11:02:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-09-10T11:02:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-09-10T11:02:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-09-10T11:02:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-09-10T11:02:16.674 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-09-10T11:02:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-09-10T11:02:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-09-10T11:02:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-09-10T11:02:16.675 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-09-10T11:02:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 312, 2024-09-10T11:02:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 1424689, 2024-09-10T11:02:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 312, 2024-09-10T11:02:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-09-10T11:02:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-09-10T11:02:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 312, 2024-09-10T11:02:16.676 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 1424689, 2024-09-10T11:02:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-09-10T11:02:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:16.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:16.677 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T11:02:17.423 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-10T11:02:17.423 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-10T11:02:17.423 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-10T11:02:17.423 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-10T11:02:17.423 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-10T11:02:17.423 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T11:02:17.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T11:02:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:17.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:17 smithi103 ceph-mon[35160]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 695 op/s 2024-09-10T11:02:17.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:17 smithi007 ceph-mon[30873]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 695 op/s 2024-09-10T11:02:17.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:17 smithi133 ceph-mon[35282]: pgmap v521: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 695 op/s 2024-09-10T11:02:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T11:02:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 305, 2024-09-10T11:02:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 295, 2024-09-10T11:02:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1467184, 2024-09-10T11:02:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 181680, 2024-09-10T11:02:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-09-10T11:02:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-09-10T11:02:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.046676108, 2024-09-10T11:02:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.017492925, 2024-09-10T11:02:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.038736095, 2024-09-10T11:02:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 294, 2024-09-10T11:02:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.013207356, 2024-09-10T11:02:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000044922 2024-09-10T11:02:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 1467184, 2024-09-10T11:02:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 181680 2024-09-10T11:02:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 7, 2024-09-10T11:02:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 228, 2024-09-10T11:02:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3472, 2024-09-10T11:02:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 276192, 2024-09-10T11:02:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 6, 2024-09-10T11:02:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 3, 2024-09-10T11:02:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.018322672, 2024-09-10T11:02:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.011310928, 2024-09-10T11:02:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.004302900, 2024-09-10T11:02:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 226, 2024-09-10T11:02:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.022133624, 2024-09-10T11:02:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000097936 2024-09-10T11:02:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-09-10T11:02:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 276192 2024-09-10T11:02:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-09-10T11:02:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-09-10T11:02:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-09-10T11:02:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-09-10T11:02:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-09-10T11:02:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-09-10T11:02:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.000312061, 2024-09-10T11:02:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T11:02:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000213781, 2024-09-10T11:02:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T11:02:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T11:02:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-09-10T11:02:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-09-10T11:02:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-09-10T11:02:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-09-10T11:02:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:18.378 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:18.378 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:18.378 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-09-10T11:02:18.378 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:18.378 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:18.378 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:18.378 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:18.378 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148714, 2024-09-10T11:02:18.379 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-09-10T11:02:18.379 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-09-10T11:02:18.379 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-09-10T11:02:18.379 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-09-10T11:02:18.379 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-09-10T11:02:18.379 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:18.379 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:18.379 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:18.380 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:18.380 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-09-10T11:02:18.380 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-09-10T11:02:18.380 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:18.380 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:18.380 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:18.380 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-09-10T11:02:18.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-09-10T11:02:18.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-09-10T11:02:18.381 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:18.381 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-09-10T11:02:18.381 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:18.381 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-09-10T11:02:18.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:18.381 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:18.381 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-09-10T11:02:18.382 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-09-10T11:02:18.382 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-09-10T11:02:18.382 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 308, 2024-09-10T11:02:18.382 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 1424169, 2024-09-10T11:02:18.382 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:18.382 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 308, 2024-09-10T11:02:18.382 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-09-10T11:02:18.383 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-09-10T11:02:18.383 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 308, 2024-09-10T11:02:18.383 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 1424169, 2024-09-10T11:02:18.383 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-09-10T11:02:18.383 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:18.383 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:18.384 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:18.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:18.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T11:02:19.033 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-10T11:02:19.033 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-10T11:02:19.033 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-10T11:02:19.033 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-10T11:02:19.033 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-10T11:02:19.033 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T11:02:19.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T11:02:19.299 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:19.684 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:19 smithi007 ceph-mon[30873]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 10 MiB/s wr, 593 op/s 2024-09-10T11:02:19.765 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T11:02:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 4, 2024-09-10T11:02:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 4, 2024-09-10T11:02:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 2992, 2024-09-10T11:02:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 448, 2024-09-10T11:02:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-09-10T11:02:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-09-10T11:02:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.003336270, 2024-09-10T11:02:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000041923, 2024-09-10T11:02:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.002338477, 2024-09-10T11:02:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 3, 2024-09-10T11:02:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.004172266, 2024-09-10T11:02:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.001390755 2024-09-10T11:02:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T11:02:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T11:02:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 3, 2024-09-10T11:02:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 230, 2024-09-10T11:02:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 480, 2024-09-10T11:02:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 276448, 2024-09-10T11:02:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 6, 2024-09-10T11:02:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 3, 2024-09-10T11:02:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.017863912, 2024-09-10T11:02:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.012508429, 2024-09-10T11:02:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.002544304, 2024-09-10T11:02:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 227, 2024-09-10T11:02:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.014127429, 2024-09-10T11:02:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000062235 2024-09-10T11:02:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T11:02:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 276448 2024-09-10T11:02:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 310, 2024-09-10T11:02:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 300, 2024-09-10T11:02:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1468544, 2024-09-10T11:02:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 184880, 2024-09-10T11:02:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-09-10T11:02:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-09-10T11:02:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.047104755, 2024-09-10T11:02:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.018275788, 2024-09-10T11:02:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.041369358, 2024-09-10T11:02:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 299, 2024-09-10T11:02:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.012962347, 2024-09-10T11:02:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000043352 2024-09-10T11:02:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 1468544, 2024-09-10T11:02:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 184880 2024-09-10T11:02:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-09-10T11:02:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-09-10T11:02:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-09-10T11:02:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-09-10T11:02:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-09-10T11:02:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148714, 2024-09-10T11:02:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-09-10T11:02:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-09-10T11:02:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-09-10T11:02:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-09-10T11:02:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-09-10T11:02:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-09-10T11:02:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-09-10T11:02:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-09-10T11:02:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-09-10T11:02:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-09-10T11:02:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-09-10T11:02:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-09-10T11:02:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T11:02:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-09-10T11:02:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-09-10T11:02:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-09-10T11:02:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 313, 2024-09-10T11:02:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 1424819, 2024-09-10T11:02:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 313, 2024-09-10T11:02:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-09-10T11:02:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-09-10T11:02:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 313, 2024-09-10T11:02:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 1424819, 2024-09-10T11:02:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-09-10T11:02:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-09-10T11:02:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-09-10T11:02:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-09-10T11:02:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T11:02:19.785 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T11:02:19.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:19 smithi103 ceph-mon[35160]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 10 MiB/s wr, 593 op/s 2024-09-10T11:02:19.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:19 smithi133 ceph-mon[35282]: pgmap v522: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 10 MiB/s wr, 593 op/s 2024-09-10T11:02:20.543 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T11:02:20.543 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T11:02:20.543 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-10T11:02:20.543 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-10T11:02:20.543 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-10T11:02:20.544 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T11:02:20.544 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T11:02:20.802 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T11:02:21.315 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T11:02:21.316 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:21.316 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 407440, 2024-09-10T11:02:21.316 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 472909, 2024-09-10T11:02:21.316 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 124144122, 2024-09-10T11:02:21.316 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 455652643, 2024-09-10T11:02:21.316 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 9, 2024-09-10T11:02:21.316 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-09-10T11:02:21.317 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 20.206402569, 2024-09-10T11:02:21.317 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 8.510976290, 2024-09-10T11:02:21.317 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 14.277578722, 2024-09-10T11:02:21.317 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.492955655, 2024-09-10T11:02:21.317 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:21.317 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 472909, 2024-09-10T11:02:21.317 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 15.597977457, 2024-09-10T11:02:21.318 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000032983 2024-09-10T11:02:21.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.318 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:21.318 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 810305, 2024-09-10T11:02:21.318 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.133967664, 2024-09-10T11:02:21.318 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000165 2024-09-10T11:02:21.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.318 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1643312, 2024-09-10T11:02:21.319 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 193088 2024-09-10T11:02:21.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.319 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:21.319 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 15720, 2024-09-10T11:02:21.319 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 15975, 2024-09-10T11:02:21.319 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3973271, 2024-09-10T11:02:21.319 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 113168874, 2024-09-10T11:02:21.319 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 17, 2024-09-10T11:02:21.320 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 6, 2024-09-10T11:02:21.320 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 1.497343861, 2024-09-10T11:02:21.320 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.686680579, 2024-09-10T11:02:21.320 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.280712654, 2024-09-10T11:02:21.320 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.427553882, 2024-09-10T11:02:21.320 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:21.320 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 15970, 2024-09-10T11:02:21.321 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.910839633, 2024-09-10T11:02:21.321 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000057034 2024-09-10T11:02:21.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.321 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:21.321 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 65, 2024-09-10T11:02:21.321 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000012184, 2024-09-10T11:02:21.321 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000187 2024-09-10T11:02:21.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.322 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3456, 2024-09-10T11:02:21.322 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 575280 2024-09-10T11:02:21.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.322 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:21.322 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 49067, 2024-09-10T11:02:21.322 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 49449, 2024-09-10T11:02:21.322 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 15912776, 2024-09-10T11:02:21.322 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 261147342, 2024-09-10T11:02:21.323 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 8, 2024-09-10T11:02:21.323 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-09-10T11:02:21.323 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 3.536055403, 2024-09-10T11:02:21.323 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 1.448876835, 2024-09-10T11:02:21.323 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 1.325149188, 2024-09-10T11:02:21.323 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.808418580, 2024-09-10T11:02:21.323 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:21.324 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 49449, 2024-09-10T11:02:21.324 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2.025769764, 2024-09-10T11:02:21.324 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000040966 2024-09-10T11:02:21.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.324 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:21.324 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 46757, 2024-09-10T11:02:21.324 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.007747101, 2024-09-10T11:02:21.325 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000165 2024-09-10T11:02:21.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.325 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T11:02:21.325 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T11:02:21.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.325 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-10T11:02:21.325 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-10T11:02:21.325 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-10T11:02:21.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.326 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-10T11:02:21.326 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-10T11:02:21.326 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-10T11:02:21.326 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 56237, 2024-09-10T11:02:21.326 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 67.993644219, 2024-09-10T11:02:21.326 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001209055 2024-09-10T11:02:21.326 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:21.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.327 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-10T11:02:21.327 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-10T11:02:21.327 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-10T11:02:21.327 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 36785, 2024-09-10T11:02:21.327 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 4.741105476, 2024-09-10T11:02:21.327 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000128886 2024-09-10T11:02:21.327 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:21.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.328 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-10T11:02:21.328 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 147697, 2024-09-10T11:02:21.328 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 140282, 2024-09-10T11:02:21.328 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-10T11:02:21.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 140282, 2024-09-10T11:02:21.328 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 465.129839050, 2024-09-10T11:02:21.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.003315677 2024-09-10T11:02:21.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.329 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-10T11:02:21.329 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 7415, 2024-09-10T11:02:21.329 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 3, 2024-09-10T11:02:21.329 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-10T11:02:21.329 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 130, 2024-09-10T11:02:21.329 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 14, 2024-09-10T11:02:21.330 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 7, 2024-09-10T11:02:21.330 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 3686, 2024-09-10T11:02:21.330 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 422, 2024-09-10T11:02:21.330 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 2740, 2024-09-10T11:02:21.330 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 524, 2024-09-10T11:02:21.330 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 2935, 2024-09-10T11:02:21.330 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 588, 2024-09-10T11:02:21.330 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 801, 2024-09-10T11:02:21.330 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 801, 2024-09-10T11:02:21.331 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 13, 2024-09-10T11:02:21.331 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 244485, 2024-09-10T11:02:21.331 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 125316, 2024-09-10T11:02:21.331 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 7404, 2024-09-10T11:02:21.331 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 86, 2024-09-10T11:02:21.331 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-10T11:02:21.331 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-10T11:02:21.332 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 2081, 2024-09-10T11:02:21.332 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 292183, 2024-09-10T11:02:21.332 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-10T11:02:21.332 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 15, 2024-09-10T11:02:21.332 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 429, 2024-09-10T11:02:21.332 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 10, 2024-09-10T11:02:21.332 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 735, 2024-09-10T11:02:21.333 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-10T11:02:21.333 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-10T11:02:21.333 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-10T11:02:21.333 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 722, 2024-09-10T11:02:21.333 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 211956321, 2024-09-10T11:02:21.333 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 6, 2024-09-10T11:02:21.333 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T11:02:21.334 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-10T11:02:21.334 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T11:02:21.334 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-10T11:02:21.334 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-10T11:02:21.334 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T11:02:21.334 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T11:02:21.334 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-10T11:02:21.335 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 10, 2024-09-10T11:02:21.335 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 6177, 2024-09-10T11:02:21.335 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 90262, 2024-09-10T11:02:21.335 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T11:02:21.335 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T11:02:21.335 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T11:02:21.335 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 46235, 2024-09-10T11:02:21.335 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 21061, 2024-09-10T11:02:21.336 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 10617, 2024-09-10T11:02:21.336 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 102454 2024-09-10T11:02:21.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.336 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-10T11:02:21.336 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 74, 2024-09-10T11:02:21.336 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 312, 2024-09-10T11:02:21.336 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 165, 2024-09-10T11:02:21.336 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 402, 2024-09-10T11:02:21.337 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 166, 2024-09-10T11:02:21.337 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 587, 2024-09-10T11:02:21.337 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-10T11:02:21.337 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-10T11:02:21.337 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 21620, 2024-09-10T11:02:21.337 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 21033, 2024-09-10T11:02:21.337 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-10T11:02:21.338 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-10T11:02:21.338 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-10T11:02:21.338 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-10T11:02:21.338 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-10T11:02:21.338 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-10T11:02:21.338 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-10T11:02:21.338 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-10T11:02:21.338 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-10T11:02:21.338 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T11:02:21.339 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 16, 2024-09-10T11:02:21.339 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-10T11:02:21.339 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 21, 2024-09-10T11:02:21.339 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-10T11:02:21.339 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-10T11:02:21.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.339 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-10T11:02:21.340 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-10T11:02:21.340 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 126595, 2024-09-10T11:02:21.340 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 36871, 2024-09-10T11:02:21.340 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 36871, 2024-09-10T11:02:21.340 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 89724, 2024-09-10T11:02:21.340 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-10T11:02:21.340 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 12290, 2024-09-10T11:02:21.341 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 175, 2024-09-10T11:02:21.341 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 47, 2024-09-10T11:02:21.341 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 47, 2024-09-10T11:02:21.341 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 128, 2024-09-10T11:02:21.341 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 8, 2024-09-10T11:02:21.341 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-10T11:02:21.341 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 16, 2024-09-10T11:02:21.341 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 61409428, 2024-09-10T11:02:21.342 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 456086270, 2024-09-10T11:02:21.342 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 0, 2024-09-10T11:02:21.342 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-10T11:02:21.342 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 31325, 2024-09-10T11:02:21.342 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 181.336876519, 2024-09-10T11:02:21.342 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.005788886 2024-09-10T11:02:21.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.342 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 0 2024-09-10T11:02:21.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.343 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-10T11:02:21.343 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 1403, 2024-09-10T11:02:21.343 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 22881, 2024-09-10T11:02:21.343 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 21478, 2024-09-10T11:02:21.343 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 205, 2024-09-10T11:02:21.343 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 453, 2024-09-10T11:02:21.343 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 248, 2024-09-10T11:02:21.344 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 3441, 2024-09-10T11:02:21.344 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 45768, 2024-09-10T11:02:21.344 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 42327, 2024-09-10T11:02:21.344 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 801, 2024-09-10T11:02:21.344 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 24308, 2024-09-10T11:02:21.344 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 23507, 2024-09-10T11:02:21.344 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 89240, 2024-09-10T11:02:21.344 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 223520 2024-09-10T11:02:21.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.345 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-10T11:02:21.345 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 214684, 2024-09-10T11:02:21.345 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 575, 2024-09-10T11:02:21.345 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 147697, 2024-09-10T11:02:21.345 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 3081, 2024-09-10T11:02:21.345 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 14696, 2024-09-10T11:02:21.346 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-10T11:02:21.346 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 21658, 2024-09-10T11:02:21.346 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 70.934549490, 2024-09-10T11:02:21.346 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.003275212 2024-09-10T11:02:21.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.346 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-10T11:02:21.346 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 922, 2024-09-10T11:02:21.346 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 13.814082269, 2024-09-10T11:02:21.346 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.014982735 2024-09-10T11:02:21.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.347 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-10T11:02:21.347 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.347 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.347 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.347 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-10T11:02:21.347 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.348 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.348 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.348 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-10T11:02:21.348 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 44671, 2024-09-10T11:02:21.348 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 83.934994317, 2024-09-10T11:02:21.348 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001878959 2024-09-10T11:02:21.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.349 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-10T11:02:21.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.349 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.349 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-10T11:02:21.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-09-10T11:02:21.350 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000112686, 2024-09-10T11:02:21.350 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000112686 2024-09-10T11:02:21.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.350 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-10T11:02:21.350 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.350 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.350 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.351 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-10T11:02:21.351 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.351 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.351 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.351 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-10T11:02:21.351 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.352 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.352 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.353 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-10T11:02:21.353 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.353 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.353 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.353 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-10T11:02:21.353 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.354 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.354 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.354 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-10T11:02:21.354 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.354 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.354 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.355 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-10T11:02:21.355 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.355 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.355 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.355 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-10T11:02:21.355 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.355 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.356 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.356 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-10T11:02:21.356 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 46908, 2024-09-10T11:02:21.356 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 197.341927404, 2024-09-10T11:02:21.356 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.004206999 2024-09-10T11:02:21.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.356 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-10T11:02:21.357 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3980, 2024-09-10T11:02:21.357 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 15.252786749, 2024-09-10T11:02:21.357 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.003832358 2024-09-10T11:02:21.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.357 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-10T11:02:21.358 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.358 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.358 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.358 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-10T11:02:21.358 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.358 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.358 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.359 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-10T11:02:21.359 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.359 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.359 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.360 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-10T11:02:21.360 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.360 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.360 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.360 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-10T11:02:21.360 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.361 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.361 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-10T11:02:21.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 14, 2024-09-10T11:02:21.361 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.138709801, 2024-09-10T11:02:21.361 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.009907842 2024-09-10T11:02:21.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.362 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-10T11:02:21.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.362 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.362 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-10T11:02:21.362 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 508, 2024-09-10T11:02:21.363 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.460336463, 2024-09-10T11:02:21.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002874678 2024-09-10T11:02:21.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.363 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-10T11:02:21.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.363 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.363 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.364 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-10T11:02:21.364 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.364 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.364 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.364 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-10T11:02:21.364 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.365 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.365 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-10T11:02:21.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 21620, 2024-09-10T11:02:21.365 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 82.252339871, 2024-09-10T11:02:21.365 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.003804456 2024-09-10T11:02:21.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.366 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-10T11:02:21.366 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-10T11:02:21.366 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-10T11:02:21.366 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.366 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.366 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.366 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:21.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.367 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-10T11:02:21.367 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 2, 2024-09-10T11:02:21.367 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 3, 2024-09-10T11:02:21.367 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 1, 2024-09-10T11:02:21.367 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 2, 2024-09-10T11:02:21.367 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-10T11:02:21.367 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 40546, 2024-09-10T11:02:21.367 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 20273, 2024-09-10T11:02:21.368 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 981, 2024-09-10T11:02:21.368 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-10T11:02:21.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.368 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-10T11:02:21.368 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:21.368 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:21.368 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:21.368 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:21.369 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:21.369 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:21.369 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:21.369 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:21.369 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:21.369 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:21.369 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:21.370 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:21.370 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:21.370 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:21.370 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:21.370 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:21.370 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:21.370 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:21.371 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:21.371 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:21.371 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:21.371 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:21.371 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:21.371 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:21.371 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:21.371 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:21.372 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:21.372 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:21.372 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:21.372 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:21.372 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:21.372 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-10T11:02:21.372 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:21.372 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:21.373 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:21.373 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:21.373 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 2696964, 2024-09-10T11:02:21.373 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 1974, 2024-09-10T11:02:21.373 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 352, 2024-09-10T11:02:21.373 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 4, 2024-09-10T11:02:21.373 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-10T11:02:21.373 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-10T11:02:21.374 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:21.374 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:21.374 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:21.374 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:21.374 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8200, 2024-09-10T11:02:21.374 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 181, 2024-09-10T11:02:21.374 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:21.374 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:21.375 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:21.375 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-10T11:02:21.375 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 7722615, 2024-09-10T11:02:21.375 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 170431, 2024-09-10T11:02:21.375 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:21.375 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-10T11:02:21.375 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:21.376 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-10T11:02:21.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.376 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-10T11:02:21.376 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-10T11:02:21.376 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-10T11:02:21.376 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 55481, 2024-09-10T11:02:21.376 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 457534619, 2024-09-10T11:02:21.376 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 29, 2024-09-10T11:02:21.377 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 55481, 2024-09-10T11:02:21.377 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-10T11:02:21.377 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 55481, 2024-09-10T11:02:21.377 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 319.607707471, 2024-09-10T11:02:21.377 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.005760669 2024-09-10T11:02:21.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.377 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-10T11:02:21.377 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-10T11:02:21.378 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 55481, 2024-09-10T11:02:21.378 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 56744 2024-09-10T11:02:21.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 55481, 2024-09-10T11:02:21.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 810, 2024-09-10T11:02:21.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 54671, 2024-09-10T11:02:21.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-10T11:02:21.378 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-10T11:02:21.379 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 0, 2024-09-10T11:02:21.379 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 21, 2024-09-10T11:02:21.379 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 800, 2024-09-10T11:02:21.379 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 32125, 2024-09-10T11:02:21.379 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 438, 2024-09-10T11:02:21.379 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-10T11:02:21.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-10T11:02:21.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-10T11:02:21.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 14, 2024-09-10T11:02:21.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 21207, 2024-09-10T11:02:21.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-10T11:02:21.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-10T11:02:21.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-10T11:02:21.380 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 3, 2024-09-10T11:02:21.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 42, 2024-09-10T11:02:21.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-10T11:02:21.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-10T11:02:21.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-10T11:02:21.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-10T11:02:21.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-10T11:02:21.381 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-10T11:02:21.382 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T11:02:21.382 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-10T11:02:21.382 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-10T11:02:21.382 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-10T11:02:21.382 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-10T11:02:21.382 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-10T11:02:21.382 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-10T11:02:21.382 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-10T11:02:21.382 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-10T11:02:21.383 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-10T11:02:21.383 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-10T11:02:21.383 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-10T11:02:21.383 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-10T11:02:21.383 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-10T11:02:21.383 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-10T11:02:21.383 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-10T11:02:21.383 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-10T11:02:21.384 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-10T11:02:21.384 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-10T11:02:21.384 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-10T11:02:21.384 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 109, 2024-09-10T11:02:21.384 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-10T11:02:21.384 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 30, 2024-09-10T11:02:21.384 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 12, 2024-09-10T11:02:21.384 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 12, 2024-09-10T11:02:21.385 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-09-10T11:02:21.385 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-10T11:02:21.385 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 1674, 2024-09-10T11:02:21.385 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 13, 2024-09-10T11:02:21.385 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 407 2024-09-10T11:02:21.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.385 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-10T11:02:21.386 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 1, 2024-09-10T11:02:21.386 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-10T11:02:21.386 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-10T11:02:21.386 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-10T11:02:21.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.386 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-10T11:02:21.386 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 2 2024-09-10T11:02:21.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.387 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-10T11:02:21.387 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-10T11:02:21.387 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T11:02:21.387 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-10T11:02:21.387 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 10, 2024-09-10T11:02:21.387 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 21033, 2024-09-10T11:02:21.387 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 21033, 2024-09-10T11:02:21.388 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-10T11:02:21.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.388 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:21.388 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:21.388 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-10T11:02:21.388 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:21.388 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 472223, 2024-09-10T11:02:21.388 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 101506319, 2024-09-10T11:02:21.389 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:21.389 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 472223, 2024-09-10T11:02:21.389 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-10T11:02:21.389 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-10T11:02:21.389 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 472223, 2024-09-10T11:02:21.389 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 101506319, 2024-09-10T11:02:21.389 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:21.389 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.390 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.390 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.390 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:21.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.390 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-10T11:02:21.390 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:21.390 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-10T11:02:21.391 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:21.391 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-10T11:02:21.391 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-10T11:02:21.391 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:21.391 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:21.391 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 55481, 2024-09-10T11:02:21.391 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 459661928, 2024-09-10T11:02:21.391 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 55481, 2024-09-10T11:02:21.392 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 459661928, 2024-09-10T11:02:21.392 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:21.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.392 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.392 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.392 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:21.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.393 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-10T11:02:21.393 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:21.393 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-10T11:02:21.393 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:21.393 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-10T11:02:21.393 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-10T11:02:21.393 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:21.393 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:21.394 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 55481, 2024-09-10T11:02:21.394 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 55481, 2024-09-10T11:02:21.394 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 55481, 2024-09-10T11:02:21.394 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 55481, 2024-09-10T11:02:21.394 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:21.394 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.394 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.395 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.395 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:21.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.395 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-10T11:02:21.395 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:21.395 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-10T11:02:21.395 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:21.395 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 21033, 2024-09-10T11:02:21.396 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 2127557, 2024-09-10T11:02:21.396 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:21.396 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 21033, 2024-09-10T11:02:21.396 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-10T11:02:21.396 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-10T11:02:21.396 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 800, 2024-09-10T11:02:21.396 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2127557, 2024-09-10T11:02:21.396 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:21.397 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.397 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.397 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.397 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:21.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:21.397 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x562430a4a960": { 2024-09-10T11:02:21.397 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 72893, 2024-09-10T11:02:21.397 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-10T11:02:21.398 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:21.398 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 126595, 2024-09-10T11:02:21.398 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 451891966, 2024-09-10T11:02:21.398 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:21.398 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 126595, 2024-09-10T11:02:21.398 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-10T11:02:21.398 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-10T11:02:21.398 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 31325, 2024-09-10T11:02:21.399 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 451819073, 2024-09-10T11:02:21.399 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:21.399 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:21.399 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:21.399 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:21.399 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:21.399 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:21.399 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T11:02:21.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:21 smithi103 ceph-mon[35160]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 371 op/s 2024-09-10T11:02:21.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:21 smithi007 ceph-mon[30873]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 371 op/s 2024-09-10T11:02:21.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:21 smithi133 ceph-mon[35282]: pgmap v523: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 371 op/s 2024-09-10T11:02:22.184 INFO:tasks.check_counter:Daemon mds.b mds.exported=15 2024-09-10T11:02:22.185 INFO:tasks.check_counter:Daemon mds.b mds.imported=10 2024-09-10T11:02:22.185 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-10T11:02:22.185 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-09-10T11:02:22.185 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T11:02:22.185 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T11:02:22.185 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T11:02:22.510 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T11:02:23.057 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T11:02:23.058 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:23.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 3, 2024-09-10T11:02:23.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 5, 2024-09-10T11:02:23.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1342, 2024-09-10T11:02:23.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 961, 2024-09-10T11:02:23.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-10T11:02:23.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-10T11:02:23.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.004191474, 2024-09-10T11:02:23.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000031865, 2024-09-10T11:02:23.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000514383, 2024-09-10T11:02:23.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000051485, 2024-09-10T11:02:23.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:23.060 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3, 2024-09-10T11:02:23.060 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.003237218, 2024-09-10T11:02:23.060 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001079072 2024-09-10T11:02:23.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:23.060 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.060 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T11:02:23.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T11:02:23.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.061 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:23.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 39, 2024-09-10T11:02:23.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 44, 2024-09-10T11:02:23.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 3257047, 2024-09-10T11:02:23.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 105809, 2024-09-10T11:02:23.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 5, 2024-09-10T11:02:23.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-09-10T11:02:23.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.006864266, 2024-09-10T11:02:23.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.001088758, 2024-09-10T11:02:23.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.004620092, 2024-09-10T11:02:23.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000798793, 2024-09-10T11:02:23.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:23.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 42, 2024-09-10T11:02:23.063 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002754299, 2024-09-10T11:02:23.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000065578 2024-09-10T11:02:23.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:23.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.064 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.064 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T11:02:23.064 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 93664 2024-09-10T11:02:23.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.064 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:23.065 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 38, 2024-09-10T11:02:23.065 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 37, 2024-09-10T11:02:23.065 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 164396, 2024-09-10T11:02:23.065 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 11525, 2024-09-10T11:02:23.065 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-10T11:02:23.065 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-10T11:02:23.065 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.008975295, 2024-09-10T11:02:23.065 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.001538783, 2024-09-10T11:02:23.066 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.013804594, 2024-09-10T11:02:23.066 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000457997, 2024-09-10T11:02:23.066 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:23.066 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 37, 2024-09-10T11:02:23.066 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.012989554, 2024-09-10T11:02:23.066 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000351069 2024-09-10T11:02:23.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.066 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 159360, 2024-09-10T11:02:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 5792 2024-09-10T11:02:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.067 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-10T11:02:23.068 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-10T11:02:23.068 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-10T11:02:23.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.068 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-10T11:02:23.068 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-10T11:02:23.068 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-10T11:02:23.068 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 89, 2024-09-10T11:02:23.069 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002086630, 2024-09-10T11:02:23.069 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000023445 2024-09-10T11:02:23.069 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:23.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.069 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-10T11:02:23.069 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-10T11:02:23.069 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-10T11:02:23.069 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.070 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.070 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.070 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:23.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.070 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-10T11:02:23.070 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-09-10T11:02:23.070 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-09-10T11:02:23.070 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-10T11:02:23.071 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.071 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.071 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.071 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-10T11:02:23.071 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-10T11:02:23.071 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-10T11:02:23.071 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-10T11:02:23.072 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-09-10T11:02:23.072 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-10T11:02:23.072 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-10T11:02:23.073 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 646, 2024-09-10T11:02:23.073 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 38, 2024-09-10T11:02:23.073 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 608, 2024-09-10T11:02:23.073 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-10T11:02:23.073 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 592, 2024-09-10T11:02:23.073 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-10T11:02:23.073 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-09-10T11:02:23.073 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-09-10T11:02:23.074 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 12, 2024-09-10T11:02:23.074 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-09-10T11:02:23.074 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-09-10T11:02:23.074 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-10T11:02:23.074 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-10T11:02:23.074 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-10T11:02:23.074 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-10T11:02:23.075 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-10T11:02:23.075 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-10T11:02:23.075 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-10T11:02:23.075 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-10T11:02:23.075 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-10T11:02:23.075 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-10T11:02:23.075 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-10T11:02:23.076 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-10T11:02:23.076 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-10T11:02:23.076 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-10T11:02:23.076 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 732, 2024-09-10T11:02:23.076 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 212144474, 2024-09-10T11:02:23.076 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 6, 2024-09-10T11:02:23.076 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T11:02:23.077 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-10T11:02:23.077 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T11:02:23.077 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-10T11:02:23.077 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-10T11:02:23.077 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T11:02:23.077 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T11:02:23.077 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-10T11:02:23.078 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-10T11:02:23.078 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T11:02:23.078 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-09-10T11:02:23.078 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T11:02:23.078 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T11:02:23.078 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T11:02:23.078 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-09-10T11:02:23.078 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-09-10T11:02:23.079 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-09-10T11:02:23.079 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-09-10T11:02:23.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.079 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-10T11:02:23.079 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-10T11:02:23.079 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-10T11:02:23.079 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-10T11:02:23.080 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-10T11:02:23.080 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-09-10T11:02:23.080 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-09-10T11:02:23.080 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-10T11:02:23.080 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-10T11:02:23.080 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-09-10T11:02:23.080 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-09-10T11:02:23.081 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-10T11:02:23.081 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-10T11:02:23.081 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-10T11:02:23.081 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-10T11:02:23.081 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-10T11:02:23.081 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-10T11:02:23.081 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-10T11:02:23.082 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-10T11:02:23.082 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-10T11:02:23.082 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T11:02:23.082 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-10T11:02:23.082 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-10T11:02:23.082 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-10T11:02:23.083 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-10T11:02:23.083 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-10T11:02:23.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.083 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-10T11:02:23.083 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-10T11:02:23.083 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-09-10T11:02:23.083 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-09-10T11:02:23.084 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-09-10T11:02:23.084 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 1407, 2024-09-10T11:02:23.084 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-10T11:02:23.084 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-09-10T11:02:23.084 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-09-10T11:02:23.084 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-09-10T11:02:23.084 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-09-10T11:02:23.084 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-09-10T11:02:23.085 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 1, 2024-09-10T11:02:23.085 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-10T11:02:23.085 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-09-10T11:02:23.085 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 709028842, 2024-09-10T11:02:23.085 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 712270044, 2024-09-10T11:02:23.085 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 712263191, 2024-09-10T11:02:23.086 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-10T11:02:23.086 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.086 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.086 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.086 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 1407 2024-09-10T11:02:23.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.086 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-10T11:02:23.087 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 252, 2024-09-10T11:02:23.087 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 681, 2024-09-10T11:02:23.087 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 429, 2024-09-10T11:02:23.087 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 116, 2024-09-10T11:02:23.087 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 116, 2024-09-10T11:02:23.087 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-09-10T11:02:23.087 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 933, 2024-09-10T11:02:23.087 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 933, 2024-09-10T11:02:23.088 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-09-10T11:02:23.088 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-09-10T11:02:23.088 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-09-10T11:02:23.088 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-09-10T11:02:23.088 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 49308, 2024-09-10T11:02:23.088 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-09-10T11:02:23.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.088 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-10T11:02:23.089 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-09-10T11:02:23.089 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-10T11:02:23.089 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-09-10T11:02:23.089 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-09-10T11:02:23.089 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-10T11:02:23.089 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-10T11:02:23.089 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.089 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.090 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.090 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-10T11:02:23.090 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.090 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.090 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.090 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-10T11:02:23.090 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.091 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.091 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.091 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-10T11:02:23.091 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.091 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.091 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.091 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-10T11:02:23.092 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.092 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.092 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.092 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-10T11:02:23.092 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.093 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.093 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.093 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-10T11:02:23.093 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.093 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.093 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.094 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-10T11:02:23.094 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.094 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.094 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.094 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-10T11:02:23.094 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.094 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.095 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.095 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-10T11:02:23.095 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.095 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.095 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.095 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-10T11:02:23.096 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.096 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.096 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.096 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-10T11:02:23.096 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.097 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.097 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.098 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-10T11:02:23.098 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.098 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.098 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.098 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-10T11:02:23.098 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.099 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.099 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.099 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-10T11:02:23.099 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.099 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.099 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.100 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-10T11:02:23.100 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.100 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.100 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.100 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-10T11:02:23.100 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.101 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.101 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.101 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-10T11:02:23.102 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.102 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.102 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.102 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-10T11:02:23.102 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.102 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.103 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.103 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-10T11:02:23.103 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.103 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.103 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.103 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-10T11:02:23.104 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.104 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.104 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.104 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-10T11:02:23.104 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.104 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.105 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.105 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-10T11:02:23.105 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.105 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.105 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.105 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-10T11:02:23.106 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.106 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.106 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.106 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-10T11:02:23.106 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.106 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.107 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.107 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-10T11:02:23.107 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.107 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.107 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.107 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-10T11:02:23.107 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.108 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.108 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.108 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-10T11:02:23.108 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.108 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.108 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.109 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-10T11:02:23.109 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.109 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.109 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.109 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-10T11:02:23.110 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-10T11:02:23.110 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-10T11:02:23.110 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.110 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.110 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.110 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:23.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.110 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-10T11:02:23.111 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 2, 2024-09-10T11:02:23.111 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 2, 2024-09-10T11:02:23.111 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 0, 2024-09-10T11:02:23.111 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 0, 2024-09-10T11:02:23.111 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-10T11:02:23.112 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-09-10T11:02:23.112 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-09-10T11:02:23.112 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 2245, 2024-09-10T11:02:23.112 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-10T11:02:23.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.112 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-10T11:02:23.112 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:23.114 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:23.114 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:23.114 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:23.114 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:23.114 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:23.114 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:23.114 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:23.114 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:23.115 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:23.115 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:23.115 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:23.115 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:23.115 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:23.115 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:23.115 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:23.116 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:23.116 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:23.116 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:23.116 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:23.116 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:23.116 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:23.116 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:23.116 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-10T11:02:23.117 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:23.117 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:23.117 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:23.117 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:23.117 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 209674, 2024-09-10T11:02:23.117 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 405, 2024-09-10T11:02:23.117 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-10T11:02:23.117 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-10T11:02:23.118 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-10T11:02:23.118 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-10T11:02:23.118 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:23.118 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:23.118 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:23.118 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:23.118 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8200, 2024-09-10T11:02:23.118 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 181, 2024-09-10T11:02:23.119 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:23.119 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:23.119 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:23.119 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-10T11:02:23.119 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 1246477, 2024-09-10T11:02:23.119 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 12456, 2024-09-10T11:02:23.119 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:23.119 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-10T11:02:23.120 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:23.120 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-10T11:02:23.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.120 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-10T11:02:23.120 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-10T11:02:23.120 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-10T11:02:23.120 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 59, 2024-09-10T11:02:23.120 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 26, 2024-09-10T11:02:23.121 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 4, 2024-09-10T11:02:23.121 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 59, 2024-09-10T11:02:23.121 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-10T11:02:23.121 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 59, 2024-09-10T11:02:23.121 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.047999054, 2024-09-10T11:02:23.121 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000813543 2024-09-10T11:02:23.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.121 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-10T11:02:23.122 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-10T11:02:23.122 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 59, 2024-09-10T11:02:23.122 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 60 2024-09-10T11:02:23.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.122 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 59, 2024-09-10T11:02:23.122 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 59, 2024-09-10T11:02:23.122 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-09-10T11:02:23.122 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-10T11:02:23.123 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-10T11:02:23.123 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 30, 2024-09-10T11:02:23.123 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-09-10T11:02:23.123 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 26, 2024-09-10T11:02:23.123 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-09-10T11:02:23.123 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-09-10T11:02:23.123 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-10T11:02:23.124 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-10T11:02:23.124 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-10T11:02:23.124 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-10T11:02:23.124 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-09-10T11:02:23.124 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-10T11:02:23.124 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-10T11:02:23.124 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-10T11:02:23.124 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 2, 2024-09-10T11:02:23.125 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-09-10T11:02:23.125 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-10T11:02:23.125 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-10T11:02:23.125 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-10T11:02:23.125 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-10T11:02:23.125 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-10T11:02:23.125 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-10T11:02:23.125 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T11:02:23.126 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-10T11:02:23.126 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-10T11:02:23.126 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-10T11:02:23.126 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-10T11:02:23.126 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-10T11:02:23.126 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-10T11:02:23.126 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-10T11:02:23.127 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-10T11:02:23.127 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-10T11:02:23.127 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-10T11:02:23.127 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-10T11:02:23.127 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-10T11:02:23.127 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-10T11:02:23.127 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-10T11:02:23.127 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-10T11:02:23.128 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-10T11:02:23.128 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-10T11:02:23.128 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-10T11:02:23.129 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-10T11:02:23.129 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 109, 2024-09-10T11:02:23.129 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-10T11:02:23.129 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 1, 2024-09-10T11:02:23.129 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 6, 2024-09-10T11:02:23.129 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 6, 2024-09-10T11:02:23.129 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 0, 2024-09-10T11:02:23.129 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-10T11:02:23.130 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-09-10T11:02:23.131 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-09-10T11:02:23.131 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-09-10T11:02:23.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.132 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-10T11:02:23.132 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-09-10T11:02:23.132 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-10T11:02:23.132 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-10T11:02:23.132 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-10T11:02:23.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.133 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-10T11:02:23.133 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-10T11:02:23.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.133 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-10T11:02:23.133 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-10T11:02:23.133 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T11:02:23.133 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-10T11:02:23.134 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-09-10T11:02:23.134 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-09-10T11:02:23.134 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-09-10T11:02:23.134 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-10T11:02:23.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.134 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:23.134 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:23.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-10T11:02:23.135 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:23.135 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 77, 2024-09-10T11:02:23.135 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 3412360, 2024-09-10T11:02:23.135 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:23.135 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 77, 2024-09-10T11:02:23.136 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-10T11:02:23.136 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-10T11:02:23.136 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 77, 2024-09-10T11:02:23.136 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3412360, 2024-09-10T11:02:23.136 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:23.136 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.136 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.137 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.137 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:23.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.137 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-10T11:02:23.137 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:23.137 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-10T11:02:23.138 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:23.138 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-10T11:02:23.138 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-10T11:02:23.138 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:23.138 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:23.138 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 59, 2024-09-10T11:02:23.138 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 3241212, 2024-09-10T11:02:23.138 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 59, 2024-09-10T11:02:23.139 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3241212, 2024-09-10T11:02:23.139 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:23.139 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.139 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.139 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.139 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:23.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.140 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-10T11:02:23.140 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:23.140 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-10T11:02:23.140 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:23.140 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-10T11:02:23.140 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-10T11:02:23.140 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:23.141 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:23.141 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 59, 2024-09-10T11:02:23.141 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 59, 2024-09-10T11:02:23.141 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 59, 2024-09-10T11:02:23.141 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 59, 2024-09-10T11:02:23.141 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:23.141 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.141 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.142 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.142 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:23.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.142 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-10T11:02:23.142 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:23.142 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-10T11:02:23.142 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:23.143 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-10T11:02:23.143 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-10T11:02:23.143 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:23.143 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:23.143 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-10T11:02:23.144 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-10T11:02:23.144 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-10T11:02:23.144 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-10T11:02:23.144 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:23.144 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.145 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.145 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.145 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:23.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:23.145 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x5628866aa640": { 2024-09-10T11:02:23.145 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:23.146 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-10T11:02:23.146 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:23.146 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-10T11:02:23.146 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-10T11:02:23.146 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:23.146 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:23.147 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-10T11:02:23.147 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-10T11:02:23.147 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-10T11:02:23.147 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-10T11:02:23.147 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:23.147 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:23.147 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:23.147 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:23.148 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:23.148 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:23.148 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T11:02:23.807 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:23 smithi103 ceph-mon[35160]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 9.0 MiB/s wr, 285 op/s 2024-09-10T11:02:23.835 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-10T11:02:23.835 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-10T11:02:23.835 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-10T11:02:23.836 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-10T11:02:23.836 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T11:02:23.836 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T11:02:23.836 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T11:02:23.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:23 smithi007 ceph-mon[30873]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 9.0 MiB/s wr, 285 op/s 2024-09-10T11:02:23.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:23 smithi133 ceph-mon[35282]: pgmap v524: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 9.0 MiB/s wr, 285 op/s 2024-09-10T11:02:24.124 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T11:02:24.655 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T11:02:24.656 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:24.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2658, 2024-09-10T11:02:24.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2660, 2024-09-10T11:02:24.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 214682583, 2024-09-10T11:02:24.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 845251, 2024-09-10T11:02:24.656 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 8, 2024-09-10T11:02:24.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-09-10T11:02:24.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.305348476, 2024-09-10T11:02:24.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.062012670, 2024-09-10T11:02:24.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.165615838, 2024-09-10T11:02:24.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.060655743, 2024-09-10T11:02:24.657 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:24.657 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2658, 2024-09-10T11:02:24.658 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.075829465, 2024-09-10T11:02:24.658 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000028528 2024-09-10T11:02:24.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.658 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:24.658 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.658 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.658 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.659 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T11:02:24.659 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T11:02:24.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.659 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:24.659 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1057, 2024-09-10T11:02:24.659 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1280, 2024-09-10T11:02:24.659 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 142895259, 2024-09-10T11:02:24.660 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 897816, 2024-09-10T11:02:24.660 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 9, 2024-09-10T11:02:24.660 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 7, 2024-09-10T11:02:24.660 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.145186057, 2024-09-10T11:02:24.660 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.027337755, 2024-09-10T11:02:24.660 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.088163298, 2024-09-10T11:02:24.660 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.022246124, 2024-09-10T11:02:24.661 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:24.661 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1278, 2024-09-10T11:02:24.661 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.041573678, 2024-09-10T11:02:24.661 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000032530 2024-09-10T11:02:24.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.661 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:24.661 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.661 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.662 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.662 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-10T11:02:24.662 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 563536 2024-09-10T11:02:24.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.662 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:24.662 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1001, 2024-09-10T11:02:24.662 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 961, 2024-09-10T11:02:24.663 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 98260254, 2024-09-10T11:02:24.663 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 389558, 2024-09-10T11:02:24.663 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-10T11:02:24.663 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-09-10T11:02:24.663 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.146385955, 2024-09-10T11:02:24.663 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.030910600, 2024-09-10T11:02:24.663 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.111034376, 2024-09-10T11:02:24.663 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.014150309, 2024-09-10T11:02:24.664 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:24.664 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 960, 2024-09-10T11:02:24.664 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.035596170, 2024-09-10T11:02:24.664 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000037079 2024-09-10T11:02:24.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.664 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:24.664 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.665 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.665 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.665 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1627392, 2024-09-10T11:02:24.665 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 189536 2024-09-10T11:02:24.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.665 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-10T11:02:24.665 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-09-10T11:02:24.666 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-10T11:02:24.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.666 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-09-10T11:02:24.666 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-10T11:02:24.666 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-10T11:02:24.666 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 7132, 2024-09-10T11:02:24.666 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.152358307, 2024-09-10T11:02:24.666 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000021362 2024-09-10T11:02:24.667 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:24.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.667 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-09-10T11:02:24.667 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-09-10T11:02:24.667 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-09-10T11:02:24.667 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.667 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.667 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.668 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:24.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.668 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-09-10T11:02:24.668 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-09-10T11:02:24.668 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-09-10T11:02:24.668 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-09-10T11:02:24.668 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.669 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.669 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.669 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-09-10T11:02:24.669 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-09-10T11:02:24.669 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-09-10T11:02:24.669 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-09-10T11:02:24.669 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-09-10T11:02:24.670 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-09-10T11:02:24.670 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-09-10T11:02:24.670 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 22594, 2024-09-10T11:02:24.670 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 2797, 2024-09-10T11:02:24.670 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 19797, 2024-09-10T11:02:24.670 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-09-10T11:02:24.670 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 18598, 2024-09-10T11:02:24.670 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-09-10T11:02:24.671 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-09-10T11:02:24.671 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-09-10T11:02:24.671 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 6, 2024-09-10T11:02:24.671 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-09-10T11:02:24.671 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-09-10T11:02:24.671 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-09-10T11:02:24.671 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-09-10T11:02:24.671 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-09-10T11:02:24.672 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-09-10T11:02:24.672 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-09-10T11:02:24.672 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-09-10T11:02:24.672 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-09-10T11:02:24.672 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-09-10T11:02:24.672 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-09-10T11:02:24.672 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-09-10T11:02:24.673 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-09-10T11:02:24.673 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-09-10T11:02:24.673 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-09-10T11:02:24.673 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-09-10T11:02:24.673 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-09-10T11:02:24.673 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-09-10T11:02:24.673 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-09-10T11:02:24.673 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T11:02:24.674 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-09-10T11:02:24.674 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T11:02:24.674 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-09-10T11:02:24.674 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-09-10T11:02:24.674 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T11:02:24.674 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T11:02:24.674 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-09-10T11:02:24.675 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-09-10T11:02:24.675 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T11:02:24.675 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-09-10T11:02:24.675 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T11:02:24.675 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T11:02:24.675 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T11:02:24.675 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-09-10T11:02:24.675 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-09-10T11:02:24.676 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-09-10T11:02:24.676 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-09-10T11:02:24.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.676 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-09-10T11:02:24.676 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-09-10T11:02:24.676 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-09-10T11:02:24.676 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-09-10T11:02:24.676 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-09-10T11:02:24.677 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-09-10T11:02:24.677 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-09-10T11:02:24.677 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-09-10T11:02:24.677 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-09-10T11:02:24.677 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-09-10T11:02:24.677 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-09-10T11:02:24.677 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-09-10T11:02:24.677 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-09-10T11:02:24.678 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-09-10T11:02:24.678 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-09-10T11:02:24.678 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-09-10T11:02:24.678 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-09-10T11:02:24.678 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-09-10T11:02:24.678 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-09-10T11:02:24.678 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-09-10T11:02:24.679 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T11:02:24.679 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-09-10T11:02:24.679 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-09-10T11:02:24.679 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-09-10T11:02:24.679 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-09-10T11:02:24.679 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-09-10T11:02:24.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.679 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-09-10T11:02:24.680 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-09-10T11:02:24.680 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-09-10T11:02:24.680 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-09-10T11:02:24.680 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-09-10T11:02:24.680 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 126714, 2024-09-10T11:02:24.680 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-09-10T11:02:24.680 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-09-10T11:02:24.680 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-09-10T11:02:24.681 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-09-10T11:02:24.681 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-09-10T11:02:24.681 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 128, 2024-09-10T11:02:24.681 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 12, 2024-09-10T11:02:24.681 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-09-10T11:02:24.681 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-09-10T11:02:24.681 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 61409428, 2024-09-10T11:02:24.681 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 457175238, 2024-09-10T11:02:24.682 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 457166528, 2024-09-10T11:02:24.682 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-09-10T11:02:24.682 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.682 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.682 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.682 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 126714 2024-09-10T11:02:24.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.683 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-09-10T11:02:24.683 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 1347, 2024-09-10T11:02:24.683 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 22761, 2024-09-10T11:02:24.683 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 21414, 2024-09-10T11:02:24.683 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 108, 2024-09-10T11:02:24.683 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 180, 2024-09-10T11:02:24.683 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 72, 2024-09-10T11:02:24.684 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 22606, 2024-09-10T11:02:24.684 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 23065, 2024-09-10T11:02:24.684 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 459, 2024-09-10T11:02:24.684 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-09-10T11:02:24.684 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-09-10T11:02:24.684 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-09-10T11:02:24.684 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 80380, 2024-09-10T11:02:24.684 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215328 2024-09-10T11:02:24.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.685 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-09-10T11:02:24.685 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-09-10T11:02:24.685 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-09-10T11:02:24.685 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-09-10T11:02:24.685 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-09-10T11:02:24.685 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-09-10T11:02:24.686 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-09-10T11:02:24.686 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.686 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.686 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.686 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-09-10T11:02:24.686 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.686 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.687 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.687 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-09-10T11:02:24.687 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.687 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.687 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.687 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-09-10T11:02:24.688 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.688 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.688 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.688 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-09-10T11:02:24.688 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.688 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.688 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.689 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-09-10T11:02:24.689 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.689 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.689 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.689 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-09-10T11:02:24.690 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.690 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.690 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.690 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-09-10T11:02:24.690 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.690 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.690 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.691 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-09-10T11:02:24.691 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.691 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.691 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.691 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-09-10T11:02:24.692 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.692 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.692 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.692 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-09-10T11:02:24.692 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.692 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.692 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.693 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-09-10T11:02:24.694 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.694 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.694 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.694 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-09-10T11:02:24.694 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.694 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.695 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.695 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-09-10T11:02:24.695 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.695 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.695 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.695 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-09-10T11:02:24.696 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.696 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.696 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.696 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-09-10T11:02:24.696 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.696 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.697 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.697 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-09-10T11:02:24.697 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.697 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.697 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.698 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-09-10T11:02:24.698 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.698 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.698 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.698 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-09-10T11:02:24.698 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.698 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.699 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.699 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-09-10T11:02:24.699 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.699 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.699 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.699 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-09-10T11:02:24.700 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.700 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.700 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.700 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-09-10T11:02:24.700 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.700 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.701 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.701 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-09-10T11:02:24.701 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.701 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.701 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.701 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.701 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-09-10T11:02:24.702 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.702 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.702 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.702 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-09-10T11:02:24.702 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.702 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.703 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.703 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.703 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-09-10T11:02:24.703 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.703 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.703 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.703 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.703 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-09-10T11:02:24.704 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.704 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.704 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.704 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-09-10T11:02:24.704 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.704 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.704 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.705 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-09-10T11:02:24.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.705 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.705 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.705 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-09-10T11:02:24.705 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-09-10T11:02:24.706 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-09-10T11:02:24.706 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.706 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.706 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.706 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:24.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.706 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-09-10T11:02:24.707 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 2, 2024-09-10T11:02:24.707 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 3, 2024-09-10T11:02:24.707 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 1, 2024-09-10T11:02:24.707 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 3, 2024-09-10T11:02:24.707 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-09-10T11:02:24.707 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-09-10T11:02:24.707 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-09-10T11:02:24.707 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 2878, 2024-09-10T11:02:24.708 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-09-10T11:02:24.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.708 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-10T11:02:24.708 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:24.708 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:24.708 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:24.708 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:24.708 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:24.709 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:24.709 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:24.709 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:24.709 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:24.709 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:24.709 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:24.709 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:24.709 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:24.710 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:24.710 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:24.710 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:24.710 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:24.710 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:24.710 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:24.710 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:24.710 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:24.711 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:24.711 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:24.711 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:24.711 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:24.711 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:24.711 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:24.711 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:24.711 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:24.712 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:24.712 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:24.712 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-10T11:02:24.712 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:24.712 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:24.712 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:24.712 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:24.713 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 2872839, 2024-09-10T11:02:24.713 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 1792, 2024-09-10T11:02:24.713 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 88, 2024-09-10T11:02:24.713 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 1, 2024-09-10T11:02:24.713 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-10T11:02:24.713 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-10T11:02:24.713 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:24.713 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:24.714 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:24.714 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:24.714 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8200, 2024-09-10T11:02:24.714 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 181, 2024-09-10T11:02:24.714 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:24.714 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:24.714 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:24.715 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-10T11:02:24.715 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 20090660, 2024-09-10T11:02:24.715 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 221320, 2024-09-10T11:02:24.715 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:24.715 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-10T11:02:24.715 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:24.715 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-10T11:02:24.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.716 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-09-10T11:02:24.716 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-09-10T11:02:24.716 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-09-10T11:02:24.716 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 4338, 2024-09-10T11:02:24.716 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-09-10T11:02:24.716 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-09-10T11:02:24.717 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 4338, 2024-09-10T11:02:24.717 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-09-10T11:02:24.717 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4338, 2024-09-10T11:02:24.717 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 5.427892981, 2024-09-10T11:02:24.717 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001251243 2024-09-10T11:02:24.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.717 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-09-10T11:02:24.717 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-09-10T11:02:24.718 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4338, 2024-09-10T11:02:24.718 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 4339 2024-09-10T11:02:24.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.718 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 4338, 2024-09-10T11:02:24.718 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 4338, 2024-09-10T11:02:24.718 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-09-10T11:02:24.718 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-09-10T11:02:24.719 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-09-10T11:02:24.719 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 2445, 2024-09-10T11:02:24.719 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-09-10T11:02:24.719 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1891, 2024-09-10T11:02:24.719 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-09-10T11:02:24.719 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-09-10T11:02:24.719 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-09-10T11:02:24.719 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-09-10T11:02:24.720 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-09-10T11:02:24.720 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-09-10T11:02:24.720 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-09-10T11:02:24.720 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-09-10T11:02:24.720 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-09-10T11:02:24.720 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-09-10T11:02:24.720 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-09-10T11:02:24.721 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-09-10T11:02:24.721 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-09-10T11:02:24.721 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-09-10T11:02:24.721 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-09-10T11:02:24.721 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-09-10T11:02:24.721 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-09-10T11:02:24.721 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-09-10T11:02:24.721 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T11:02:24.722 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-09-10T11:02:24.722 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-09-10T11:02:24.722 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-09-10T11:02:24.722 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-09-10T11:02:24.722 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-09-10T11:02:24.722 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-09-10T11:02:24.722 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-09-10T11:02:24.722 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-09-10T11:02:24.723 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-09-10T11:02:24.723 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-09-10T11:02:24.723 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-09-10T11:02:24.723 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-09-10T11:02:24.723 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-09-10T11:02:24.723 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-09-10T11:02:24.723 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-09-10T11:02:24.724 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-09-10T11:02:24.724 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-09-10T11:02:24.724 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-09-10T11:02:24.724 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-09-10T11:02:24.724 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 109, 2024-09-10T11:02:24.724 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-09-10T11:02:24.724 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 29, 2024-09-10T11:02:24.724 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 11, 2024-09-10T11:02:24.725 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 12, 2024-09-10T11:02:24.725 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 1, 2024-09-10T11:02:24.725 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-09-10T11:02:24.725 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-09-10T11:02:24.725 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-09-10T11:02:24.725 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-09-10T11:02:24.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.726 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-09-10T11:02:24.726 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-09-10T11:02:24.726 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-09-10T11:02:24.726 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-09-10T11:02:24.726 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-09-10T11:02:24.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.726 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-09-10T11:02:24.726 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-09-10T11:02:24.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.727 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-09-10T11:02:24.727 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-09-10T11:02:24.727 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T11:02:24.727 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-09-10T11:02:24.727 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-09-10T11:02:24.727 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-09-10T11:02:24.727 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-09-10T11:02:24.727 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-09-10T11:02:24.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.728 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:24.728 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:24.728 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-10T11:02:24.728 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:24.728 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 4712, 2024-09-10T11:02:24.728 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 455392176, 2024-09-10T11:02:24.729 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:24.729 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 4712, 2024-09-10T11:02:24.729 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-10T11:02:24.729 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-10T11:02:24.729 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4712, 2024-09-10T11:02:24.729 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 455392176, 2024-09-10T11:02:24.729 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:24.729 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.730 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.730 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.730 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:24.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.730 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-09-10T11:02:24.730 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:24.730 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-10T11:02:24.730 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:24.731 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-10T11:02:24.731 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-10T11:02:24.731 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:24.731 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:24.731 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 4338, 2024-09-10T11:02:24.731 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 453014764, 2024-09-10T11:02:24.731 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4338, 2024-09-10T11:02:24.731 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 453014764, 2024-09-10T11:02:24.732 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:24.732 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.732 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.732 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.732 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:24.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.732 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-09-10T11:02:24.732 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:24.733 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-09-10T11:02:24.733 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:24.733 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-10T11:02:24.733 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-10T11:02:24.733 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:24.733 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:24.733 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 4338, 2024-09-10T11:02:24.733 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 4338, 2024-09-10T11:02:24.734 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4338, 2024-09-10T11:02:24.734 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 4338, 2024-09-10T11:02:24.734 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:24.734 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.734 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.734 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.734 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:24.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.735 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-09-10T11:02:24.735 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:24.735 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-10T11:02:24.735 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:24.735 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-10T11:02:24.735 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-10T11:02:24.735 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:24.735 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:24.736 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-10T11:02:24.736 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-10T11:02:24.736 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-10T11:02:24.736 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-10T11:02:24.736 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:24.736 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.736 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.736 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.737 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:24.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:24.737 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x563fa9558b40": { 2024-09-10T11:02:24.737 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:24.737 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-09-10T11:02:24.737 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:24.737 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-09-10T11:02:24.738 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-09-10T11:02:24.738 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:24.738 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:24.738 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-10T11:02:24.738 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-10T11:02:24.738 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-09-10T11:02:24.738 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-09-10T11:02:24.738 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:24.738 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:24.739 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:24.739 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:24.739 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:24.739 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:24.739 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T11:02:25.516 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-10T11:02:25.517 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-10T11:02:25.517 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-10T11:02:25.517 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-10T11:02:25.517 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T11:02:25.517 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T11:02:25.517 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T11:02:25.762 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T11:02:25.883 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:25 smithi103 ceph-mon[35160]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.6 MiB/s wr, 274 op/s 2024-09-10T11:02:25.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:25 smithi007 ceph-mon[30873]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.6 MiB/s wr, 274 op/s 2024-09-10T11:02:25.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:25 smithi133 ceph-mon[35282]: pgmap v525: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 8.6 MiB/s wr, 274 op/s 2024-09-10T11:02:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-10T11:02:26.290 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:26.290 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 312, 2024-09-10T11:02:26.290 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 302, 2024-09-10T11:02:26.291 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1469088, 2024-09-10T11:02:26.291 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 186160, 2024-09-10T11:02:26.291 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-10T11:02:26.291 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-10T11:02:26.291 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.046339465, 2024-09-10T11:02:26.291 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.017691859, 2024-09-10T11:02:26.291 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.041540554, 2024-09-10T11:02:26.292 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:26.292 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:26.292 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 301, 2024-09-10T11:02:26.292 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.015012834, 2024-09-10T11:02:26.292 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000049876 2024-09-10T11:02:26.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.292 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:26.293 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:26.293 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:26.293 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:26.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.293 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1469088, 2024-09-10T11:02:26.293 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 186160 2024-09-10T11:02:26.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.293 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:26.294 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 3, 2024-09-10T11:02:26.294 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 232, 2024-09-10T11:02:26.294 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 480, 2024-09-10T11:02:26.294 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 276800, 2024-09-10T11:02:26.294 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-09-10T11:02:26.294 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 3, 2024-09-10T11:02:26.294 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.014211760, 2024-09-10T11:02:26.295 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.009054300, 2024-09-10T11:02:26.295 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002471838, 2024-09-10T11:02:26.295 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:26.295 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:26.295 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 229, 2024-09-10T11:02:26.295 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.021398524, 2024-09-10T11:02:26.295 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000093443 2024-09-10T11:02:26.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.296 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:26.296 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:26.296 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:26.296 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:26.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.296 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T11:02:26.296 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 276800 2024-09-10T11:02:26.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.297 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:26.297 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2024-09-10T11:02:26.297 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 4, 2024-09-10T11:02:26.297 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2992, 2024-09-10T11:02:26.297 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 448, 2024-09-10T11:02:26.297 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-10T11:02:26.297 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-10T11:02:26.298 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.002328538, 2024-09-10T11:02:26.298 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000039926, 2024-09-10T11:02:26.298 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002275583, 2024-09-10T11:02:26.298 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:26.298 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:26.298 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3, 2024-09-10T11:02:26.298 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.004092165, 2024-09-10T11:02:26.299 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001364055 2024-09-10T11:02:26.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.299 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:26.299 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:26.299 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:26.299 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:26.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.300 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T11:02:26.300 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T11:02:26.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.300 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-10T11:02:26.300 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-10T11:02:26.300 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-10T11:02:26.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.301 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-10T11:02:26.301 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:26.301 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:26.301 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:26.301 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:26.301 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:26.301 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:26.301 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:26.302 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:26.302 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:26.302 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:26.302 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:26.302 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:26.302 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:26.302 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:26.302 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:26.303 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:26.304 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:26.304 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:26.304 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:26.304 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:26.304 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:26.304 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:26.305 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-10T11:02:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:26.306 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148754, 2024-09-10T11:02:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 31, 2024-09-10T11:02:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-10T11:02:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-10T11:02:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-10T11:02:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-10T11:02:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:26.307 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-10T11:02:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-10T11:02:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:26.308 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-10T11:02:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 928, 2024-09-10T11:02:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-10T11:02:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-10T11:02:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-10T11:02:26.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-10T11:02:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-10T11:02:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-10T11:02:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-10T11:02:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 315, 2024-09-10T11:02:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1425079, 2024-09-10T11:02:26.310 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 315, 2024-09-10T11:02:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-10T11:02:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-10T11:02:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 315, 2024-09-10T11:02:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1425079, 2024-09-10T11:02:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-10T11:02:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-10T11:02:26.311 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-10T11:02:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-10T11:02:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:26.312 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-10T11:02:26.312 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-10T11:02:26.944 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-10T11:02:26.944 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-10T11:02:26.944 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-10T11:02:26.944 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-10T11:02:26.945 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-10T11:02:26.945 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T11:02:26.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T11:02:27.302 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T11:02:27.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:27 smithi007 ceph-mon[30873]: pgmap v526: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 10 MiB/s wr, 338 op/s 2024-09-10T11:02:27.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:27 smithi133 ceph-mon[35282]: pgmap v526: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 10 MiB/s wr, 338 op/s 2024-09-10T11:02:28.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:27 smithi103 ceph-mon[35160]: pgmap v526: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 10 MiB/s wr, 338 op/s 2024-09-10T11:02:28.195 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-10T11:02:28.196 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:28.196 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 285187, 2024-09-10T11:02:28.196 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 450327, 2024-09-10T11:02:28.196 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 87867755, 2024-09-10T11:02:28.196 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 487328314, 2024-09-10T11:02:28.196 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 14, 2024-09-10T11:02:28.196 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-09-10T11:02:28.197 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 17.307918939, 2024-09-10T11:02:28.197 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 7.241135059, 2024-09-10T11:02:28.197 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 10.522413065, 2024-09-10T11:02:28.197 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.086990158, 2024-09-10T11:02:28.197 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:28.197 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 450325, 2024-09-10T11:02:28.197 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 30.876123933, 2024-09-10T11:02:28.197 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000068564 2024-09-10T11:02:28.198 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.198 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:28.198 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 635090, 2024-09-10T11:02:28.198 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.123199208, 2024-09-10T11:02:28.198 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000193 2024-09-10T11:02:28.198 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.198 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T11:02:28.198 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T11:02:28.199 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.199 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:28.199 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 34301, 2024-09-10T11:02:28.199 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 34333, 2024-09-10T11:02:28.199 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 8640752, 2024-09-10T11:02:28.199 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 188047070, 2024-09-10T11:02:28.199 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 13, 2024-09-10T11:02:28.199 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-09-10T11:02:28.200 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 3.348676170, 2024-09-10T11:02:28.200 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 1.429154896, 2024-09-10T11:02:28.200 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.592339400, 2024-09-10T11:02:28.200 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.152013360, 2024-09-10T11:02:28.200 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:28.200 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 34331, 2024-09-10T11:02:28.200 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.179919287, 2024-09-10T11:02:28.200 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000063497 2024-09-10T11:02:28.201 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.201 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:28.201 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 24, 2024-09-10T11:02:28.201 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000003736, 2024-09-10T11:02:28.201 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000155 2024-09-10T11:02:28.201 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.201 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 5920, 2024-09-10T11:02:28.201 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 126784 2024-09-10T11:02:28.202 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.202 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:28.202 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 237416, 2024-09-10T11:02:28.202 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 233771, 2024-09-10T11:02:28.202 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 71785688, 2024-09-10T11:02:28.202 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 212189708, 2024-09-10T11:02:28.202 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-09-10T11:02:28.202 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-09-10T11:02:28.203 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 14.049044990, 2024-09-10T11:02:28.203 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 6.230473445, 2024-09-10T11:02:28.203 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 8.313876655, 2024-09-10T11:02:28.203 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.104615011, 2024-09-10T11:02:28.203 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:28.203 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 233770, 2024-09-10T11:02:28.203 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 7.120185289, 2024-09-10T11:02:28.203 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000030458 2024-09-10T11:02:28.204 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.204 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:28.204 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 422677, 2024-09-10T11:02:28.204 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.061683789, 2024-09-10T11:02:28.204 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000145 2024-09-10T11:02:28.204 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.204 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1638784, 2024-09-10T11:02:28.204 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 642016 2024-09-10T11:02:28.205 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.205 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-09-10T11:02:28.205 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-09-10T11:02:28.205 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-09-10T11:02:28.205 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.205 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-09-10T11:02:28.205 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-09-10T11:02:28.205 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-09-10T11:02:28.206 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 74367, 2024-09-10T11:02:28.206 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 87.659949612, 2024-09-10T11:02:28.206 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001178747 2024-09-10T11:02:28.206 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:28.206 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.206 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-09-10T11:02:28.206 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-09-10T11:02:28.207 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-09-10T11:02:28.207 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 51429, 2024-09-10T11:02:28.207 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 7.448144987, 2024-09-10T11:02:28.207 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000144823 2024-09-10T11:02:28.207 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:28.207 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.207 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-09-10T11:02:28.207 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 181792, 2024-09-10T11:02:28.208 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 174422, 2024-09-10T11:02:28.208 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-09-10T11:02:28.208 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 174422, 2024-09-10T11:02:28.208 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 513.343070918, 2024-09-10T11:02:28.208 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002943109 2024-09-10T11:02:28.208 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.208 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-09-10T11:02:28.208 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 7370, 2024-09-10T11:02:28.209 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 1, 2024-09-10T11:02:28.209 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-09-10T11:02:28.209 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 105, 2024-09-10T11:02:28.209 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 11, 2024-09-10T11:02:28.209 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 3, 2024-09-10T11:02:28.209 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 3158, 2024-09-10T11:02:28.209 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 290, 2024-09-10T11:02:28.209 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 2844, 2024-09-10T11:02:28.210 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 24, 2024-09-10T11:02:28.210 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 2715, 2024-09-10T11:02:28.210 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 115, 2024-09-10T11:02:28.210 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 621, 2024-09-10T11:02:28.210 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 621, 2024-09-10T11:02:28.210 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 14, 2024-09-10T11:02:28.210 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 354366, 2024-09-10T11:02:28.210 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 167176, 2024-09-10T11:02:28.211 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 7368, 2024-09-10T11:02:28.211 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 36, 2024-09-10T11:02:28.211 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-09-10T11:02:28.211 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-09-10T11:02:28.211 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 3899, 2024-09-10T11:02:28.211 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 288278, 2024-09-10T11:02:28.211 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-09-10T11:02:28.211 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 2, 2024-09-10T11:02:28.212 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 70, 2024-09-10T11:02:28.212 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 10, 2024-09-10T11:02:28.212 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 632, 2024-09-10T11:02:28.212 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-09-10T11:02:28.212 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-09-10T11:02:28.212 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-09-10T11:02:28.212 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 739, 2024-09-10T11:02:28.213 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 212139398, 2024-09-10T11:02:28.213 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2024-09-10T11:02:28.213 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T11:02:28.213 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-09-10T11:02:28.213 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T11:02:28.213 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-09-10T11:02:28.213 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-09-10T11:02:28.213 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T11:02:28.214 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T11:02:28.214 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-09-10T11:02:28.214 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 19, 2024-09-10T11:02:28.214 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 5988, 2024-09-10T11:02:28.214 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 120903, 2024-09-10T11:02:28.214 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T11:02:28.214 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T11:02:28.214 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T11:02:28.215 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 36782, 2024-09-10T11:02:28.215 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 16171, 2024-09-10T11:02:28.215 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 7391, 2024-09-10T11:02:28.215 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 146932 2024-09-10T11:02:28.215 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.215 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-09-10T11:02:28.215 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 142, 2024-09-10T11:02:28.215 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 180, 2024-09-10T11:02:28.216 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 101, 2024-09-10T11:02:28.216 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 185, 2024-09-10T11:02:28.216 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 223, 2024-09-10T11:02:28.216 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 377, 2024-09-10T11:02:28.216 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-09-10T11:02:28.216 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-09-10T11:02:28.216 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 32491, 2024-09-10T11:02:28.217 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 32114, 2024-09-10T11:02:28.217 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-09-10T11:02:28.217 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-09-10T11:02:28.217 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-09-10T11:02:28.217 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-09-10T11:02:28.217 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-09-10T11:02:28.217 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-09-10T11:02:28.217 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-09-10T11:02:28.218 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-09-10T11:02:28.218 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-09-10T11:02:28.218 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T11:02:28.218 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 4, 2024-09-10T11:02:28.218 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-09-10T11:02:28.218 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 15, 2024-09-10T11:02:28.218 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-09-10T11:02:28.218 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-09-10T11:02:28.218 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.219 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-09-10T11:02:28.219 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-09-10T11:02:28.219 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 156812, 2024-09-10T11:02:28.219 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 57461, 2024-09-10T11:02:28.219 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 49161, 2024-09-10T11:02:28.219 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 107651, 2024-09-10T11:02:28.219 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-09-10T11:02:28.219 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 8300, 2024-09-10T11:02:28.220 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 199, 2024-09-10T11:02:28.220 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 71, 2024-09-10T11:02:28.220 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 60, 2024-09-10T11:02:28.220 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 139, 2024-09-10T11:02:28.220 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 9, 2024-09-10T11:02:28.220 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-09-10T11:02:28.220 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 11, 2024-09-10T11:02:28.220 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 80810403, 2024-09-10T11:02:28.221 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 490167609, 2024-09-10T11:02:28.221 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-09-10T11:02:28.221 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-09-10T11:02:28.221 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 39897, 2024-09-10T11:02:28.221 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 187.574382635, 2024-09-10T11:02:28.221 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004701465 2024-09-10T11:02:28.221 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.221 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-09-10T11:02:28.222 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.222 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-09-10T11:02:28.222 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 1449, 2024-09-10T11:02:28.222 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 33640, 2024-09-10T11:02:28.222 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 32191, 2024-09-10T11:02:28.222 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 176, 2024-09-10T11:02:28.222 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 290, 2024-09-10T11:02:28.222 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 114, 2024-09-10T11:02:28.223 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 3165, 2024-09-10T11:02:28.223 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 53824, 2024-09-10T11:02:28.223 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 50659, 2024-09-10T11:02:28.223 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 451, 2024-09-10T11:02:28.223 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 34396, 2024-09-10T11:02:28.223 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 33945, 2024-09-10T11:02:28.223 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 87768, 2024-09-10T11:02:28.223 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 223520 2024-09-10T11:02:28.224 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.224 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-09-10T11:02:28.224 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 310791, 2024-09-10T11:02:28.224 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 7, 2024-09-10T11:02:28.224 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 181792, 2024-09-10T11:02:28.224 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 3576, 2024-09-10T11:02:28.224 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 42418, 2024-09-10T11:02:28.224 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-09-10T11:02:28.225 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 32544, 2024-09-10T11:02:28.225 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 95.682471617, 2024-09-10T11:02:28.225 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002940095 2024-09-10T11:02:28.225 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.225 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-09-10T11:02:28.225 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 708, 2024-09-10T11:02:28.225 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 8.435222093, 2024-09-10T11:02:28.225 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.011914155 2024-09-10T11:02:28.226 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.226 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-09-10T11:02:28.226 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.226 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.226 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.226 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.226 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-09-10T11:02:28.226 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.226 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.227 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.227 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.227 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-09-10T11:02:28.227 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 49802, 2024-09-10T11:02:28.227 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 86.370907327, 2024-09-10T11:02:28.227 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001734285 2024-09-10T11:02:28.227 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.227 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-09-10T11:02:28.228 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.228 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.228 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.228 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.228 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-09-10T11:02:28.228 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-09-10T11:02:28.228 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000122452, 2024-09-10T11:02:28.228 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000122452 2024-09-10T11:02:28.229 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.229 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-09-10T11:02:28.229 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.229 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.229 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.229 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.229 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-09-10T11:02:28.229 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.230 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.230 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.230 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.230 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-09-10T11:02:28.230 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.231 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.231 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.231 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.231 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-09-10T11:02:28.231 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.231 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.231 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.232 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.232 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-09-10T11:02:28.232 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.232 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.232 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.232 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.232 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-09-10T11:02:28.232 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.233 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.233 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.233 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.233 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-09-10T11:02:28.233 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.233 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.233 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.233 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.234 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-09-10T11:02:28.234 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.234 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.234 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.234 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.234 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-09-10T11:02:28.234 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 51480, 2024-09-10T11:02:28.234 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 203.335989878, 2024-09-10T11:02:28.235 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003949805 2024-09-10T11:02:28.235 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.235 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-09-10T11:02:28.235 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 5712, 2024-09-10T11:02:28.235 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 22.348488565, 2024-09-10T11:02:28.235 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003912550 2024-09-10T11:02:28.235 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.235 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-09-10T11:02:28.236 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.236 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.236 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.236 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.236 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-09-10T11:02:28.236 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.236 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.236 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.237 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.237 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-09-10T11:02:28.237 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.237 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.237 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.237 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.237 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-09-10T11:02:28.237 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.238 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.238 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.238 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.238 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-09-10T11:02:28.238 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.238 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.238 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.238 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.239 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-09-10T11:02:28.239 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8, 2024-09-10T11:02:28.239 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.108155259, 2024-09-10T11:02:28.239 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.013519407 2024-09-10T11:02:28.239 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.239 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-09-10T11:02:28.239 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.239 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.240 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.240 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.240 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-09-10T11:02:28.240 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1676, 2024-09-10T11:02:28.240 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.529806215, 2024-09-10T11:02:28.240 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000912772 2024-09-10T11:02:28.241 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.241 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-09-10T11:02:28.241 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.241 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.241 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.241 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.241 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-09-10T11:02:28.241 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.242 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.242 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.242 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.242 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-09-10T11:02:28.242 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.242 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.242 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.242 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.243 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-09-10T11:02:28.243 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 32491, 2024-09-10T11:02:28.243 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 95.531907512, 2024-09-10T11:02:28.243 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002940257 2024-09-10T11:02:28.243 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.243 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-09-10T11:02:28.243 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-09-10T11:02:28.243 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-09-10T11:02:28.244 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.244 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.244 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.244 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:28.244 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.244 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-09-10T11:02:28.244 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-09-10T11:02:28.244 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 3, 2024-09-10T11:02:28.245 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 2, 2024-09-10T11:02:28.245 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-09-10T11:02:28.245 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-09-10T11:02:28.245 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 36888, 2024-09-10T11:02:28.245 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 36888, 2024-09-10T11:02:28.245 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 991, 2024-09-10T11:02:28.245 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-09-10T11:02:28.246 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.246 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-09-10T11:02:28.246 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:28.246 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:28.246 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:28.246 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:28.246 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:28.246 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:28.247 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:28.247 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:28.247 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:28.247 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:28.247 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:28.247 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:28.247 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:28.247 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:28.247 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:28.248 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:28.248 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:28.248 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:28.248 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:28.248 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:28.248 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:28.248 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:28.249 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:28.249 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:28.249 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:28.249 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:28.249 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:28.249 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:28.249 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:28.249 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:28.250 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:28.250 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-09-10T11:02:28.250 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:28.250 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:28.250 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:28.250 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:28.250 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 3877624, 2024-09-10T11:02:28.250 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 1872, 2024-09-10T11:02:28.251 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 88, 2024-09-10T11:02:28.251 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 1, 2024-09-10T11:02:28.251 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-09-10T11:02:28.251 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-09-10T11:02:28.251 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:28.251 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:28.251 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:28.251 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:28.252 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 8200, 2024-09-10T11:02:28.252 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 181, 2024-09-10T11:02:28.252 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:28.252 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:28.252 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:28.252 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-09-10T11:02:28.252 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 6785081, 2024-09-10T11:02:28.252 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 125272, 2024-09-10T11:02:28.253 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:28.253 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-09-10T11:02:28.253 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:28.253 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-09-10T11:02:28.253 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.253 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-09-10T11:02:28.253 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-09-10T11:02:28.253 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-09-10T11:02:28.254 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 75649, 2024-09-10T11:02:28.254 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 491826028, 2024-09-10T11:02:28.254 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 33, 2024-09-10T11:02:28.254 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 75649, 2024-09-10T11:02:28.254 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-09-10T11:02:28.254 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 75649, 2024-09-10T11:02:28.254 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 347.832710038, 2024-09-10T11:02:28.254 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004597981 2024-09-10T11:02:28.255 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.255 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-09-10T11:02:28.255 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-09-10T11:02:28.255 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 75649, 2024-09-10T11:02:28.255 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 77000 2024-09-10T11:02:28.255 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.255 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 75649, 2024-09-10T11:02:28.255 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 1037, 2024-09-10T11:02:28.256 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 74612, 2024-09-10T11:02:28.256 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-09-10T11:02:28.256 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-09-10T11:02:28.256 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 2, 2024-09-10T11:02:28.256 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 21, 2024-09-10T11:02:28.256 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 1036, 2024-09-10T11:02:28.256 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 40933, 2024-09-10T11:02:28.256 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 534, 2024-09-10T11:02:28.257 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-09-10T11:02:28.257 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-09-10T11:02:28.257 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-09-10T11:02:28.257 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 11, 2024-09-10T11:02:28.257 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 32271, 2024-09-10T11:02:28.257 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-09-10T11:02:28.257 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-09-10T11:02:28.257 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-09-10T11:02:28.258 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 0, 2024-09-10T11:02:28.258 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 42, 2024-09-10T11:02:28.258 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-09-10T11:02:28.258 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-09-10T11:02:28.258 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-09-10T11:02:28.258 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-09-10T11:02:28.258 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-09-10T11:02:28.258 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-09-10T11:02:28.259 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T11:02:28.259 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-09-10T11:02:28.259 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-09-10T11:02:28.259 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-09-10T11:02:28.259 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-09-10T11:02:28.259 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-09-10T11:02:28.259 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-09-10T11:02:28.259 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-09-10T11:02:28.260 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-09-10T11:02:28.260 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-09-10T11:02:28.260 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-09-10T11:02:28.260 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-09-10T11:02:28.260 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-09-10T11:02:28.260 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-09-10T11:02:28.260 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-09-10T11:02:28.261 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-09-10T11:02:28.261 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-09-10T11:02:28.261 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-09-10T11:02:28.261 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-09-10T11:02:28.261 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-09-10T11:02:28.261 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 109, 2024-09-10T11:02:28.261 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-09-10T11:02:28.261 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 30, 2024-09-10T11:02:28.262 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 12, 2024-09-10T11:02:28.262 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 12, 2024-09-10T11:02:28.262 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-09-10T11:02:28.262 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-09-10T11:02:28.262 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 1630, 2024-09-10T11:02:28.262 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2024-09-10T11:02:28.262 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 518 2024-09-10T11:02:28.262 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.263 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-09-10T11:02:28.263 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-09-10T11:02:28.263 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-09-10T11:02:28.263 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-09-10T11:02:28.263 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-09-10T11:02:28.263 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.263 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-09-10T11:02:28.263 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-09-10T11:02:28.264 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.264 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-09-10T11:02:28.264 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-09-10T11:02:28.264 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T11:02:28.264 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-09-10T11:02:28.264 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 10, 2024-09-10T11:02:28.264 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 32114, 2024-09-10T11:02:28.264 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 32114, 2024-09-10T11:02:28.265 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-09-10T11:02:28.265 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.265 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:28.265 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:28.265 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-10T11:02:28.265 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:28.265 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 556900, 2024-09-10T11:02:28.265 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 118149220, 2024-09-10T11:02:28.266 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:28.266 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 556900, 2024-09-10T11:02:28.266 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-10T11:02:28.266 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-10T11:02:28.266 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 556900, 2024-09-10T11:02:28.266 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 118149220, 2024-09-10T11:02:28.266 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:28.266 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.267 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.267 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.267 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:28.267 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.267 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-09-10T11:02:28.267 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:28.267 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-10T11:02:28.268 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:28.268 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-10T11:02:28.268 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-10T11:02:28.268 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:28.268 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:28.268 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 75649, 2024-09-10T11:02:28.268 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 495072566, 2024-09-10T11:02:28.268 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 75649, 2024-09-10T11:02:28.269 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 495072566, 2024-09-10T11:02:28.269 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:28.269 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.269 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.269 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.269 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:28.269 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.269 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-09-10T11:02:28.270 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:28.270 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-09-10T11:02:28.270 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:28.270 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-10T11:02:28.270 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-10T11:02:28.270 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:28.270 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:28.270 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 75649, 2024-09-10T11:02:28.271 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 75649, 2024-09-10T11:02:28.271 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 75649, 2024-09-10T11:02:28.271 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 75649, 2024-09-10T11:02:28.271 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:28.271 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.271 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.271 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.271 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:28.271 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.272 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-09-10T11:02:28.272 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:28.272 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-09-10T11:02:28.272 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:28.272 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 32114, 2024-09-10T11:02:28.272 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 3246554, 2024-09-10T11:02:28.272 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:28.272 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 32114, 2024-09-10T11:02:28.273 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-10T11:02:28.273 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-10T11:02:28.273 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1036, 2024-09-10T11:02:28.273 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 3246554, 2024-09-10T11:02:28.273 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:28.273 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.273 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.273 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.274 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:28.274 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:28.274 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55e8ffc6a960": { 2024-09-10T11:02:28.274 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 1677694, 2024-09-10T11:02:28.274 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-09-10T11:02:28.274 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:28.274 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 156812, 2024-09-10T11:02:28.274 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 485973305, 2024-09-10T11:02:28.275 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:28.275 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 156812, 2024-09-10T11:02:28.275 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-10T11:02:28.275 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-10T11:02:28.275 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 39897, 2024-09-10T11:02:28.275 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 484295611, 2024-09-10T11:02:28.275 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:28.275 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:28.276 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:28.276 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:28.276 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:28.276 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:28.276 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-10T11:02:29.234 INFO:tasks.check_counter:Daemon mds.c mds.exported=2 2024-09-10T11:02:29.234 INFO:tasks.check_counter:Daemon mds.c mds.imported=10 2024-09-10T11:02:29.234 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-10T11:02:29.234 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-10T11:02:29.235 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T11:02:29.235 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T11:02:29.235 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T11:02:29.485 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T11:02:29.908 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:29 smithi133 ceph-mon[35282]: pgmap v527: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 7.5 MiB/s wr, 233 op/s 2024-09-10T11:02:30.016 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-10T11:02:30.016 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:30.016 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2024-09-10T11:02:30.016 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2024-09-10T11:02:30.016 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2024-09-10T11:02:30.017 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2024-09-10T11:02:30.017 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-09-10T11:02:30.017 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2024-09-10T11:02:30.017 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.001472573, 2024-09-10T11:02:30.017 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T11:02:30.017 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000552504, 2024-09-10T11:02:30.017 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:30.018 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:30.018 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:30.018 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:30.018 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:30.018 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.018 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:30.018 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:30.018 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:30.019 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:30.019 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.019 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T11:02:30.019 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T11:02:30.019 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.019 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:30.019 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 5, 2024-09-10T11:02:30.019 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 31, 2024-09-10T11:02:30.020 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 3152, 2024-09-10T11:02:30.020 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 84032, 2024-09-10T11:02:30.020 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 3, 2024-09-10T11:02:30.020 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2024-09-10T11:02:30.020 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.006706304, 2024-09-10T11:02:30.020 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.001292864, 2024-09-10T11:02:30.020 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.005128853, 2024-09-10T11:02:30.020 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:30.021 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:30.021 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 30, 2024-09-10T11:02:30.021 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.007361535, 2024-09-10T11:02:30.021 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000245384 2024-09-10T11:02:30.021 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.021 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:30.021 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:30.021 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:30.022 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:30.022 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.022 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-10T11:02:30.022 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 84032 2024-09-10T11:02:30.022 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.022 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:30.022 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 41, 2024-09-10T11:02:30.022 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 40, 2024-09-10T11:02:30.023 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 151728, 2024-09-10T11:02:30.023 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 22896, 2024-09-10T11:02:30.023 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2024-09-10T11:02:30.023 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2024-09-10T11:02:30.023 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.005874517, 2024-09-10T11:02:30.023 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.001530413, 2024-09-10T11:02:30.023 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.006097866, 2024-09-10T11:02:30.024 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T11:02:30.024 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:30.024 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 39, 2024-09-10T11:02:30.024 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.003378792, 2024-09-10T11:02:30.024 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000086635 2024-09-10T11:02:30.024 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.024 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:30.024 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:30.025 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:30.025 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:30.025 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.025 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 151728, 2024-09-10T11:02:30.025 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 22896 2024-09-10T11:02:30.025 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.025 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-09-10T11:02:30.025 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2024-09-10T11:02:30.026 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-09-10T11:02:30.026 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.026 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-09-10T11:02:30.026 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:30.026 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:30.026 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:30.026 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:30.027 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:30.027 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:30.027 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:30.027 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:30.027 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:30.027 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:30.027 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:30.027 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:30.028 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:30.028 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:30.028 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:30.028 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:30.028 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:30.028 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:30.028 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:30.028 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:30.029 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:30.029 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:30.029 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:30.029 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:30.029 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:30.029 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:30.029 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:30.030 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:30.030 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:30.030 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:30.030 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:30.030 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-09-10T11:02:30.030 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:30.030 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:30.030 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:30.031 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:30.031 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 149070, 2024-09-10T11:02:30.031 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 26, 2024-09-10T11:02:30.031 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-09-10T11:02:30.031 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-09-10T11:02:30.031 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-09-10T11:02:30.031 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-09-10T11:02:30.031 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:30.032 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:30.032 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:30.032 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:30.032 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2024-09-10T11:02:30.032 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2024-09-10T11:02:30.032 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:30.032 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:30.033 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:30.033 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-09-10T11:02:30.033 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2024-09-10T11:02:30.033 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2024-09-10T11:02:30.033 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:30.033 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-09-10T11:02:30.033 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:30.033 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-09-10T11:02:30.034 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:30.034 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:30.034 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:30.034 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-10T11:02:30.034 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:30.034 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 42, 2024-09-10T11:02:30.034 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 145978, 2024-09-10T11:02:30.034 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:30.035 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 42, 2024-09-10T11:02:30.035 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-10T11:02:30.035 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-10T11:02:30.035 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 42, 2024-09-10T11:02:30.035 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 145978, 2024-09-10T11:02:30.035 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:30.035 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:30.035 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:30.036 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:30.036 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:30.036 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:30.036 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-10T11:02:30.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:29 smithi103 ceph-mon[35160]: pgmap v527: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 7.5 MiB/s wr, 233 op/s 2024-09-10T11:02:30.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:29 smithi007 ceph-mon[30873]: pgmap v527: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 7.5 MiB/s wr, 233 op/s 2024-09-10T11:02:30.967 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-09-10T11:02:30.967 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-09-10T11:02:30.968 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-09-10T11:02:30.968 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-09-10T11:02:30.968 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-09-10T11:02:30.968 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T11:02:30.968 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T11:02:31.220 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T11:02:31.816 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-10T11:02:31.816 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:31.816 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 190257, 2024-09-10T11:02:31.816 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 229912, 2024-09-10T11:02:31.817 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 48987830, 2024-09-10T11:02:31.817 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 351786075, 2024-09-10T11:02:31.817 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 156, 2024-09-10T11:02:31.817 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-09-10T11:02:31.817 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 12.210088645, 2024-09-10T11:02:31.817 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 5.128940431, 2024-09-10T11:02:31.817 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 6.182847657, 2024-09-10T11:02:31.818 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.505918671, 2024-09-10T11:02:31.818 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:31.818 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 229914, 2024-09-10T11:02:31.818 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 26.612053882, 2024-09-10T11:02:31.818 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000115747 2024-09-10T11:02:31.818 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.818 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:31.818 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 318881, 2024-09-10T11:02:31.819 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.051021486, 2024-09-10T11:02:31.819 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000160 2024-09-10T11:02:31.819 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.819 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 45520, 2024-09-10T11:02:31.819 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 1824 2024-09-10T11:02:31.819 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.819 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:31.819 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 257637, 2024-09-10T11:02:31.820 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 261619, 2024-09-10T11:02:31.820 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 68085672, 2024-09-10T11:02:31.820 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 313274743, 2024-09-10T11:02:31.820 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 8, 2024-09-10T11:02:31.820 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-09-10T11:02:31.820 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 15.440543671, 2024-09-10T11:02:31.820 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 7.016083598, 2024-09-10T11:02:31.820 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 8.501615505, 2024-09-10T11:02:31.821 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.302647307, 2024-09-10T11:02:31.821 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:31.821 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 261621, 2024-09-10T11:02:31.821 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 24.531420997, 2024-09-10T11:02:31.821 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000093767 2024-09-10T11:02:31.821 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.821 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:31.822 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 423128, 2024-09-10T11:02:31.822 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.059549361, 2024-09-10T11:02:31.822 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000140 2024-09-10T11:02:31.822 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.822 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1650272, 2024-09-10T11:02:31.822 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 651616 2024-09-10T11:02:31.822 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.822 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:31.823 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 414537, 2024-09-10T11:02:31.823 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 569648, 2024-09-10T11:02:31.823 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 127486156, 2024-09-10T11:02:31.823 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 626615988, 2024-09-10T11:02:31.823 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 18, 2024-09-10T11:02:31.823 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 8, 2024-09-10T11:02:31.823 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 22.810637412, 2024-09-10T11:02:31.823 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 9.899663695, 2024-09-10T11:02:31.824 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 14.395680355, 2024-09-10T11:02:31.824 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.435142321, 2024-09-10T11:02:31.824 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:31.824 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 569638, 2024-09-10T11:02:31.824 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 49.057295012, 2024-09-10T11:02:31.824 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000086120 2024-09-10T11:02:31.824 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.825 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:31.825 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 815099, 2024-09-10T11:02:31.825 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.157499519, 2024-09-10T11:02:31.825 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000193 2024-09-10T11:02:31.825 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.825 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 626832, 2024-09-10T11:02:31.826 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 327600 2024-09-10T11:02:31.826 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.826 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-09-10T11:02:31.826 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-09-10T11:02:31.826 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-09-10T11:02:31.826 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.826 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-09-10T11:02:31.826 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-09-10T11:02:31.827 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-09-10T11:02:31.827 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 151895, 2024-09-10T11:02:31.827 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 142.887416780, 2024-09-10T11:02:31.827 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000940698 2024-09-10T11:02:31.827 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:31.827 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.828 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-09-10T11:02:31.828 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-09-10T11:02:31.828 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-09-10T11:02:31.828 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 50240, 2024-09-10T11:02:31.828 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 10.261434913, 2024-09-10T11:02:31.828 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000204248 2024-09-10T11:02:31.828 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:31.829 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.829 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-09-10T11:02:31.829 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 254170, 2024-09-10T11:02:31.829 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 234160, 2024-09-10T11:02:31.829 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-09-10T11:02:31.829 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 234160, 2024-09-10T11:02:31.829 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1184.057571501, 2024-09-10T11:02:31.829 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005056617 2024-09-10T11:02:31.830 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.830 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-09-10T11:02:31.830 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 20010, 2024-09-10T11:02:31.830 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 16, 2024-09-10T11:02:31.830 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-09-10T11:02:31.830 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 3036, 2024-09-10T11:02:31.830 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 14, 2024-09-10T11:02:31.830 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 10, 2024-09-10T11:02:31.831 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 2894, 2024-09-10T11:02:31.831 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 831, 2024-09-10T11:02:31.831 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 1977, 2024-09-10T11:02:31.831 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 86, 2024-09-10T11:02:31.831 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 1707, 2024-09-10T11:02:31.831 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 2134, 2024-09-10T11:02:31.831 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 473, 2024-09-10T11:02:31.831 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 473, 2024-09-10T11:02:31.832 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 14, 2024-09-10T11:02:31.832 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 379100, 2024-09-10T11:02:31.832 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 244156, 2024-09-10T11:02:31.832 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 19996, 2024-09-10T11:02:31.832 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 93, 2024-09-10T11:02:31.832 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 3, 2024-09-10T11:02:31.832 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-09-10T11:02:31.832 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 2770, 2024-09-10T11:02:31.833 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 285473, 2024-09-10T11:02:31.833 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 9, 2024-09-10T11:02:31.833 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 17, 2024-09-10T11:02:31.833 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 1223, 2024-09-10T11:02:31.833 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 14, 2024-09-10T11:02:31.833 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 355, 2024-09-10T11:02:31.833 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-09-10T11:02:31.834 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-09-10T11:02:31.834 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-09-10T11:02:31.834 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 739, 2024-09-10T11:02:31.834 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 212139398, 2024-09-10T11:02:31.834 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2024-09-10T11:02:31.834 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T11:02:31.834 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-09-10T11:02:31.834 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T11:02:31.835 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-09-10T11:02:31.835 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-09-10T11:02:31.835 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T11:02:31.835 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T11:02:31.835 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-09-10T11:02:31.835 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 12, 2024-09-10T11:02:31.835 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 38821, 2024-09-10T11:02:31.835 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 126072, 2024-09-10T11:02:31.836 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T11:02:31.836 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T11:02:31.836 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T11:02:31.836 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 94722, 2024-09-10T11:02:31.836 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 31958, 2024-09-10T11:02:31.836 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 13705, 2024-09-10T11:02:31.837 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 132638 2024-09-10T11:02:31.837 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.837 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-09-10T11:02:31.837 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 22, 2024-09-10T11:02:31.837 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 36, 2024-09-10T11:02:31.837 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 24, 2024-09-10T11:02:31.837 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 105, 2024-09-10T11:02:31.837 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 303, 2024-09-10T11:02:31.838 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 207, 2024-09-10T11:02:31.838 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-09-10T11:02:31.838 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-09-10T11:02:31.838 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 31782, 2024-09-10T11:02:31.838 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 31575, 2024-09-10T11:02:31.838 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-09-10T11:02:31.838 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-09-10T11:02:31.839 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-09-10T11:02:31.839 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-09-10T11:02:31.839 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-09-10T11:02:31.839 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-09-10T11:02:31.839 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-09-10T11:02:31.839 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-09-10T11:02:31.839 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-09-10T11:02:31.839 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T11:02:31.840 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 19, 2024-09-10T11:02:31.840 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-09-10T11:02:31.840 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 24, 2024-09-10T11:02:31.840 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-09-10T11:02:31.840 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-09-10T11:02:31.840 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.840 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-09-10T11:02:31.840 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-09-10T11:02:31.841 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 191507, 2024-09-10T11:02:31.841 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 189326, 2024-09-10T11:02:31.841 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 189326, 2024-09-10T11:02:31.841 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 2181, 2024-09-10T11:02:31.841 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-09-10T11:02:31.841 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 4766, 2024-09-10T11:02:31.841 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 327, 2024-09-10T11:02:31.841 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 324, 2024-09-10T11:02:31.842 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 324, 2024-09-10T11:02:31.842 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 3, 2024-09-10T11:02:31.842 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 1, 2024-09-10T11:02:31.842 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-09-10T11:02:31.842 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 8, 2024-09-10T11:02:31.842 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 709028842, 2024-09-10T11:02:31.842 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 713600253, 2024-09-10T11:02:31.843 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-09-10T11:02:31.843 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-09-10T11:02:31.843 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 80766, 2024-09-10T11:02:31.843 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 371.716755759, 2024-09-10T11:02:31.843 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004602391 2024-09-10T11:02:31.843 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.843 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-09-10T11:02:31.843 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.844 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-09-10T11:02:31.844 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 1281, 2024-09-10T11:02:31.844 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 33997, 2024-09-10T11:02:31.844 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 32716, 2024-09-10T11:02:31.844 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 219, 2024-09-10T11:02:31.845 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 394, 2024-09-10T11:02:31.845 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 175, 2024-09-10T11:02:31.845 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 2897, 2024-09-10T11:02:31.845 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 70693, 2024-09-10T11:02:31.845 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 67796, 2024-09-10T11:02:31.845 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 473, 2024-09-10T11:02:31.845 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 34360, 2024-09-10T11:02:31.845 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 33887, 2024-09-10T11:02:31.846 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 94296, 2024-09-10T11:02:31.846 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 231712 2024-09-10T11:02:31.846 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.846 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-09-10T11:02:31.846 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 340076, 2024-09-10T11:02:31.846 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 55437, 2024-09-10T11:02:31.846 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 254170, 2024-09-10T11:02:31.846 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 5640, 2024-09-10T11:02:31.847 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 110660, 2024-09-10T11:02:31.847 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-09-10T11:02:31.847 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 32430, 2024-09-10T11:02:31.847 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 112.460002651, 2024-09-10T11:02:31.847 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003467776 2024-09-10T11:02:31.847 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.847 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-09-10T11:02:31.847 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 35731, 2024-09-10T11:02:31.848 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 232.491722895, 2024-09-10T11:02:31.848 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.006506723 2024-09-10T11:02:31.848 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.848 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-09-10T11:02:31.848 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.848 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.848 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.848 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.849 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-09-10T11:02:31.849 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.849 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.849 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.849 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.849 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-09-10T11:02:31.849 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 66355, 2024-09-10T11:02:31.850 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 326.928115934, 2024-09-10T11:02:31.850 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004926955 2024-09-10T11:02:31.850 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.850 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-09-10T11:02:31.850 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.850 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.850 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.850 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.851 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-09-10T11:02:31.851 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.851 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.851 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.851 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.851 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-09-10T11:02:31.851 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.851 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.852 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.852 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.852 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-09-10T11:02:31.852 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.852 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.852 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.852 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.853 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-09-10T11:02:31.853 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.853 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.854 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.854 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.854 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-09-10T11:02:31.854 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 17, 2024-09-10T11:02:31.854 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.004852940, 2024-09-10T11:02:31.854 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000285467 2024-09-10T11:02:31.854 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.854 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-09-10T11:02:31.855 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 119, 2024-09-10T11:02:31.855 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.679513087, 2024-09-10T11:02:31.855 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005710194 2024-09-10T11:02:31.855 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.855 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-09-10T11:02:31.855 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.855 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.855 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.856 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.856 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-09-10T11:02:31.856 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 17, 2024-09-10T11:02:31.856 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.873973027, 2024-09-10T11:02:31.856 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.051410178 2024-09-10T11:02:31.856 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.856 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-09-10T11:02:31.856 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 10, 2024-09-10T11:02:31.857 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.011478829, 2024-09-10T11:02:31.857 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001147882 2024-09-10T11:02:31.857 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.857 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-09-10T11:02:31.857 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 58689, 2024-09-10T11:02:31.857 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 346.879964683, 2024-09-10T11:02:31.857 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005910476 2024-09-10T11:02:31.857 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.858 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-09-10T11:02:31.858 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8378, 2024-09-10T11:02:31.858 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 25.667607123, 2024-09-10T11:02:31.858 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.003063691 2024-09-10T11:02:31.858 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.858 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-09-10T11:02:31.859 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.859 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.859 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.859 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.859 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-09-10T11:02:31.859 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.859 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.860 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.860 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.860 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-09-10T11:02:31.860 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-09-10T11:02:31.860 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000742677, 2024-09-10T11:02:31.860 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000742677 2024-09-10T11:02:31.860 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.860 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-09-10T11:02:31.861 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 11, 2024-09-10T11:02:31.861 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 5.902902873, 2024-09-10T11:02:31.861 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.536627533 2024-09-10T11:02:31.861 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.861 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-09-10T11:02:31.861 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.861 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.861 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.862 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.862 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-09-10T11:02:31.862 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 17, 2024-09-10T11:02:31.862 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.221077427, 2024-09-10T11:02:31.862 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.013004554 2024-09-10T11:02:31.862 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.862 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-09-10T11:02:31.862 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.863 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.863 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.863 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-09-10T11:02:31.863 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 596, 2024-09-10T11:02:31.863 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.701345504, 2024-09-10T11:02:31.863 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002854606 2024-09-10T11:02:31.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.864 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-09-10T11:02:31.864 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.864 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.864 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.864 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.864 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-09-10T11:02:31.864 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7, 2024-09-10T11:02:31.864 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.017675576, 2024-09-10T11:02:31.865 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002525082 2024-09-10T11:02:31.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.865 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-09-10T11:02:31.865 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.865 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.865 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.865 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-09-10T11:02:31.866 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 31782, 2024-09-10T11:02:31.866 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 130.216596275, 2024-09-10T11:02:31.866 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004097180 2024-09-10T11:02:31.866 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.866 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-09-10T11:02:31.866 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-09-10T11:02:31.866 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-09-10T11:02:31.866 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.867 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.867 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.867 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:31.867 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.867 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-09-10T11:02:31.867 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2024-09-10T11:02:31.867 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 5, 2024-09-10T11:02:31.867 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 3, 2024-09-10T11:02:31.868 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 2, 2024-09-10T11:02:31.868 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-09-10T11:02:31.868 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 41628, 2024-09-10T11:02:31.868 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 20814, 2024-09-10T11:02:31.868 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 990, 2024-09-10T11:02:31.868 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-09-10T11:02:31.868 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.868 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-09-10T11:02:31.869 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 360, 2024-09-10T11:02:31.869 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 360, 2024-09-10T11:02:31.869 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:31.869 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:31.869 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:31.869 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:31.869 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:31.870 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:31.870 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:31.870 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:31.870 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:31.870 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:31.870 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:31.870 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:31.870 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:31.871 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:31.871 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:31.871 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:31.871 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:31.871 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:31.871 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:31.871 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:31.871 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:31.872 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:31.872 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:31.872 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:31.872 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:31.872 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:31.872 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:31.872 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:31.872 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:31.873 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-09-10T11:02:31.873 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:31.873 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:31.873 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:31.873 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:31.873 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 2057897, 2024-09-10T11:02:31.873 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 1682, 2024-09-10T11:02:31.873 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-09-10T11:02:31.874 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-09-10T11:02:31.874 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-09-10T11:02:31.874 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-09-10T11:02:31.874 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:31.874 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:31.874 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:31.874 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:31.874 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 8200, 2024-09-10T11:02:31.875 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 181, 2024-09-10T11:02:31.875 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:31.875 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:31.875 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:31.875 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-09-10T11:02:31.875 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 6628224, 2024-09-10T11:02:31.875 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 117638, 2024-09-10T11:02:31.875 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:31.876 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-09-10T11:02:31.876 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:31.876 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-09-10T11:02:31.876 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.876 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-09-10T11:02:31.876 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-09-10T11:02:31.876 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-09-10T11:02:31.877 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 138299, 2024-09-10T11:02:31.877 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 742181638, 2024-09-10T11:02:31.877 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 37, 2024-09-10T11:02:31.877 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 138299, 2024-09-10T11:02:31.877 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-09-10T11:02:31.877 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 138299, 2024-09-10T11:02:31.877 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2398.625712847, 2024-09-10T11:02:31.877 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.017343767 2024-09-10T11:02:31.878 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.878 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-09-10T11:02:31.878 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-09-10T11:02:31.878 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 138299, 2024-09-10T11:02:31.878 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 183346 2024-09-10T11:02:31.878 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.878 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 138299, 2024-09-10T11:02:31.878 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 1027, 2024-09-10T11:02:31.879 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 137272, 2024-09-10T11:02:31.879 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-09-10T11:02:31.879 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-09-10T11:02:31.879 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 2783, 2024-09-10T11:02:31.879 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 18303, 2024-09-10T11:02:31.879 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 973, 2024-09-10T11:02:31.879 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 81739, 2024-09-10T11:02:31.879 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 787, 2024-09-10T11:02:31.880 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-09-10T11:02:31.880 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-09-10T11:02:31.880 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-09-10T11:02:31.880 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 14, 2024-09-10T11:02:31.880 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 32476, 2024-09-10T11:02:31.880 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-09-10T11:02:31.880 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-09-10T11:02:31.880 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-09-10T11:02:31.881 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 5, 2024-09-10T11:02:31.881 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 36606, 2024-09-10T11:02:31.881 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-09-10T11:02:31.881 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-09-10T11:02:31.881 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-09-10T11:02:31.881 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-09-10T11:02:31.881 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-09-10T11:02:31.881 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-09-10T11:02:31.882 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T11:02:31.882 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-09-10T11:02:31.882 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-09-10T11:02:31.882 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-09-10T11:02:31.882 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-09-10T11:02:31.882 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-09-10T11:02:31.882 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-09-10T11:02:31.882 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-09-10T11:02:31.883 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-09-10T11:02:31.883 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-09-10T11:02:31.883 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-09-10T11:02:31.883 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-09-10T11:02:31.883 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-09-10T11:02:31.883 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-09-10T11:02:31.883 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-09-10T11:02:31.883 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-09-10T11:02:31.884 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-09-10T11:02:31.884 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-09-10T11:02:31.884 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-09-10T11:02:31.884 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-09-10T11:02:31.884 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 109, 2024-09-10T11:02:31.884 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-09-10T11:02:31.884 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 30, 2024-09-10T11:02:31.885 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 12, 2024-09-10T11:02:31.885 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 12, 2024-09-10T11:02:31.885 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-09-10T11:02:31.885 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-09-10T11:02:31.885 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 7351, 2024-09-10T11:02:31.885 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 70, 2024-09-10T11:02:31.885 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 2239 2024-09-10T11:02:31.885 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.886 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-09-10T11:02:31.886 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-09-10T11:02:31.886 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-09-10T11:02:31.886 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-09-10T11:02:31.886 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-09-10T11:02:31.886 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.886 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-09-10T11:02:31.886 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 1 2024-09-10T11:02:31.887 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.887 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-09-10T11:02:31.887 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-09-10T11:02:31.887 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T11:02:31.887 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-09-10T11:02:31.887 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 10, 2024-09-10T11:02:31.887 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 31575, 2024-09-10T11:02:31.887 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 31575, 2024-09-10T11:02:31.888 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-09-10T11:02:31.888 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.888 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:31.888 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:31.888 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-10T11:02:31.888 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:31.888 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 862427, 2024-09-10T11:02:31.888 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 166846656, 2024-09-10T11:02:31.889 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:31.889 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 862427, 2024-09-10T11:02:31.889 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-10T11:02:31.889 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-10T11:02:31.889 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 862427, 2024-09-10T11:02:31.889 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 166846656, 2024-09-10T11:02:31.889 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:31.889 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.890 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.890 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.890 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:31.890 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.890 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-09-10T11:02:31.890 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:31.890 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-10T11:02:31.891 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:31.891 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-10T11:02:31.891 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-10T11:02:31.891 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:31.891 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:31.891 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 138299, 2024-09-10T11:02:31.891 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 745372886, 2024-09-10T11:02:31.891 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 138299, 2024-09-10T11:02:31.891 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 745372886, 2024-09-10T11:02:31.892 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:31.892 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.892 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.892 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.892 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:31.892 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.892 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-09-10T11:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-09-10T11:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-10T11:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-10T11:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:31.893 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 138299, 2024-09-10T11:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 138299, 2024-09-10T11:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 138299, 2024-09-10T11:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 138299, 2024-09-10T11:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.894 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-09-10T11:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-09-10T11:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 31575, 2024-09-10T11:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 3192595, 2024-09-10T11:02:31.895 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:31.896 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 31575, 2024-09-10T11:02:31.896 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-10T11:02:31.896 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-10T11:02:31.896 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 973, 2024-09-10T11:02:31.896 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 3192595, 2024-09-10T11:02:31.896 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:31.896 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.897 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.897 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.897 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:31.897 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:31.897 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55cb744ac960": { 2024-09-10T11:02:31.897 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:31.897 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-09-10T11:02:31.897 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:31.898 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 191507, 2024-09-10T11:02:31.898 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 709405949, 2024-09-10T11:02:31.898 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:31.898 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 191507, 2024-09-10T11:02:31.898 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-10T11:02:31.898 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-10T11:02:31.898 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 80766, 2024-09-10T11:02:31.898 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 709405949, 2024-09-10T11:02:31.899 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:31.899 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:31.899 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:31.899 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:31.899 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:31.899 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:31.899 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-10T11:02:31.900 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:31 smithi133 ceph-mon[35282]: pgmap v528: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 313 op/s 2024-09-10T11:02:31.951 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:31 smithi103 ceph-mon[35160]: pgmap v528: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 313 op/s 2024-09-10T11:02:32.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:31 smithi007 ceph-mon[30873]: pgmap v528: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 313 op/s 2024-09-10T11:02:32.771 INFO:tasks.check_counter:Daemon mds.i mds.exported=17 2024-09-10T11:02:32.772 INFO:tasks.check_counter:Daemon mds.i mds.imported=14 2024-09-10T11:02:32.772 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-10T11:02:32.772 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=17 2024-09-10T11:02:32.772 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=11 2024-09-10T11:02:32.772 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T11:02:32.772 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T11:02:33.092 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T11:02:33.620 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-10T11:02:33.620 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T11:02:33.620 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1256, 2024-09-10T11:02:33.621 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1214, 2024-09-10T11:02:33.621 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 165455696, 2024-09-10T11:02:33.621 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 472084, 2024-09-10T11:02:33.621 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-09-10T11:02:33.621 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-09-10T11:02:33.621 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.198476473, 2024-09-10T11:02:33.621 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.035859105, 2024-09-10T11:02:33.622 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.152861747, 2024-09-10T11:02:33.622 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.019857654, 2024-09-10T11:02:33.622 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:33.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1214, 2024-09-10T11:02:33.622 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.047895842, 2024-09-10T11:02:33.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000039452 2024-09-10T11:02:33.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.623 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:33.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.623 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.623 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1632288, 2024-09-10T11:02:33.623 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 194464 2024-09-10T11:02:33.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.624 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T11:02:33.624 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1275, 2024-09-10T11:02:33.624 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1506, 2024-09-10T11:02:33.624 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 184878336, 2024-09-10T11:02:33.624 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 973811, 2024-09-10T11:02:33.624 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 9, 2024-09-10T11:02:33.624 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 7, 2024-09-10T11:02:33.624 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.184039338, 2024-09-10T11:02:33.625 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.035951415, 2024-09-10T11:02:33.625 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.109501079, 2024-09-10T11:02:33.625 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.029268431, 2024-09-10T11:02:33.625 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:33.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1503, 2024-09-10T11:02:33.626 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.056191702, 2024-09-10T11:02:33.626 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000037386 2024-09-10T11:02:33.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.626 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:33.626 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.626 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.626 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.627 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T11:02:33.627 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 568992 2024-09-10T11:02:33.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.627 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T11:02:33.627 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2157, 2024-09-10T11:02:33.627 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2159, 2024-09-10T11:02:33.628 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 139076219, 2024-09-10T11:02:33.628 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 685984, 2024-09-10T11:02:33.628 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 6, 2024-09-10T11:02:33.628 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2024-09-10T11:02:33.629 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.249624804, 2024-09-10T11:02:33.629 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.049009948, 2024-09-10T11:02:33.629 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.128787393, 2024-09-10T11:02:33.629 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.056695375, 2024-09-10T11:02:33.629 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T11:02:33.629 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2157, 2024-09-10T11:02:33.630 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.073535248, 2024-09-10T11:02:33.630 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000034091 2024-09-10T11:02:33.630 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.630 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-09-10T11:02:33.630 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.630 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.631 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.631 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T11:02:33.631 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T11:02:33.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.631 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-09-10T11:02:33.632 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-09-10T11:02:33.632 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-09-10T11:02:33.632 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.632 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-09-10T11:02:33.632 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-09-10T11:02:33.632 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-09-10T11:02:33.633 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7063, 2024-09-10T11:02:33.633 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.180155391, 2024-09-10T11:02:33.633 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000025506 2024-09-10T11:02:33.633 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:33.634 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.634 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-09-10T11:02:33.634 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-09-10T11:02:33.634 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-09-10T11:02:33.634 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.635 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.635 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.635 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:33.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.635 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-09-10T11:02:33.636 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 0, 2024-09-10T11:02:33.636 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2024-09-10T11:02:33.636 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-09-10T11:02:33.636 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.637 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.637 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.637 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.637 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-09-10T11:02:33.638 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-09-10T11:02:33.638 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-09-10T11:02:33.638 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-09-10T11:02:33.638 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 0, 2024-09-10T11:02:33.638 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-09-10T11:02:33.638 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-09-10T11:02:33.638 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 33809, 2024-09-10T11:02:33.638 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 7207, 2024-09-10T11:02:33.639 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 26602, 2024-09-10T11:02:33.639 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-09-10T11:02:33.639 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 23513, 2024-09-10T11:02:33.639 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-09-10T11:02:33.639 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-09-10T11:02:33.639 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-09-10T11:02:33.639 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 10, 2024-09-10T11:02:33.639 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 0, 2024-09-10T11:02:33.639 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 0, 2024-09-10T11:02:33.640 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-09-10T11:02:33.640 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-09-10T11:02:33.640 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-09-10T11:02:33.640 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-09-10T11:02:33.640 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-09-10T11:02:33.640 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-09-10T11:02:33.640 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-09-10T11:02:33.640 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-09-10T11:02:33.641 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-09-10T11:02:33.641 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-09-10T11:02:33.641 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-09-10T11:02:33.641 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-09-10T11:02:33.641 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-09-10T11:02:33.641 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-09-10T11:02:33.641 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2024-09-10T11:02:33.641 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2024-09-10T11:02:33.642 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-09-10T11:02:33.642 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T11:02:33.642 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-09-10T11:02:33.642 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T11:02:33.642 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-09-10T11:02:33.642 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-09-10T11:02:33.642 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T11:02:33.642 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T11:02:33.643 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-09-10T11:02:33.643 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-09-10T11:02:33.643 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T11:02:33.643 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-09-10T11:02:33.643 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T11:02:33.643 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T11:02:33.644 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T11:02:33.644 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2024-09-10T11:02:33.644 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-09-10T11:02:33.644 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-09-10T11:02:33.644 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2024-09-10T11:02:33.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.644 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-09-10T11:02:33.645 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-09-10T11:02:33.645 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-09-10T11:02:33.645 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-09-10T11:02:33.645 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-09-10T11:02:33.645 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-09-10T11:02:33.645 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-09-10T11:02:33.645 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-09-10T11:02:33.645 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-09-10T11:02:33.646 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-09-10T11:02:33.646 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-09-10T11:02:33.646 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-09-10T11:02:33.646 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-09-10T11:02:33.646 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-09-10T11:02:33.646 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-09-10T11:02:33.646 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-09-10T11:02:33.646 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-09-10T11:02:33.647 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-09-10T11:02:33.647 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-09-10T11:02:33.647 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-09-10T11:02:33.647 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T11:02:33.647 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-09-10T11:02:33.647 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-09-10T11:02:33.647 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-09-10T11:02:33.648 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-09-10T11:02:33.648 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-09-10T11:02:33.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.648 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-09-10T11:02:33.648 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-09-10T11:02:33.648 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 0, 2024-09-10T11:02:33.648 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-09-10T11:02:33.648 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-09-10T11:02:33.649 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 157041, 2024-09-10T11:02:33.649 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-09-10T11:02:33.649 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-09-10T11:02:33.649 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 0, 2024-09-10T11:02:33.649 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-09-10T11:02:33.649 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-09-10T11:02:33.649 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 139, 2024-09-10T11:02:33.649 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 14, 2024-09-10T11:02:33.650 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-09-10T11:02:33.650 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-09-10T11:02:33.650 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 80810403, 2024-09-10T11:02:33.650 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 490728118, 2024-09-10T11:02:33.650 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 490718935, 2024-09-10T11:02:33.650 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-09-10T11:02:33.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.650 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.651 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.651 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 157041 2024-09-10T11:02:33.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.651 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-09-10T11:02:33.651 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 1262, 2024-09-10T11:02:33.651 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 33585, 2024-09-10T11:02:33.651 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 32323, 2024-09-10T11:02:33.652 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 128, 2024-09-10T11:02:33.652 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 155, 2024-09-10T11:02:33.652 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 27, 2024-09-10T11:02:33.652 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 33824, 2024-09-10T11:02:33.652 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 33900, 2024-09-10T11:02:33.652 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 76, 2024-09-10T11:02:33.652 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-09-10T11:02:33.653 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2024-09-10T11:02:33.653 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2024-09-10T11:02:33.653 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 85540, 2024-09-10T11:02:33.653 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215328 2024-09-10T11:02:33.653 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.653 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-09-10T11:02:33.653 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 0, 2024-09-10T11:02:33.654 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-09-10T11:02:33.654 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 0, 2024-09-10T11:02:33.654 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 0, 2024-09-10T11:02:33.654 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-09-10T11:02:33.654 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-09-10T11:02:33.654 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.654 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.654 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.655 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-09-10T11:02:33.655 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.655 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.655 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.655 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-09-10T11:02:33.655 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.655 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.656 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.656 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-09-10T11:02:33.656 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.656 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.656 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.657 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-09-10T11:02:33.657 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.657 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.657 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.657 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.658 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-09-10T11:02:33.658 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.658 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.658 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.658 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-09-10T11:02:33.658 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.658 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.659 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.659 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.659 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-09-10T11:02:33.659 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.659 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.659 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.660 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.660 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-09-10T11:02:33.660 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.660 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.660 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.660 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.660 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-09-10T11:02:33.660 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.661 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.661 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.661 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.661 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-09-10T11:02:33.661 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.661 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.662 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.662 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.662 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-09-10T11:02:33.663 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.663 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.663 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.663 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.663 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-09-10T11:02:33.663 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.664 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.664 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.664 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-09-10T11:02:33.664 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.664 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.665 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.665 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.665 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-09-10T11:02:33.665 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.665 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.665 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.666 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-09-10T11:02:33.666 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.666 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.666 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.667 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-09-10T11:02:33.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.667 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.667 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.667 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-09-10T11:02:33.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.668 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.668 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-09-10T11:02:33.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.668 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.669 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-09-10T11:02:33.669 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.669 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.669 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.669 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-09-10T11:02:33.669 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.670 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.670 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.670 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.670 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-09-10T11:02:33.670 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.670 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.670 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.670 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.671 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-09-10T11:02:33.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.671 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.671 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-09-10T11:02:33.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.671 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.672 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-09-10T11:02:33.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.672 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.673 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-09-10T11:02:33.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.673 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.674 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-09-10T11:02:33.674 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.674 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.674 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.674 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.675 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-09-10T11:02:33.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.675 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.675 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-09-10T11:02:33.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.676 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.676 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-09-10T11:02:33.676 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-09-10T11:02:33.676 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-09-10T11:02:33.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.676 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.677 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.677 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:33.677 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.677 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-09-10T11:02:33.677 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-09-10T11:02:33.677 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 3, 2024-09-10T11:02:33.677 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 2, 2024-09-10T11:02:33.678 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 3, 2024-09-10T11:02:33.678 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-09-10T11:02:33.678 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2024-09-10T11:02:33.678 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2024-09-10T11:02:33.678 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 4912, 2024-09-10T11:02:33.678 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-09-10T11:02:33.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.679 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-09-10T11:02:33.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-09-10T11:02:33.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-09-10T11:02:33.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T11:02:33.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-09-10T11:02:33.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T11:02:33.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-09-10T11:02:33.679 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T11:02:33.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T11:02:33.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T11:02:33.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T11:02:33.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T11:02:33.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-09-10T11:02:33.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T11:02:33.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T11:02:33.680 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-09-10T11:02:33.681 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-09-10T11:02:33.681 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-09-10T11:02:33.681 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-09-10T11:02:33.681 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T11:02:33.681 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T11:02:33.681 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T11:02:33.681 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T11:02:33.681 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T11:02:33.682 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-09-10T11:02:33.682 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-09-10T11:02:33.682 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-09-10T11:02:33.682 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T11:02:33.682 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T11:02:33.682 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-09-10T11:02:33.682 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-09-10T11:02:33.682 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-09-10T11:02:33.683 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-09-10T11:02:33.683 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T11:02:33.683 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-09-10T11:02:33.683 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T11:02:33.683 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-09-10T11:02:33.683 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 434259, 2024-09-10T11:02:33.683 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 1633, 2024-09-10T11:02:33.683 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 88, 2024-09-10T11:02:33.684 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 1, 2024-09-10T11:02:33.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-09-10T11:02:33.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-09-10T11:02:33.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-09-10T11:02:33.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-09-10T11:02:33.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-09-10T11:02:33.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-09-10T11:02:33.684 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 8200, 2024-09-10T11:02:33.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 181, 2024-09-10T11:02:33.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T11:02:33.685 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-09-10T11:02:33.685 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-09-10T11:02:33.685 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-09-10T11:02:33.685 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 27563577, 2024-09-10T11:02:33.685 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 211925, 2024-09-10T11:02:33.685 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-09-10T11:02:33.686 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-09-10T11:02:33.686 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-09-10T11:02:33.686 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-09-10T11:02:33.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.686 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-09-10T11:02:33.686 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-09-10T11:02:33.686 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-09-10T11:02:33.687 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 4303, 2024-09-10T11:02:33.687 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 21, 2024-09-10T11:02:33.687 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 3, 2024-09-10T11:02:33.687 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 4303, 2024-09-10T11:02:33.687 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-09-10T11:02:33.687 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4303, 2024-09-10T11:02:33.687 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 5.095907699, 2024-09-10T11:02:33.687 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001184268 2024-09-10T11:02:33.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.688 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-09-10T11:02:33.688 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-09-10T11:02:33.688 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4303, 2024-09-10T11:02:33.688 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 4304 2024-09-10T11:02:33.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.688 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 4303, 2024-09-10T11:02:33.688 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 4303, 2024-09-10T11:02:33.689 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 0, 2024-09-10T11:02:33.689 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-09-10T11:02:33.689 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-09-10T11:02:33.689 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 2408, 2024-09-10T11:02:33.689 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-09-10T11:02:33.689 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 1893, 2024-09-10T11:02:33.689 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 0, 2024-09-10T11:02:33.689 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 0, 2024-09-10T11:02:33.690 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-09-10T11:02:33.690 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-09-10T11:02:33.690 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-09-10T11:02:33.690 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-09-10T11:02:33.690 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 0, 2024-09-10T11:02:33.690 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-09-10T11:02:33.690 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-09-10T11:02:33.690 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-09-10T11:02:33.691 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 1, 2024-09-10T11:02:33.691 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-09-10T11:02:33.691 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-09-10T11:02:33.691 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-09-10T11:02:33.691 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-09-10T11:02:33.691 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-09-10T11:02:33.691 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-09-10T11:02:33.691 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-09-10T11:02:33.692 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T11:02:33.692 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-09-10T11:02:33.692 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-09-10T11:02:33.692 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-09-10T11:02:33.692 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-09-10T11:02:33.692 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-09-10T11:02:33.692 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-09-10T11:02:33.692 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-09-10T11:02:33.693 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-09-10T11:02:33.693 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-09-10T11:02:33.693 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-09-10T11:02:33.693 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-09-10T11:02:33.693 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-09-10T11:02:33.693 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-09-10T11:02:33.693 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-09-10T11:02:33.693 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-09-10T11:02:33.694 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-09-10T11:02:33.694 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-09-10T11:02:33.694 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-09-10T11:02:33.694 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-09-10T11:02:33.694 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 109, 2024-09-10T11:02:33.694 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-09-10T11:02:33.694 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 29, 2024-09-10T11:02:33.694 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 11, 2024-09-10T11:02:33.695 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 11, 2024-09-10T11:02:33.695 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-09-10T11:02:33.695 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-09-10T11:02:33.695 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 0, 2024-09-10T11:02:33.695 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2024-09-10T11:02:33.695 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 0 2024-09-10T11:02:33.695 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.696 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-09-10T11:02:33.696 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-09-10T11:02:33.696 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-09-10T11:02:33.696 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-09-10T11:02:33.696 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-09-10T11:02:33.696 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.696 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-09-10T11:02:33.696 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-09-10T11:02:33.697 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.697 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-09-10T11:02:33.697 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-09-10T11:02:33.697 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T11:02:33.697 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-09-10T11:02:33.697 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-09-10T11:02:33.698 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-09-10T11:02:33.698 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-09-10T11:02:33.698 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-09-10T11:02:33.698 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.698 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T11:02:33.698 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:33.698 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-10T11:02:33.699 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:33.699 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 4684, 2024-09-10T11:02:33.699 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 488966487, 2024-09-10T11:02:33.699 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:33.699 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 4684, 2024-09-10T11:02:33.699 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-10T11:02:33.700 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-10T11:02:33.700 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 4684, 2024-09-10T11:02:33.700 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 488966487, 2024-09-10T11:02:33.700 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:33.700 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.701 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.701 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.701 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:33.701 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.701 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-09-10T11:02:33.701 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:33.701 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-09-10T11:02:33.702 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:33.702 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-10T11:02:33.702 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-10T11:02:33.702 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:33.702 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:33.702 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 4303, 2024-09-10T11:02:33.703 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 486592455, 2024-09-10T11:02:33.703 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 4303, 2024-09-10T11:02:33.703 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 486592455, 2024-09-10T11:02:33.703 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:33.703 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.703 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.704 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.704 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:33.704 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.704 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-09-10T11:02:33.704 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:33.704 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-09-10T11:02:33.705 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:33.705 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-10T11:02:33.705 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-10T11:02:33.705 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:33.705 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:33.706 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 4303, 2024-09-10T11:02:33.706 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 4303, 2024-09-10T11:02:33.706 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 4303, 2024-09-10T11:02:33.706 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 4303, 2024-09-10T11:02:33.706 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:33.706 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.707 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.707 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.707 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:33.707 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.707 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-09-10T11:02:33.707 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:33.708 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-09-10T11:02:33.708 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:33.708 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-10T11:02:33.708 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-10T11:02:33.708 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:33.708 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:33.709 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-10T11:02:33.709 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-10T11:02:33.709 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-09-10T11:02:33.709 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-09-10T11:02:33.709 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:33.709 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.710 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.710 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.710 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:33.710 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-10T11:02:33.710 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55722d600b40": { 2024-09-10T11:02:33.710 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-09-10T11:02:33.710 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-09-10T11:02:33.710 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-09-10T11:02:33.711 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-09-10T11:02:33.711 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-09-10T11:02:33.711 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-09-10T11:02:33.711 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-09-10T11:02:33.711 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-09-10T11:02:33.711 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-09-10T11:02:33.711 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-09-10T11:02:33.711 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-09-10T11:02:33.712 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-09-10T11:02:33.712 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-09-10T11:02:33.712 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-09-10T11:02:33.712 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-09-10T11:02:33.712 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:33.712 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-10T11:02:33.712 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-10T11:02:33.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:33 smithi133 ceph-mon[35282]: pgmap v529: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 6.3 MiB/s wr, 212 op/s 2024-09-10T11:02:34.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:33 smithi103 ceph-mon[35160]: pgmap v529: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 6.3 MiB/s wr, 212 op/s 2024-09-10T11:02:34.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:33 smithi007 ceph-mon[30873]: pgmap v529: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 6.3 MiB/s wr, 212 op/s 2024-09-10T11:02:34.299 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-10T11:02:34.299 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-10T11:02:34.299 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-10T11:02:34.299 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-10T11:02:34.300 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T11:02:34.300 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T11:02:34.310 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T11:02:34.310 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T11:02:34.310 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:02:34.331 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T11:02:34.332 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:02:34.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T11:02:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:35.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.164+0000 7fb464652640 1 -- 172.21.15.7:0/2842772166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45c101de0 msgr2=0x7fb45c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:35.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.164+0000 7fb464652640 1 --2- 172.21.15.7:0/2842772166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45c101de0 0x7fb45c1021c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb458009900 tx=0x7fb45802a410 comp rx=0 tx=0).stop 2024-09-10T11:02:35.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 -- 172.21.15.7:0/2842772166 shutdown_connections 2024-09-10T11:02:35.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 --2- 172.21.15.7:0/2842772166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c106f10 0x7fb45c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:35.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 --2- 172.21.15.7:0/2842772166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb45c102790 0x7fb45c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:35.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 --2- 172.21.15.7:0/2842772166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45c101de0 0x7fb45c1021c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:35.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 -- 172.21.15.7:0/2842772166 >> 172.21.15.7:0/2842772166 conn(0x7fb45c0fd8c0 msgr2=0x7fb45c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:35.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 -- 172.21.15.7:0/2842772166 shutdown_connections 2024-09-10T11:02:35.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 -- 172.21.15.7:0/2842772166 wait complete. 2024-09-10T11:02:35.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 Processor -- start 2024-09-10T11:02:35.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 -- start start 2024-09-10T11:02:35.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb45c101de0 0x7fb45c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:35.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45c102790 0x7fb45c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:35.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c106f10 0x7fb45c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:35.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c111f70 con 0x7fb45c102790 2024-09-10T11:02:35.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c111df0 con 0x7fb45c106f10 2024-09-10T11:02:35.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.165+0000 7fb464652640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c1120f0 con 0x7fb45c101de0 2024-09-10T11:02:35.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb461bed640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45c102790 0x7fb45c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:35.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb461bed640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45c102790 0x7fb45c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53502/0 (socket says 172.21.15.7:53502) 2024-09-10T11:02:35.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb461bed640 1 -- 172.21.15.7:0/703982349 learned_addr learned my addr 172.21.15.7:0/703982349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb462bef640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c106f10 0x7fb45c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb4623ee640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb45c101de0 0x7fb45c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb461bed640 1 -- 172.21.15.7:0/703982349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb45c101de0 msgr2=0x7fb45c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb461bed640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb45c101de0 0x7fb45c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb461bed640 1 -- 172.21.15.7:0/703982349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c106f10 msgr2=0x7fb45c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb461bed640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c106f10 0x7fb45c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb461bed640 1 -- 172.21.15.7:0/703982349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c19d420 con 0x7fb45c102790 2024-09-10T11:02:35.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb461bed640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45c102790 0x7fb45c1989b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00c970 tx=0x7fb44c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:35.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb4537fe640 1 -- 172.21.15.7:0/703982349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c007bb0 con 0x7fb45c102790 2024-09-10T11:02:35.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb464652640 1 -- 172.21.15.7:0/703982349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c19d710 con 0x7fb45c102790 2024-09-10T11:02:35.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb4537fe640 1 -- 172.21.15.7:0/703982349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb44c007d50 con 0x7fb45c102790 2024-09-10T11:02:35.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb4537fe640 1 -- 172.21.15.7:0/703982349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00f450 con 0x7fb45c102790 2024-09-10T11:02:35.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.166+0000 7fb464652640 1 -- 172.21.15.7:0/703982349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c1a4ff0 con 0x7fb45c102790 2024-09-10T11:02:35.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.167+0000 7fb464652640 1 -- 172.21.15.7:0/703982349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45c1021c0 con 0x7fb45c102790 2024-09-10T11:02:35.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.168+0000 7fb4537fe640 1 -- 172.21.15.7:0/703982349 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb44c005020 con 0x7fb45c102790 2024-09-10T11:02:35.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.169+0000 7fb4537fe640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb4480a2cf0 0x7fb4480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:35.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.169+0000 7fb4623ee640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb4480a2cf0 0x7fb4480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:35.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.169+0000 7fb4537fe640 1 -- 172.21.15.7:0/703982349 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8564+0+0 (secure 0 0 0) 0x7fb44c0cf110 con 0x7fb45c102790 2024-09-10T11:02:35.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.170+0000 7fb4623ee640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb4480a2cf0 0x7fb4480a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb458009230 tx=0x7fb4580058d0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:35.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.173+0000 7fb4537fe640 1 -- 172.21.15.7:0/703982349 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb44c014030 con 0x7fb45c102790 2024-09-10T11:02:35.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.359+0000 7fb464652640 1 -- 172.21.15.7:0/703982349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb45c109b60 con 0x7fb45c102790 2024-09-10T11:02:35.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.360+0000 7fb4537fe640 1 -- 172.21.15.7:0/703982349 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v109) ==== 81+0+982 (secure 0 0 0) 0x7fb44c09b750 con 0x7fb45c102790 2024-09-10T11:02:35.362 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1740947437 2024-09-11T10:45:53.785414+0000 2024-09-10T11:02:35.362 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3214781379 2024-09-11T10:45:53.785414+0000 2024-09-10T11:02:35.362 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3099326751 2024-09-11T10:45:53.785414+0000 2024-09-10T11:02:35.362 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/959611916 2024-09-11T10:45:53.785414+0000 2024-09-10T11:02:35.362 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/11850255 2024-09-11T10:36:55.407956+0000 2024-09-10T11:02:35.362 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/2266895831 2024-09-11T10:37:20.588263+0000 2024-09-10T11:02:35.362 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3768226338 2024-09-11T10:36:55.407956+0000 2024-09-10T11:02:35.363 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/4050686422 2024-09-11T10:36:55.407956+0000 2024-09-10T11:02:35.363 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/262340568 2024-09-11T10:37:20.588263+0000 2024-09-10T11:02:35.363 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/258851660 2024-09-11T10:37:20.588263+0000 2024-09-10T11:02:35.363 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/4050686422 2024-09-11T10:36:55.407956+0000 2024-09-10T11:02:35.363 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3411487770 2024-09-11T10:36:55.407956+0000 2024-09-10T11:02:35.363 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3525096194 2024-09-11T10:45:53.785414+0000 2024-09-10T11:02:35.363 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3394895925 2024-09-11T10:37:20.588263+0000 2024-09-10T11:02:35.363 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/2266895831 2024-09-11T10:37:20.588263+0000 2024-09-10T11:02:35.364 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3525096194 2024-09-11T10:45:53.785414+0000 2024-09-10T11:02:35.364 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1846259971 2024-09-11T10:45:53.785414+0000 2024-09-10T11:02:35.364 INFO:teuthology.orchestra.run.smithi007.stderr:listed 17 entries 2024-09-10T11:02:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 -- 172.21.15.7:0/703982349 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb4480a2cf0 msgr2=0x7fb4480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb4480a2cf0 0x7fb4480a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb458009230 tx=0x7fb4580058d0 comp rx=0 tx=0).stop 2024-09-10T11:02:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 -- 172.21.15.7:0/703982349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45c102790 msgr2=0x7fb45c1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45c102790 0x7fb45c1989b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00c970 tx=0x7fb44c00ce30 comp rx=0 tx=0).stop 2024-09-10T11:02:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 -- 172.21.15.7:0/703982349 shutdown_connections 2024-09-10T11:02:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb45c106f10 0x7fb45c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fb4480a2cf0 0x7fb4480a51b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:35.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45c102790 0x7fb45c1989b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:35.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 --2- 172.21.15.7:0/703982349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb45c101de0 0x7fb45c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:35.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 -- 172.21.15.7:0/703982349 >> 172.21.15.7:0/703982349 conn(0x7fb45c0fd8c0 msgr2=0x7fb45c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:35.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 -- 172.21.15.7:0/703982349 shutdown_connections 2024-09-10T11:02:35.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:35.363+0000 7fb464652640 1 -- 172.21.15.7:0/703982349 wait complete. 2024-09-10T11:02:35.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:35 smithi007 ceph-mon[30873]: pgmap v530: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 5.8 MiB/s wr, 194 op/s 2024-09-10T11:02:35.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:35 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/703982349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T11:02:35.974 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:35 smithi103 ceph-mon[35160]: pgmap v530: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 5.8 MiB/s wr, 194 op/s 2024-09-10T11:02:35.974 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:35 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/703982349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T11:02:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:35 smithi133 ceph-mon[35282]: pgmap v530: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 5.8 MiB/s wr, 194 op/s 2024-09-10T11:02:35.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:35 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/703982349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T11:02:36.266 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T11:02:36.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:02:36.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T11:02:36.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T11:02:36.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:02:36.760 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:02:36.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:02:36.852 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T11:02:36.895 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:02:36.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:02:36.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:02:36.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:02:36.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-10T11:02:37.091 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T11:02:37.102 INFO:tasks.cephadm:Teardown begin 2024-09-10T11:02:37.102 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:02:37.102 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T11:02:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:37.884 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:37 smithi007 ceph-mon[30873]: pgmap v531: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 7.2 MiB/s wr, 272 op/s 2024-09-10T11:02:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.884+0000 7f03a60e5640 1 -- 172.21.15.7:0/2679878218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a0106f10 msgr2=0x7f03a010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.884+0000 7f03a60e5640 1 --2- 172.21.15.7:0/2679878218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a0106f10 0x7f03a010ea90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0398009ca0 tx=0x7f039802a8c0 comp rx=0 tx=0).stop 2024-09-10T11:02:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.884+0000 7f03a60e5640 1 -- 172.21.15.7:0/2679878218 shutdown_connections 2024-09-10T11:02:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.884+0000 7f03a60e5640 1 --2- 172.21.15.7:0/2679878218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a0106f10 0x7f03a010ea90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.884+0000 7f03a60e5640 1 --2- 172.21.15.7:0/2679878218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a0102790 0x7f03a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.884+0000 7f03a60e5640 1 --2- 172.21.15.7:0/2679878218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a0101de0 0x7f03a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.884+0000 7f03a60e5640 1 -- 172.21.15.7:0/2679878218 >> 172.21.15.7:0/2679878218 conn(0x7f03a00fd8c0 msgr2=0x7f03a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.884+0000 7f03a60e5640 1 -- 172.21.15.7:0/2679878218 shutdown_connections 2024-09-10T11:02:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.884+0000 7f03a60e5640 1 -- 172.21.15.7:0/2679878218 wait complete. 2024-09-10T11:02:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f03a60e5640 1 Processor -- start 2024-09-10T11:02:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f03a60e5640 1 -- start start 2024-09-10T11:02:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f03a60e5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a0101de0 0x7f03a0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f03a60e5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a0102790 0x7f03a0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f03a60e5640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a0106f10 0x7f03a019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f03a60e5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03a0111f70 con 0x7f03a0102790 2024-09-10T11:02:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f03a60e5640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03a0111df0 con 0x7f03a0106f10 2024-09-10T11:02:37.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f03a60e5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f03a01120f0 con 0x7f03a0101de0 2024-09-10T11:02:37.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f039effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a0102790 0x7f03a0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:37.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f039effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a0102790 0x7f03a0198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53520/0 (socket says 172.21.15.7:53520) 2024-09-10T11:02:37.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f039effd640 1 -- 172.21.15.7:0/1812757208 learned_addr learned my addr 172.21.15.7:0/1812757208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:37.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f039f7fe640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a0101de0 0x7f03a0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:37.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.885+0000 7f039ffff640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a0106f10 0x7f03a019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:37.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.886+0000 7f039effd640 1 -- 172.21.15.7:0/1812757208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a0101de0 msgr2=0x7f03a0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:37.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.886+0000 7f039effd640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a0101de0 0x7f03a0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:37.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.886+0000 7f039effd640 1 -- 172.21.15.7:0/1812757208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a0106f10 msgr2=0x7f03a019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:37.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.886+0000 7f039effd640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a0106f10 0x7f03a019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:37.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.886+0000 7f039effd640 1 -- 172.21.15.7:0/1812757208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a019d3c0 con 0x7f03a0102790 2024-09-10T11:02:37.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.886+0000 7f039effd640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a0102790 0x7f03a0198950 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f039400c970 tx=0x7f039400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:37.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.886+0000 7f039cff9640 1 -- 172.21.15.7:0/1812757208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394007bb0 con 0x7f03a0102790 2024-09-10T11:02:37.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.886+0000 7f039cff9640 1 -- 172.21.15.7:0/1812757208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0394007d50 con 0x7f03a0102790 2024-09-10T11:02:37.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.886+0000 7f03a60e5640 1 -- 172.21.15.7:0/1812757208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a019d6b0 con 0x7f03a0102790 2024-09-10T11:02:37.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.886+0000 7f039cff9640 1 -- 172.21.15.7:0/1812757208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039400f450 con 0x7f03a0102790 2024-09-10T11:02:37.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.887+0000 7f03a60e5640 1 -- 172.21.15.7:0/1812757208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a01a4f90 con 0x7f03a0102790 2024-09-10T11:02:37.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.888+0000 7f03a60e5640 1 -- 172.21.15.7:0/1812757208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a01021c0 con 0x7f03a0102790 2024-09-10T11:02:37.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.888+0000 7f039cff9640 1 -- 172.21.15.7:0/1812757208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0394004fe0 con 0x7f03a0102790 2024-09-10T11:02:37.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.889+0000 7f039cff9640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03880a2d70 0x7f03880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:37.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.889+0000 7f039cff9640 1 -- 172.21.15.7:0/1812757208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8564+0+0 (secure 0 0 0) 0x7f03940cedb0 con 0x7f03a0102790 2024-09-10T11:02:37.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.889+0000 7f039f7fe640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03880a2d70 0x7f03880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:37.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.889+0000 7f039f7fe640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03880a2d70 0x7f03880a5230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0390009bd0 tx=0x7f0390005f90 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:37.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:37.893+0000 7f039cff9640 1 -- 172.21.15.7:0/1812757208 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03940d3070 con 0x7f03a0102790 2024-09-10T11:02:37.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:37 smithi103 ceph-mon[35160]: pgmap v531: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 7.2 MiB/s wr, 272 op/s 2024-09-10T11:02:37.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:37 smithi133 ceph-mon[35282]: pgmap v531: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 7.2 MiB/s wr, 272 op/s 2024-09-10T11:02:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.103+0000 7f03a60e5640 1 -- 172.21.15.7:0/1812757208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03a010a180 con 0x7f03a0102790 2024-09-10T11:02:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.104+0000 7f039cff9640 1 -- 172.21.15.7:0/1812757208 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11913 (secure 0 0 0) 0x7f0394014030 con 0x7f03a0102790 2024-09-10T11:02:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T11:02:38.106 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"btime":"2024-09-10T11:02:08:650543+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1984917528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1984917528},{"type":"v1","addr":"172.21.15.7:6839","nonce":1984917528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1887416712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1887416712},{"type":"v1","addr":"172.21.15.7:6841","nonce":1887416712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":16425,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/364818207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":364818207},{"type":"v1","addr":"172.21.15.133:6835","nonce":364818207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:44:11.549269+0000","modified":"2024-09-10T11:02:08.650513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/2537001809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2537001809},{"type":"v1","addr":"172.21.15.103:6839","nonce":2537001809}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26177":{"gid":26177,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/1460675138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1460675138},{"type":"v1","addr":"172.21.15.103:6835","nonce":1460675138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24499,24482,24481]},"id":1}]} 2024-09-10T11:02:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 53 2024-09-10T11:02:38.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 -- 172.21.15.7:0/1812757208 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03880a2d70 msgr2=0x7f03880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:38.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03880a2d70 0x7f03880a5230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0390009bd0 tx=0x7f0390005f90 comp rx=0 tx=0).stop 2024-09-10T11:02:38.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 -- 172.21.15.7:0/1812757208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a0102790 msgr2=0x7f03a0198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:38.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a0102790 0x7f03a0198950 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f039400c970 tx=0x7f039400ce30 comp rx=0 tx=0).stop 2024-09-10T11:02:38.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 -- 172.21.15.7:0/1812757208 shutdown_connections 2024-09-10T11:02:38.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03a0106f10 0x7f03a019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:38.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f03880a2d70 0x7f03880a5230 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:38.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a0102790 0x7f03a0198950 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:38.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 --2- 172.21.15.7:0/1812757208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03a0101de0 0x7f03a0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:38.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 -- 172.21.15.7:0/1812757208 >> 172.21.15.7:0/1812757208 conn(0x7f03a00fd8c0 msgr2=0x7f03a00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:38.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 -- 172.21.15.7:0/1812757208 shutdown_connections 2024-09-10T11:02:38.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:38.106+0000 7f03a60e5640 1 -- 172.21.15.7:0/1812757208 wait complete. 2024-09-10T11:02:38.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T11:02:38.920 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:38.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:38 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1812757208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T11:02:38.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:38 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1812757208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T11:02:39.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:38 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1812757208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T11:02:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.357+0000 7f2434a45640 1 -- 172.21.15.7:0/2946101916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2430106f10 msgr2=0x7f243010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.357+0000 7f2434a45640 1 --2- 172.21.15.7:0/2946101916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2430106f10 0x7f243010ea90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2428009ca0 tx=0x7f242802a8c0 comp rx=0 tx=0).stop 2024-09-10T11:02:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.357+0000 7f2434a45640 1 -- 172.21.15.7:0/2946101916 shutdown_connections 2024-09-10T11:02:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.357+0000 7f2434a45640 1 --2- 172.21.15.7:0/2946101916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2430106f10 0x7f243010ea90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.357+0000 7f2434a45640 1 --2- 172.21.15.7:0/2946101916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2430102790 0x7f24301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.357+0000 7f2434a45640 1 --2- 172.21.15.7:0/2946101916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2430101de0 0x7f24301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.357+0000 7f2434a45640 1 -- 172.21.15.7:0/2946101916 >> 172.21.15.7:0/2946101916 conn(0x7f24300fd8c0 msgr2=0x7f24300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.357+0000 7f2434a45640 1 -- 172.21.15.7:0/2946101916 shutdown_connections 2024-09-10T11:02:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.358+0000 7f2434a45640 1 -- 172.21.15.7:0/2946101916 wait complete. 2024-09-10T11:02:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f2434a45640 1 Processor -- start 2024-09-10T11:02:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f2434a45640 1 -- start start 2024-09-10T11:02:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f2434a45640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2430101de0 0x7f2430198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f2434a45640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2430102790 0x7f24301989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f2434a45640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2430106f10 0x7f243019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f2434a45640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2430111f90 con 0x7f2430102790 2024-09-10T11:02:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f2434a45640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2430111e10 con 0x7f2430101de0 2024-09-10T11:02:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f2434a45640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2430112110 con 0x7f2430106f10 2024-09-10T11:02:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2430102790 0x7f24301989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2430102790 0x7f24301989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53538/0 (socket says 172.21.15.7:53538) 2024-09-10T11:02:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242dd9b640 1 -- 172.21.15.7:0/2463005202 learned_addr learned my addr 172.21.15.7:0/2463005202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242ed9d640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2430106f10 0x7f243019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242e59c640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2430101de0 0x7f2430198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242dd9b640 1 -- 172.21.15.7:0/2463005202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2430106f10 msgr2=0x7f243019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242dd9b640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2430106f10 0x7f243019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242dd9b640 1 -- 172.21.15.7:0/2463005202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2430101de0 msgr2=0x7f2430198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242dd9b640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2430101de0 0x7f2430198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242dd9b640 1 -- 172.21.15.7:0/2463005202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243019d410 con 0x7f2430102790 2024-09-10T11:02:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242ed9d640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2430106f10 0x7f243019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:02:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.359+0000 7f242e59c640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2430101de0 0x7f2430198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:02:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.360+0000 7f242dd9b640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2430102790 0x7f24301989a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f241800d950 tx=0x7f241800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.360+0000 7f24277fe640 1 -- 172.21.15.7:0/2463005202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418017070 con 0x7f2430102790 2024-09-10T11:02:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.360+0000 7f2434a45640 1 -- 172.21.15.7:0/2463005202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243019d700 con 0x7f2430102790 2024-09-10T11:02:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.360+0000 7f24277fe640 1 -- 172.21.15.7:0/2463005202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2418009c30 con 0x7f2430102790 2024-09-10T11:02:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.360+0000 7f24277fe640 1 -- 172.21.15.7:0/2463005202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241800f6b0 con 0x7f2430102790 2024-09-10T11:02:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.360+0000 7f2434a45640 1 -- 172.21.15.7:0/2463005202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24301a4fe0 con 0x7f2430102790 2024-09-10T11:02:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.361+0000 7f2434a45640 1 -- 172.21.15.7:0/2463005202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24301021c0 con 0x7f2430102790 2024-09-10T11:02:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.362+0000 7f24277fe640 1 -- 172.21.15.7:0/2463005202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f24180109d0 con 0x7f2430102790 2024-09-10T11:02:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.363+0000 7f24277fe640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f241c0a2cf0 0x7f241c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.363+0000 7f242e59c640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f241c0a2cf0 0x7f241c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.363+0000 7f24277fe640 1 -- 172.21.15.7:0/2463005202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8564+0+0 (secure 0 0 0) 0x7f24180cf150 con 0x7f2430102790 2024-09-10T11:02:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.364+0000 7f242e59c640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f241c0a2cf0 0x7f241c0a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2420006fd0 tx=0x7f2420008040 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.366+0000 7f24277fe640 1 -- 172.21.15.7:0/2463005202 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2418009de0 con 0x7f2430102790 2024-09-10T11:02:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.581+0000 7f2434a45640 1 -- 172.21.15.7:0/2463005202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f243010a150 con 0x7f2430102790 2024-09-10T11:02:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.581+0000 7f24277fe640 1 -- 172.21.15.7:0/2463005202 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11913 (secure 0 0 0) 0x7f2418097030 con 0x7f2430102790 2024-09-10T11:02:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 53 2024-09-10T11:02:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 -- 172.21.15.7:0/2463005202 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f241c0a2cf0 msgr2=0x7f241c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f241c0a2cf0 0x7f241c0a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2420006fd0 tx=0x7f2420008040 comp rx=0 tx=0).stop 2024-09-10T11:02:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 -- 172.21.15.7:0/2463005202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2430102790 msgr2=0x7f24301989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2430102790 0x7f24301989a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f241800d950 tx=0x7f241800de10 comp rx=0 tx=0).stop 2024-09-10T11:02:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 -- 172.21.15.7:0/2463005202 shutdown_connections 2024-09-10T11:02:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2430106f10 0x7f243019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f241c0a2cf0 0x7f241c0a51b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2430102790 0x7f24301989a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 --2- 172.21.15.7:0/2463005202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2430101de0 0x7f2430198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 -- 172.21.15.7:0/2463005202 >> 172.21.15.7:0/2463005202 conn(0x7f24300fd8c0 msgr2=0x7f24300fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 -- 172.21.15.7:0/2463005202 shutdown_connections 2024-09-10T11:02:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:39.584+0000 7f2434a45640 1 -- 172.21.15.7:0/2463005202 wait complete. 2024-09-10T11:02:39.684 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T11:02:39.685 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"btime":"2024-09-10T11:02:08:650543+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3661855188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3661855188},{"type":"v1","addr":"172.21.15.7:6835","nonce":3661855188}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2814690837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2814690837},{"type":"v1","addr":"172.21.15.7:6837","nonce":2814690837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1984917528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1984917528},{"type":"v1","addr":"172.21.15.7:6839","nonce":1984917528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/1887416712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":1887416712},{"type":"v1","addr":"172.21.15.7:6841","nonce":1887416712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":16425,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/364818207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":364818207},{"type":"v1","addr":"172.21.15.133:6835","nonce":364818207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/2889573301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2889573301},{"type":"v1","addr":"172.21.15.103:6833","nonce":2889573301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:44:11.549269+0000","modified":"2024-09-10T11:02:08.650513+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24499,"mds_1":24482,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24439":{"gid":24439,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/691908645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":691908645},{"type":"v1","addr":"172.21.15.133:6833","nonce":691908645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6837/2960409846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2960409846},{"type":"v1","addr":"172.21.15.103:6837","nonce":2960409846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3030361099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3030361099},{"type":"v1","addr":"172.21.15.133:6837","nonce":3030361099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.103:6839/2537001809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2537001809},{"type":"v1","addr":"172.21.15.103:6839","nonce":2537001809}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/1182528850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1182528850},{"type":"v1","addr":"172.21.15.133:6839","nonce":1182528850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26177":{"gid":26177,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6835/1460675138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1460675138},{"type":"v1","addr":"172.21.15.103:6835","nonce":1460675138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24499,"qdb_cluster":[24499,24482,24481]},"id":1}]} 2024-09-10T11:02:39.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:39 smithi007 ceph-mon[30873]: pgmap v532: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 4.4 MiB/s wr, 171 op/s 2024-09-10T11:02:39.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:39 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2463005202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T11:02:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:39 smithi133 ceph-mon[35282]: pgmap v532: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 4.4 MiB/s wr, 171 op/s 2024-09-10T11:02:39.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:39 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2463005202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T11:02:40.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:39 smithi103 ceph-mon[35160]: pgmap v532: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 4.4 MiB/s wr, 171 op/s 2024-09-10T11:02:40.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:39 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2463005202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T11:02:40.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T11:02:40.705 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:41.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.107+0000 7fc9eefef640 1 -- 172.21.15.7:0/3606225779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e8106f10 msgr2=0x7fc9e810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.107+0000 7fc9eefef640 1 --2- 172.21.15.7:0/3606225779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e8106f10 0x7fc9e810ea90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009ca0 tx=0x7fc9e002a8c0 comp rx=0 tx=0).stop 2024-09-10T11:02:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.107+0000 7fc9eefef640 1 -- 172.21.15.7:0/3606225779 shutdown_connections 2024-09-10T11:02:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.107+0000 7fc9eefef640 1 --2- 172.21.15.7:0/3606225779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e8106f10 0x7fc9e810ea90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.107+0000 7fc9eefef640 1 --2- 172.21.15.7:0/3606225779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9e8102790 0x7fc9e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.107+0000 7fc9eefef640 1 --2- 172.21.15.7:0/3606225779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9e8101de0 0x7fc9e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:41.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.107+0000 7fc9eefef640 1 -- 172.21.15.7:0/3606225779 >> 172.21.15.7:0/3606225779 conn(0x7fc9e80fd8c0 msgr2=0x7fc9e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.107+0000 7fc9eefef640 1 -- 172.21.15.7:0/3606225779 shutdown_connections 2024-09-10T11:02:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.107+0000 7fc9eefef640 1 -- 172.21.15.7:0/3606225779 wait complete. 2024-09-10T11:02:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9eefef640 1 Processor -- start 2024-09-10T11:02:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9eefef640 1 -- start start 2024-09-10T11:02:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9eefef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9eefef640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9e8102790 0x7fc9e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:41.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9eefef640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9e8106f10 0x7fc9e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9eefef640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8111f70 con 0x7fc9e8101de0 2024-09-10T11:02:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9eefef640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8111df0 con 0x7fc9e8102790 2024-09-10T11:02:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9eefef640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81120f0 con 0x7fc9e8106f10 2024-09-10T11:02:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9ecd8b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9ecd8b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48162/0 (socket says 172.21.15.7:48162) 2024-09-10T11:02:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9ecd8b640 1 -- 172.21.15.7:0/2824477573 learned_addr learned my addr 172.21.15.7:0/2824477573 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.108+0000 7fc9e7fff640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9e8102790 0x7fc9e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:41.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9ecd8b640 1 -- 172.21.15.7:0/2824477573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9e8106f10 msgr2=0x7fc9e8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9ecd8b640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9e8106f10 0x7fc9e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9ecd8b640 1 -- 172.21.15.7:0/2824477573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9e8102790 msgr2=0x7fc9e8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9ecd8b640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9e8102790 0x7fc9e8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9ecd8b640 1 -- 172.21.15.7:0/2824477573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e81a6920 con 0x7fc9e8101de0 2024-09-10T11:02:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9ecd8b640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00d950 tx=0x7fc9dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9e5ffb640 1 -- 172.21.15.7:0/2824477573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc017070 con 0x7fc9e8101de0 2024-09-10T11:02:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9eefef640 1 -- 172.21.15.7:0/2824477573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e81a6c10 con 0x7fc9e8101de0 2024-09-10T11:02:41.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9e5ffb640 1 -- 172.21.15.7:0/2824477573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc009c30 con 0x7fc9e8101de0 2024-09-10T11:02:41.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9e5ffb640 1 -- 172.21.15.7:0/2824477573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00f6b0 con 0x7fc9e8101de0 2024-09-10T11:02:41.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.109+0000 7fc9eefef640 1 -- 172.21.15.7:0/2824477573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e81a7150 con 0x7fc9e8101de0 2024-09-10T11:02:41.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.110+0000 7fc9eefef640 1 -- 172.21.15.7:0/2824477573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e81021c0 con 0x7fc9e8101de0 2024-09-10T11:02:41.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.111+0000 7fc9e5ffb640 1 -- 172.21.15.7:0/2824477573 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc9dc0109d0 con 0x7fc9e8101de0 2024-09-10T11:02:41.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.112+0000 7fc9e5ffb640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc9d80a2cf0 0x7fc9d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:41.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.112+0000 7fc9e7fff640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc9d80a2cf0 0x7fc9d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:41.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.112+0000 7fc9e5ffb640 1 -- 172.21.15.7:0/2824477573 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8564+0+0 (secure 0 0 0) 0x7fc9dc0cf090 con 0x7fc9e8101de0 2024-09-10T11:02:41.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.113+0000 7fc9e7fff640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc9d80a2cf0 0x7fc9d80a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc9e81087b0 tx=0x7fc9d400b390 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:41.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.117+0000 7fc9e5ffb640 1 -- 172.21.15.7:0/2824477573 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9dc009de0 con 0x7fc9e8101de0 2024-09-10T11:02:41.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.301+0000 7fc9eefef640 1 -- 172.21.15.7:0/2824477573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9e810a150 con 0x7fc9e8101de0 2024-09-10T11:02:41.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.303+0000 7fc9e5ffb640 1 -- 172.21.15.7:0/2824477573 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+23172 (secure 0 0 0) 0x7fc9dc09bc80 con 0x7fc9e8101de0 2024-09-10T11:02:41.304 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T11:02:41.304 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":109,"fsid":"65af4d96-6f60-11ef-bcea-c7b262605968","created":"2024-09-10T10:36:33.117443+0000","modified":"2024-09-10T11:02:13.652552+0000","last_up_change":"2024-09-10T10:43:23.913978+0000","last_in_change":"2024-09-10T10:42:59.733844+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:39:37.104114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:07.441548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:09.970273+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9239e99-3a62-4883-b2f1-8964ebb54492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6803","nonce":704452390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6805","nonce":704452390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6809","nonce":704452390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":704452390},{"type":"v1","addr":"172.21.15.7:6807","nonce":704452390}]},"public_addr":"172.21.15.7:6803/704452390","cluster_addr":"172.21.15.7:6805/704452390","heartbeat_back_addr":"172.21.15.7:6809/704452390","heartbeat_front_addr":"172.21.15.7:6807/704452390","state":["exists","up"]},{"osd":1,"uuid":"d750ff16-48c8-421d-b46b-49f0d2b7f4e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6801","nonce":2409158189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6803","nonce":2409158189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6807","nonce":2409158189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2409158189},{"type":"v1","addr":"172.21.15.103:6805","nonce":2409158189}]},"public_addr":"172.21.15.103:6801/2409158189","cluster_addr":"172.21.15.103:6803/2409158189","heartbeat_back_addr":"172.21.15.103:6807/2409158189","heartbeat_front_addr":"172.21.15.103:6805/2409158189","state":["exists","up"]},{"osd":2,"uuid":"1ecb070a-a2f1-499a-99e7-f1967ebb52ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6801","nonce":1657349858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6803","nonce":1657349858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6807","nonce":1657349858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1657349858},{"type":"v1","addr":"172.21.15.133:6805","nonce":1657349858}]},"public_addr":"172.21.15.133:6801/1657349858","cluster_addr":"172.21.15.133:6803/1657349858","heartbeat_back_addr":"172.21.15.133:6807/1657349858","heartbeat_front_addr":"172.21.15.133:6805/1657349858","state":["exists","up"]},{"osd":3,"uuid":"8e53d79e-038f-4a7f-9ac9-16b60f47bb4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6811","nonce":871781144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6813","nonce":871781144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6817","nonce":871781144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":871781144},{"type":"v1","addr":"172.21.15.7:6815","nonce":871781144}]},"public_addr":"172.21.15.7:6811/871781144","cluster_addr":"172.21.15.7:6813/871781144","heartbeat_back_addr":"172.21.15.7:6817/871781144","heartbeat_front_addr":"172.21.15.7:6815/871781144","state":["exists","up"]},{"osd":4,"uuid":"e4f04c13-594c-43dd-9a98-d2138061b1a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6809","nonce":1937243043}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6811","nonce":1937243043}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6815","nonce":1937243043}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1937243043},{"type":"v1","addr":"172.21.15.103:6813","nonce":1937243043}]},"public_addr":"172.21.15.103:6809/1937243043","cluster_addr":"172.21.15.103:6811/1937243043","heartbeat_back_addr":"172.21.15.103:6815/1937243043","heartbeat_front_addr":"172.21.15.103:6813/1937243043","state":["exists","up"]},{"osd":5,"uuid":"73143170-5816-407d-ae03-dcf896057681","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6809","nonce":577882774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6811","nonce":577882774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6815","nonce":577882774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":577882774},{"type":"v1","addr":"172.21.15.133:6813","nonce":577882774}]},"public_addr":"172.21.15.133:6809/577882774","cluster_addr":"172.21.15.133:6811/577882774","heartbeat_back_addr":"172.21.15.133:6815/577882774","heartbeat_front_addr":"172.21.15.133:6813/577882774","state":["exists","up"]},{"osd":6,"uuid":"3a5904d5-36de-47b0-97fa-5a10b24d0c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6819","nonce":2646192734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6821","nonce":2646192734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6825","nonce":2646192734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2646192734},{"type":"v1","addr":"172.21.15.7:6823","nonce":2646192734}]},"public_addr":"172.21.15.7:6819/2646192734","cluster_addr":"172.21.15.7:6821/2646192734","heartbeat_back_addr":"172.21.15.7:6825/2646192734","heartbeat_front_addr":"172.21.15.7:6823/2646192734","state":["exists","up"]},{"osd":7,"uuid":"91e79d42-decd-4405-b9f2-9480f7edfca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6817","nonce":3858867236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6819","nonce":3858867236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6823","nonce":3858867236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":3858867236},{"type":"v1","addr":"172.21.15.103:6821","nonce":3858867236}]},"public_addr":"172.21.15.103:6817/3858867236","cluster_addr":"172.21.15.103:6819/3858867236","heartbeat_back_addr":"172.21.15.103:6823/3858867236","heartbeat_front_addr":"172.21.15.103:6821/3858867236","state":["exists","up"]},{"osd":8,"uuid":"64373754-e81a-430c-967d-4b4f289f3473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6817","nonce":131428321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6819","nonce":131428321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6823","nonce":131428321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":131428321},{"type":"v1","addr":"172.21.15.133:6821","nonce":131428321}]},"public_addr":"172.21.15.133:6817/131428321","cluster_addr":"172.21.15.133:6819/131428321","heartbeat_back_addr":"172.21.15.133:6823/131428321","heartbeat_front_addr":"172.21.15.133:6821/131428321","state":["exists","up"]},{"osd":9,"uuid":"91ebe2c9-bda3-4596-ac76-5678c71f850b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6827","nonce":945633240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6829","nonce":945633240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6833","nonce":945633240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":945633240},{"type":"v1","addr":"172.21.15.7:6831","nonce":945633240}]},"public_addr":"172.21.15.7:6827/945633240","cluster_addr":"172.21.15.7:6829/945633240","heartbeat_back_addr":"172.21.15.7:6833/945633240","heartbeat_front_addr":"172.21.15.7:6831/945633240","state":["exists","up"]},{"osd":10,"uuid":"2f65568f-89d1-4a67-a222-8e263d34c710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6825","nonce":907835659}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6827","nonce":907835659}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6831","nonce":907835659}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":907835659},{"type":"v1","addr":"172.21.15.103:6829","nonce":907835659}]},"public_addr":"172.21.15.103:6825/907835659","cluster_addr":"172.21.15.103:6827/907835659","heartbeat_back_addr":"172.21.15.103:6831/907835659","heartbeat_front_addr":"172.21.15.103:6829/907835659","state":["exists","up"]},{"osd":11,"uuid":"4b49839b-32e9-4105-abee-a253f1aa3f7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6825","nonce":2538184038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6827","nonce":2538184038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6831","nonce":2538184038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2538184038},{"type":"v1","addr":"172.21.15.133:6829","nonce":2538184038}]},"public_addr":"172.21.15.133:6825/2538184038","cluster_addr":"172.21.15.133:6827/2538184038","heartbeat_back_addr":"172.21.15.133:6831/2538184038","heartbeat_front_addr":"172.21.15.133:6829/2538184038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:38:44.541279+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:10.103733+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:39:35.290268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:01.070763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:25.666941+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:40:50.925733+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:16.486537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:41:41.464155+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:06.610089+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:31.994080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:42:57.000696+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:21.398669+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1740947437":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/3214781379":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/3099326751":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/959611916":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/11850255":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/3768226338":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6801/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/262340568":"2024-09-11T10:37:20.588263+0000","172.21.15.7:0/258851660":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6800/4050686422":"2024-09-11T10:36:55.407956+0000","172.21.15.7:0/3411487770":"2024-09-11T10:36:55.407956+0000","172.21.15.7:6800/3525096194":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/3394895925":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/2266895831":"2024-09-11T10:37:20.588263+0000","172.21.15.7:6801/3525096194":"2024-09-11T10:45:53.785414+0000","172.21.15.7:0/1846259971":"2024-09-11T10:45:53.785414+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:02:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 -- 172.21.15.7:0/2824477573 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc9d80a2cf0 msgr2=0x7fc9d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc9d80a2cf0 0x7fc9d80a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc9e81087b0 tx=0x7fc9d400b390 comp rx=0 tx=0).stop 2024-09-10T11:02:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 -- 172.21.15.7:0/2824477573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e8101de0 msgr2=0x7fc9e80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00d950 tx=0x7fc9dc00de10 comp rx=0 tx=0).stop 2024-09-10T11:02:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 -- 172.21.15.7:0/2824477573 shutdown_connections 2024-09-10T11:02:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9e8106f10 0x7fc9e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9e8102790 0x7fc9e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fc9d80a2cf0 0x7fc9d80a51b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 --2- 172.21.15.7:0/2824477573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e8101de0 0x7fc9e80744f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 -- 172.21.15.7:0/2824477573 >> 172.21.15.7:0/2824477573 conn(0x7fc9e80fd8c0 msgr2=0x7fc9e80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:41.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 -- 172.21.15.7:0/2824477573 shutdown_connections 2024-09-10T11:02:41.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:41.305+0000 7fc9eefef640 1 -- 172.21.15.7:0/2824477573 wait complete. 2024-09-10T11:02:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:41 smithi133 ceph-mon[35282]: pgmap v533: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 6.3 MiB/s wr, 236 op/s 2024-09-10T11:02:41.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:41 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2824477573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:02:41.990 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:41 smithi007 ceph-mon[30873]: pgmap v533: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 6.3 MiB/s wr, 236 op/s 2024-09-10T11:02:41.990 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:41 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2824477573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:02:41.991 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T11:02:42.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:41 smithi103 ceph-mon[35160]: pgmap v533: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 6.3 MiB/s wr, 236 op/s 2024-09-10T11:02:42.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:41 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2824477573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:02:42.252 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.665+0000 7fae9c827640 1 -- 172.21.15.7:0/1660196561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae98106f10 msgr2=0x7fae9810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.665+0000 7fae9c827640 1 --2- 172.21.15.7:0/1660196561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae98106f10 0x7fae9810ea90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fae90009ca0 tx=0x7fae9002a8c0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.665+0000 7fae9c827640 1 -- 172.21.15.7:0/1660196561 shutdown_connections 2024-09-10T11:02:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.665+0000 7fae9c827640 1 --2- 172.21.15.7:0/1660196561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae98106f10 0x7fae9810ea90 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.665+0000 7fae9c827640 1 --2- 172.21.15.7:0/1660196561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae98102790 0x7fae981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.665+0000 7fae9c827640 1 --2- 172.21.15.7:0/1660196561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae98101de0 0x7fae981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.665+0000 7fae9c827640 1 -- 172.21.15.7:0/1660196561 >> 172.21.15.7:0/1660196561 conn(0x7fae980fd8c0 msgr2=0x7fae980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:42.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.666+0000 7fae9c827640 1 -- 172.21.15.7:0/1660196561 shutdown_connections 2024-09-10T11:02:42.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.666+0000 7fae9c827640 1 -- 172.21.15.7:0/1660196561 wait complete. 2024-09-10T11:02:42.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.666+0000 7fae9c827640 1 Processor -- start 2024-09-10T11:02:42.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.666+0000 7fae9c827640 1 -- start start 2024-09-10T11:02:42.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae9c827640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae98101de0 0x7fae981983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:42.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae9c827640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae98102790 0x7fae981988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:42.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae9c827640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae98106f10 0x7fae9819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:42.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae9c827640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae98111f70 con 0x7fae98102790 2024-09-10T11:02:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae9c827640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae98111df0 con 0x7fae98101de0 2024-09-10T11:02:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae9c827640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fae981120f0 con 0x7fae98106f10 2024-09-10T11:02:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae95d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae98102790 0x7fae981988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae95d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae98102790 0x7fae981988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48188/0 (socket says 172.21.15.7:48188) 2024-09-10T11:02:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae95d9b640 1 -- 172.21.15.7:0/1868913124 learned_addr learned my addr 172.21.15.7:0/1868913124 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae96d9d640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae98106f10 0x7fae9819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae9659c640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae98101de0 0x7fae981983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:42.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae95d9b640 1 -- 172.21.15.7:0/1868913124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae98106f10 msgr2=0x7fae9819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:42.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae95d9b640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae98106f10 0x7fae9819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae95d9b640 1 -- 172.21.15.7:0/1868913124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae98101de0 msgr2=0x7fae981983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:42.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae95d9b640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae98101de0 0x7fae981983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae95d9b640 1 -- 172.21.15.7:0/1868913124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9819d360 con 0x7fae98102790 2024-09-10T11:02:42.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae9659c640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae98101de0 0x7fae981983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:02:42.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.667+0000 7fae95d9b640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae98102790 0x7fae981988f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00d770 tx=0x7fae8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:42.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.668+0000 7fae877fe640 1 -- 172.21.15.7:0/1868913124 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c017070 con 0x7fae98102790 2024-09-10T11:02:42.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.668+0000 7fae877fe640 1 -- 172.21.15.7:0/1868913124 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae8c005500 con 0x7fae98102790 2024-09-10T11:02:42.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.668+0000 7fae9c827640 1 -- 172.21.15.7:0/1868913124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9819d650 con 0x7fae98102790 2024-09-10T11:02:42.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.668+0000 7fae877fe640 1 -- 172.21.15.7:0/1868913124 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c004e00 con 0x7fae98102790 2024-09-10T11:02:42.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.668+0000 7fae9c827640 1 -- 172.21.15.7:0/1868913124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981a4f30 con 0x7fae98102790 2024-09-10T11:02:42.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.669+0000 7fae9c827640 1 -- 172.21.15.7:0/1868913124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae981021c0 con 0x7fae98102790 2024-09-10T11:02:42.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.670+0000 7fae877fe640 1 -- 172.21.15.7:0/1868913124 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fae8c004fa0 con 0x7fae98102790 2024-09-10T11:02:42.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.670+0000 7fae877fe640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fae800a2cf0 0x7fae800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:42.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.671+0000 7fae9659c640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fae800a2cf0 0x7fae800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:42.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.671+0000 7fae877fe640 1 -- 172.21.15.7:0/1868913124 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8564+0+0 (secure 0 0 0) 0x7fae8c0ceea0 con 0x7fae98102790 2024-09-10T11:02:42.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.671+0000 7fae9659c640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fae800a2cf0 0x7fae800a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae88005fd0 tx=0x7fae8800b5f0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:42.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.675+0000 7fae877fe640 1 -- 172.21.15.7:0/1868913124 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae8c09ba90 con 0x7fae98102790 2024-09-10T11:02:42.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.890+0000 7fae9c827640 1 -- 172.21.15.7:0/1868913124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fae9810a0f0 con 0x7fae98102790 2024-09-10T11:02:42.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.892+0000 7fae877fe640 1 -- 172.21.15.7:0/1868913124 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 161+0+0 (secure 0 0 0) 0x7fae8c09b390 con 0x7fae98102790 2024-09-10T11:02:42.893 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:02:42.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 -- 172.21.15.7:0/1868913124 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fae800a2cf0 msgr2=0x7fae800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:42.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fae800a2cf0 0x7fae800a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae88005fd0 tx=0x7fae8800b5f0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 -- 172.21.15.7:0/1868913124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae98102790 msgr2=0x7fae981988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:42.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae98102790 0x7fae981988f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00d770 tx=0x7fae8c00dc30 comp rx=0 tx=0).stop 2024-09-10T11:02:42.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 -- 172.21.15.7:0/1868913124 shutdown_connections 2024-09-10T11:02:42.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae98106f10 0x7fae9819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fae800a2cf0 0x7fae800a51b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae98102790 0x7fae981988f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 --2- 172.21.15.7:0/1868913124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae98101de0 0x7fae981983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:42.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 -- 172.21.15.7:0/1868913124 >> 172.21.15.7:0/1868913124 conn(0x7fae980fd8c0 msgr2=0x7fae980fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:42.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 -- 172.21.15.7:0/1868913124 shutdown_connections 2024-09-10T11:02:42.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:42.895+0000 7fae9c827640 1 -- 172.21.15.7:0/1868913124 wait complete. 2024-09-10T11:02:42.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:42 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a[30869]: 2024-09-10T11:02:42.890+0000 7f79bdec7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:02:43.225 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c[65774]: ignoring --setuser ceph since I am not root 2024-09-10T11:02:43.226 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c[65774]: ignoring --setgroup ceph since I am not root 2024-09-10T11:02:43.226 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-mds[65778]: -- 172.21.15.133:0/1215835204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55935cda1860 con 0x55935cd7e800 2024-09-10T11:02:43.226 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c[65774]: starting mds.c at 2024-09-10T11:02:43.227 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-i[66074]: ignoring --setuser ceph since I am not root 2024-09-10T11:02:43.227 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-i[66074]: ignoring --setgroup ceph since I am not root 2024-09-10T11:02:43.227 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-mds[66078]: -- 172.21.15.133:0/258789588 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fb777c3860 con 0x55fb777a0800 2024-09-10T11:02:43.227 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-i[66074]: starting mds.i at 2024-09-10T11:02:43.228 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l[65164]: ignoring --setuser ceph since I am not root 2024-09-10T11:02:43.228 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l[65164]: ignoring --setgroup ceph since I am not root 2024-09-10T11:02:43.228 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-mds[65168]: -- 172.21.15.133:0/1601922334 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557d4fb63860 con 0x557d4fb40800 2024-09-10T11:02:43.228 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:02:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l[65164]: starting mds.l at 2024-09-10T11:02:43.422 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b[65911]: ignoring --setuser ceph since I am not root 2024-09-10T11:02:43.423 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b[65911]: ignoring --setgroup ceph since I am not root 2024-09-10T11:02:43.423 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-mds[65915]: -- 172.21.15.103:0/4089374113 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b834397860 con 0x55b834375000 2024-09-10T11:02:43.423 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b[65911]: starting mds.b at 2024-09-10T11:02:43.423 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h[65605]: ignoring --setuser ceph since I am not root 2024-09-10T11:02:43.423 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h[65605]: ignoring --setgroup ceph since I am not root 2024-09-10T11:02:43.424 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-mds[65609]: -- 172.21.15.103:0/1627665971 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565240d7860 con 0x5565240b5400 2024-09-10T11:02:43.424 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h[65605]: starting mds.h at 2024-09-10T11:02:43.424 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e[65303]: ignoring --setuser ceph since I am not root 2024-09-10T11:02:43.424 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e[65303]: ignoring --setgroup ceph since I am not root 2024-09-10T11:02:43.424 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-mds[65307]: -- 172.21.15.103:0/3865897014 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f096d9c20 con 0x558f096b7400 2024-09-10T11:02:43.424 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:02:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e[65303]: starting mds.e at 2024-09-10T11:02:43.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:02:43.818 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:43 smithi133 ceph-mon[35282]: pgmap v534: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 4.5 MiB/s wr, 169 op/s 2024-09-10T11:02:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:43 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1868913124' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T11:02:43.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:43 smithi133 ceph-mon[35282]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T11:02:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:43 smithi133 ceph-mon[35282]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:02:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:43 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/1868913124' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T11:02:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:43 smithi133 ceph-mon[35282]: osdmap e110: 12 total, 12 up, 12 in 2024-09-10T11:02:43.976 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:43 smithi133 ceph-mon[35282]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T11:02:44.051 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:43 smithi007 ceph-mon[30873]: pgmap v534: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 4.5 MiB/s wr, 169 op/s 2024-09-10T11:02:44.051 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:43 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1868913124' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T11:02:44.051 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:43 smithi007 ceph-mon[30873]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T11:02:44.052 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:43 smithi007 ceph-mon[30873]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:02:44.052 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:43 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/1868913124' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T11:02:44.052 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:43 smithi007 ceph-mon[30873]: osdmap e110: 12 total, 12 up, 12 in 2024-09-10T11:02:44.052 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:43 smithi007 ceph-mon[30873]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T11:02:44.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:43 smithi103 ceph-mon[35160]: pgmap v534: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 4.5 MiB/s wr, 169 op/s 2024-09-10T11:02:44.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:43 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1868913124' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T11:02:44.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:43 smithi103 ceph-mon[35160]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T11:02:44.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:43 smithi103 ceph-mon[35160]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:02:44.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:43 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/1868913124' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T11:02:44.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:43 smithi103 ceph-mon[35160]: osdmap e110: 12 total, 12 up, 12 in 2024-09-10T11:02:44.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:43 smithi103 ceph-mon[35160]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-10T11:02:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.255+0000 7fa7b5117640 1 -- 172.21.15.7:0/476075098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0102ef0 msgr2=0x7fa7b010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.255+0000 7fa7b5117640 1 --2- 172.21.15.7:0/476075098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0102ef0 0x7fa7b010da40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009ca0 tx=0x7fa7a802a950 comp rx=0 tx=0).stop 2024-09-10T11:02:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.255+0000 7fa7b5117640 1 -- 172.21.15.7:0/476075098 shutdown_connections 2024-09-10T11:02:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.255+0000 7fa7b5117640 1 --2- 172.21.15.7:0/476075098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0102ef0 0x7fa7b010da40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.255+0000 7fa7b5117640 1 --2- 172.21.15.7:0/476075098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b0102570 0x7fa7b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.255+0000 7fa7b5117640 1 --2- 172.21.15.7:0/476075098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b0100e20 0x7fa7b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.255+0000 7fa7b5117640 1 -- 172.21.15.7:0/476075098 >> 172.21.15.7:0/476075098 conn(0x7fa7b00fc7d0 msgr2=0x7fa7b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.255+0000 7fa7b5117640 1 -- 172.21.15.7:0/476075098 shutdown_connections 2024-09-10T11:02:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.255+0000 7fa7b5117640 1 -- 172.21.15.7:0/476075098 wait complete. 2024-09-10T11:02:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.255+0000 7fa7b5117640 1 Processor -- start 2024-09-10T11:02:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7b5117640 1 -- start start 2024-09-10T11:02:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7b5117640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0100e20 0x7fa7b01970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7b5117640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b0102570 0x7fa7b0197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7b5117640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b0102ef0 0x7fa7b0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7b5117640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0110fd0 con 0x7fa7b0100e20 2024-09-10T11:02:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7b5117640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0110e50 con 0x7fa7b0102ef0 2024-09-10T11:02:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7b5117640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0111150 con 0x7fa7b0102570 2024-09-10T11:02:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7aed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0100e20 0x7fa7b01970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7aed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0100e20 0x7fa7b01970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48212/0 (socket says 172.21.15.7:48212) 2024-09-10T11:02:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7aed9d640 1 -- 172.21.15.7:0/651646200 learned_addr learned my addr 172.21.15.7:0/651646200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7af59e640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b0102ef0 0x7fa7b0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7ae59c640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b0102570 0x7fa7b0197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7aed9d640 1 -- 172.21.15.7:0/651646200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b0102570 msgr2=0x7fa7b0197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7aed9d640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b0102570 0x7fa7b0197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7aed9d640 1 -- 172.21.15.7:0/651646200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b0102ef0 msgr2=0x7fa7b0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7aed9d640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b0102ef0 0x7fa7b0197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.256+0000 7fa7aed9d640 1 -- 172.21.15.7:0/651646200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b01a6960 con 0x7fa7b0100e20 2024-09-10T11:02:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.257+0000 7fa7aed9d640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0100e20 0x7fa7b01970f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000c890 tx=0x7fa7a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.257+0000 7fa7a7fff640 1 -- 172.21.15.7:0/651646200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0004220 con 0x7fa7b0100e20 2024-09-10T11:02:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.257+0000 7fa7a7fff640 1 -- 172.21.15.7:0/651646200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a00043c0 con 0x7fa7b0100e20 2024-09-10T11:02:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.257+0000 7fa7b5117640 1 -- 172.21.15.7:0/651646200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b01a6c50 con 0x7fa7b0100e20 2024-09-10T11:02:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.257+0000 7fa7a7fff640 1 -- 172.21.15.7:0/651646200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a00049e0 con 0x7fa7b0100e20 2024-09-10T11:02:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.257+0000 7fa7b5117640 1 -- 172.21.15.7:0/651646200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b01a7190 con 0x7fa7b0100e20 2024-09-10T11:02:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.258+0000 7fa7b5117640 1 -- 172.21.15.7:0/651646200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b0101200 con 0x7fa7b0100e20 2024-09-10T11:02:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.259+0000 7fa7a7fff640 1 -- 172.21.15.7:0/651646200 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa7a00079b0 con 0x7fa7b0100e20 2024-09-10T11:02:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.260+0000 7fa7a7fff640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fa79c0a2cf0 0x7fa79c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.260+0000 7fa7a7fff640 1 -- 172.21.15.7:0/651646200 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8778+0+0 (secure 0 0 0) 0x7fa7a00cf5b0 con 0x7fa7b0100e20 2024-09-10T11:02:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.260+0000 7fa7ae59c640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fa79c0a2cf0 0x7fa79c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.261+0000 7fa7ae59c640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fa79c0a2cf0 0x7fa79c0a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa798005fd0 tx=0x7fa798005850 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.263+0000 7fa7a7fff640 1 -- 172.21.15.7:0/651646200 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7a0014030 con 0x7fa7b0100e20 2024-09-10T11:02:44.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.478+0000 7fa7b5117640 1 -- 172.21.15.7:0/651646200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa7b01a7450 con 0x7fa7b0100e20 2024-09-10T11:02:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.899+0000 7fa7a7fff640 1 -- 172.21.15.7:0/651646200 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) ==== 104+0+0 (secure 0 0 0) 0x7fa7a009c0d0 con 0x7fa7b0100e20 2024-09-10T11:02:44.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 -- 172.21.15.7:0/651646200 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fa79c0a2cf0 msgr2=0x7fa79c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:44.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fa79c0a2cf0 0x7fa79c0a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa798005fd0 tx=0x7fa798005850 comp rx=0 tx=0).stop 2024-09-10T11:02:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 -- 172.21.15.7:0/651646200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0100e20 msgr2=0x7fa7b01970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0100e20 0x7fa7b01970f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000c890 tx=0x7fa7a000cd50 comp rx=0 tx=0).stop 2024-09-10T11:02:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 -- 172.21.15.7:0/651646200 shutdown_connections 2024-09-10T11:02:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7b0102ef0 0x7fa7b0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7b0102570 0x7fa7b0197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7fa79c0a2cf0 0x7fa79c0a51b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 --2- 172.21.15.7:0/651646200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0100e20 0x7fa7b01970f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 -- 172.21.15.7:0/651646200 >> 172.21.15.7:0/651646200 conn(0x7fa7b00fc7d0 msgr2=0x7fa7b0106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 -- 172.21.15.7:0/651646200 shutdown_connections 2024-09-10T11:02:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:44.902+0000 7fa7b5117640 1 -- 172.21.15.7:0/651646200 wait complete. 2024-09-10T11:02:45.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.103:6834/2067118910,v1:172.21.15.103:6835/2067118910] up:boot 2024-09-10T11:02:45.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.103:6837/1953597393,v1:172.21.15.103:6839/1953597393] up:boot 2024-09-10T11:02:45.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.103:6836/3750798650,v1:172.21.15.103:6838/3750798650] up:boot 2024-09-10T11:02:45.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6833/3725916604,v1:172.21.15.133:6837/3725916604] up:boot 2024-09-10T11:02:45.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6832/1595205429,v1:172.21.15.133:6838/1595205429] up:boot 2024-09-10T11:02:45.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: mds.? [v2:172.21.15.133:6836/2589773535,v1:172.21.15.133:6839/2589773535] up:boot 2024-09-10T11:02:45.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T11:02:45.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T11:02:45.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T11:02:45.173 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T11:02:45.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T11:02:45.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T11:02:45.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T11:02:45.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: pgmap v536: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 4.8 MiB/s wr, 186 op/s 2024-09-10T11:02:45.174 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:44 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/651646200' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T11:02:45.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.103:6834/2067118910,v1:172.21.15.103:6835/2067118910] up:boot 2024-09-10T11:02:45.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.103:6837/1953597393,v1:172.21.15.103:6839/1953597393] up:boot 2024-09-10T11:02:45.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.103:6836/3750798650,v1:172.21.15.103:6838/3750798650] up:boot 2024-09-10T11:02:45.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6833/3725916604,v1:172.21.15.133:6837/3725916604] up:boot 2024-09-10T11:02:45.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6832/1595205429,v1:172.21.15.133:6838/1595205429] up:boot 2024-09-10T11:02:45.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: mds.? [v2:172.21.15.133:6836/2589773535,v1:172.21.15.133:6839/2589773535] up:boot 2024-09-10T11:02:45.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T11:02:45.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T11:02:45.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T11:02:45.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T11:02:45.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T11:02:45.185 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T11:02:45.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T11:02:45.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: pgmap v536: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 4.8 MiB/s wr, 186 op/s 2024-09-10T11:02:45.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:44 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/651646200' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T11:02:45.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.103:6834/2067118910,v1:172.21.15.103:6835/2067118910] up:boot 2024-09-10T11:02:45.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.103:6837/1953597393,v1:172.21.15.103:6839/1953597393] up:boot 2024-09-10T11:02:45.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.103:6836/3750798650,v1:172.21.15.103:6838/3750798650] up:boot 2024-09-10T11:02:45.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6833/3725916604,v1:172.21.15.133:6837/3725916604] up:boot 2024-09-10T11:02:45.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6832/1595205429,v1:172.21.15.133:6838/1595205429] up:boot 2024-09-10T11:02:45.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: mds.? [v2:172.21.15.133:6836/2589773535,v1:172.21.15.133:6839/2589773535] up:boot 2024-09-10T11:02:45.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-10T11:02:45.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T11:02:45.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T11:02:45.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T11:02:45.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T11:02:45.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T11:02:45.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: from='mgr.15132 172.21.15.7:0/1914648659' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T11:02:45.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: pgmap v536: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 4.8 MiB/s wr, 186 op/s 2024-09-10T11:02:45.227 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:44 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/651646200' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T11:02:45.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:02:45.902 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:45 smithi007 ceph-mon[30873]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T11:02:45.902 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:45 smithi007 ceph-mon[30873]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T11:02:45.953 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:46.160 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:45 smithi007 ceph-mon[30873]: Cluster is now healthy 2024-09-10T11:02:46.160 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:45 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/651646200' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T11:02:46.160 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:45 smithi007 ceph-mon[30873]: fsmap 12 up:standby 2024-09-10T11:02:46.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:45 smithi103 ceph-mon[35160]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T11:02:46.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:45 smithi103 ceph-mon[35160]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T11:02:46.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:45 smithi103 ceph-mon[35160]: Cluster is now healthy 2024-09-10T11:02:46.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:45 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/651646200' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T11:02:46.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:45 smithi103 ceph-mon[35160]: fsmap 12 up:standby 2024-09-10T11:02:46.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:45 smithi133 ceph-mon[35282]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T11:02:46.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:45 smithi133 ceph-mon[35282]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T11:02:46.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:45 smithi133 ceph-mon[35282]: Cluster is now healthy 2024-09-10T11:02:46.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:45 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/651646200' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T11:02:46.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:45 smithi133 ceph-mon[35282]: fsmap 12 up:standby 2024-09-10T11:02:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.415+0000 7f9909116640 1 -- 172.21.15.7:0/2428264510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904106a60 msgr2=0x7f9904106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.415+0000 7f9909116640 1 --2- 172.21.15.7:0/2428264510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904106a60 0x7f9904106e40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009930 tx=0x7f98f402a420 comp rx=0 tx=0).stop 2024-09-10T11:02:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.415+0000 7f9909116640 1 -- 172.21.15.7:0/2428264510 shutdown_connections 2024-09-10T11:02:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.415+0000 7f9909116640 1 --2- 172.21.15.7:0/2428264510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9904101400 0x7f990410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.415+0000 7f9909116640 1 --2- 172.21.15.7:0/2428264510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9904100a60 0x7f9904100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.415+0000 7f9909116640 1 --2- 172.21.15.7:0/2428264510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904106a60 0x7f9904106e40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.415+0000 7f9909116640 1 -- 172.21.15.7:0/2428264510 >> 172.21.15.7:0/2428264510 conn(0x7f99040fc780 msgr2=0x7f99040feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.415+0000 7f9909116640 1 -- 172.21.15.7:0/2428264510 shutdown_connections 2024-09-10T11:02:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.415+0000 7f9909116640 1 -- 172.21.15.7:0/2428264510 wait complete. 2024-09-10T11:02:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.416+0000 7f9909116640 1 Processor -- start 2024-09-10T11:02:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.416+0000 7f9909116640 1 -- start start 2024-09-10T11:02:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.416+0000 7f9909116640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9904100a60 0x7f99041984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.416+0000 7f9909116640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9904101400 0x7f99041989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.416+0000 7f9909116640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904106a60 0x7f990419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.416+0000 7f9909116640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9904110e20 con 0x7f9904106a60 2024-09-10T11:02:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.416+0000 7f9909116640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9904110ca0 con 0x7f9904101400 2024-09-10T11:02:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.416+0000 7f9909116640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9904110fa0 con 0x7f9904100a60 2024-09-10T11:02:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f990359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904106a60 0x7f990419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f990359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904106a60 0x7f990419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48228/0 (socket says 172.21.15.7:48228) 2024-09-10T11:02:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f990359e640 1 -- 172.21.15.7:0/829037402 learned_addr learned my addr 172.21.15.7:0/829037402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f990259c640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9904101400 0x7f99041989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f9902d9d640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9904100a60 0x7f99041984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f990359e640 1 -- 172.21.15.7:0/829037402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9904100a60 msgr2=0x7f99041984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f990359e640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9904100a60 0x7f99041984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f990359e640 1 -- 172.21.15.7:0/829037402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9904101400 msgr2=0x7f99041989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f990359e640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9904101400 0x7f99041989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f990359e640 1 -- 172.21.15.7:0/829037402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990419d450 con 0x7f9904106a60 2024-09-10T11:02:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f9902d9d640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9904100a60 0x7f99041984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:02:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f990359e640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904106a60 0x7f990419cd70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00cad0 tx=0x7f98fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f98ebfff640 1 -- 172.21.15.7:0/829037402 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc00ae20 con 0x7f9904106a60 2024-09-10T11:02:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f9909116640 1 -- 172.21.15.7:0/829037402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990419d740 con 0x7f9904106a60 2024-09-10T11:02:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f98ebfff640 1 -- 172.21.15.7:0/829037402 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98fc005020 con 0x7f9904106a60 2024-09-10T11:02:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.417+0000 7f98ebfff640 1 -- 172.21.15.7:0/829037402 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc00f6f0 con 0x7f9904106a60 2024-09-10T11:02:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.418+0000 7f9909116640 1 -- 172.21.15.7:0/829037402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99041a5020 con 0x7f9904106a60 2024-09-10T11:02:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.418+0000 7f9909116640 1 -- 172.21.15.7:0/829037402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9904100ec0 con 0x7f9904106a60 2024-09-10T11:02:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.421+0000 7f98ebfff640 1 -- 172.21.15.7:0/829037402 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f98fc0051d0 con 0x7f9904106a60 2024-09-10T11:02:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.421+0000 7f98ebfff640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98ec0a2cf0 0x7f98ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.422+0000 7f98ebfff640 1 -- 172.21.15.7:0/829037402 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8778+0+0 (secure 0 0 0) 0x7f98fc0cf530 con 0x7f9904106a60 2024-09-10T11:02:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.422+0000 7f9902d9d640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98ec0a2cf0 0x7f98ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.422+0000 7f9902d9d640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98ec0a2cf0 0x7f98ec0a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000c00 tx=0x7f98f4005a20 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.424+0000 7f98ebfff640 1 -- 172.21.15.7:0/829037402 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98fc014030 con 0x7f9904106a60 2024-09-10T11:02:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.606+0000 7f9909116640 1 -- 172.21.15.7:0/829037402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f99041a52c0 con 0x7f9904106a60 2024-09-10T11:02:46.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.906+0000 7f98ebfff640 1 -- 172.21.15.7:0/829037402 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) ==== 181+0+0 (secure 0 0 0) 0x7f98fc09c050 con 0x7f9904106a60 2024-09-10T11:02:46.907 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:02:46.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.909+0000 7f9909116640 1 -- 172.21.15.7:0/829037402 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98ec0a2cf0 msgr2=0x7f98ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.910+0000 7f9909116640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98ec0a2cf0 0x7f98ec0a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000c00 tx=0x7f98f4005a20 comp rx=0 tx=0).stop 2024-09-10T11:02:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.910+0000 7f9909116640 1 -- 172.21.15.7:0/829037402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904106a60 msgr2=0x7f990419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.910+0000 7f9909116640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904106a60 0x7f990419cd70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00cad0 tx=0x7f98fc00cf90 comp rx=0 tx=0).stop 2024-09-10T11:02:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.910+0000 7f9909116640 1 -- 172.21.15.7:0/829037402 shutdown_connections 2024-09-10T11:02:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.910+0000 7f9909116640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f98ec0a2cf0 0x7f98ec0a51b0 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000c00 tx=0x7f98f4005a20 comp rx=0 tx=0).stop 2024-09-10T11:02:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.910+0000 7f9909116640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9904106a60 0x7f990419cd70 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.910+0000 7f9909116640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9904101400 0x7f99041989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:46.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.910+0000 7f9909116640 1 --2- 172.21.15.7:0/829037402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9904100a60 0x7f99041984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:46.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.910+0000 7f9909116640 1 -- 172.21.15.7:0/829037402 >> 172.21.15.7:0/829037402 conn(0x7f99040fc780 msgr2=0x7f9904104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:46.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.911+0000 7f9909116640 1 -- 172.21.15.7:0/829037402 shutdown_connections 2024-09-10T11:02:46.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:46.912+0000 7f9909116640 1 -- 172.21.15.7:0/829037402 wait complete. 2024-09-10T11:02:47.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:46 smithi103 ceph-mon[35160]: pgmap v537: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.2 MiB/s wr, 97 op/s 2024-09-10T11:02:47.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:46 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/829037402' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T11:02:47.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:46 smithi007 ceph-mon[30873]: pgmap v537: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.2 MiB/s wr, 97 op/s 2024-09-10T11:02:47.183 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:46 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/829037402' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T11:02:47.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:46 smithi133 ceph-mon[35282]: pgmap v537: 97 pgs: 97 active+clean; 1.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.2 MiB/s wr, 97 op/s 2024-09-10T11:02:47.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:46 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/829037402' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T11:02:47.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:02:47.816 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T11:02:48.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:47 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/829037402' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T11:02:48.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:47 smithi103 ceph-mon[35160]: osdmap e111: 12 total, 12 up, 12 in 2024-09-10T11:02:48.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:47 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/829037402' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T11:02:48.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:47 smithi007 ceph-mon[30873]: osdmap e111: 12 total, 12 up, 12 in 2024-09-10T11:02:48.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:47 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/829037402' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T11:02:48.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:47 smithi133 ceph-mon[35282]: osdmap e111: 12 total, 12 up, 12 in 2024-09-10T11:02:48.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.312+0000 7f4c95dad640 1 -- 172.21.15.7:0/3884679734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c90102da0 msgr2=0x7f4c90103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:48.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.312+0000 7f4c95dad640 1 --2- 172.21.15.7:0/3884679734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c90102da0 0x7f4c90103180 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009900 tx=0x7f4c8002a480 comp rx=0 tx=0).stop 2024-09-10T11:02:48.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.312+0000 7f4c95dad640 1 -- 172.21.15.7:0/3884679734 shutdown_connections 2024-09-10T11:02:48.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.312+0000 7f4c95dad640 1 --2- 172.21.15.7:0/3884679734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9010c180 0x7f4c9010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:48.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.312+0000 7f4c95dad640 1 --2- 172.21.15.7:0/3884679734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c901036c0 0x7f4c9010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:48.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.312+0000 7f4c95dad640 1 --2- 172.21.15.7:0/3884679734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c90102da0 0x7f4c90103180 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:48.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.312+0000 7f4c95dad640 1 -- 172.21.15.7:0/3884679734 >> 172.21.15.7:0/3884679734 conn(0x7f4c900fc7a0 msgr2=0x7f4c900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:48.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.312+0000 7f4c95dad640 1 -- 172.21.15.7:0/3884679734 shutdown_connections 2024-09-10T11:02:48.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.312+0000 7f4c95dad640 1 -- 172.21.15.7:0/3884679734 wait complete. 2024-09-10T11:02:48.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c95dad640 1 Processor -- start 2024-09-10T11:02:48.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c95dad640 1 -- start start 2024-09-10T11:02:48.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c95dad640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c90102da0 0x7f4c90198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:48.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c95dad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901036c0 0x7f4c90198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:48.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c95dad640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9010c180 0x7f4c9019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:48.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c95dad640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90111f60 con 0x7f4c901036c0 2024-09-10T11:02:48.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c95dad640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90111de0 con 0x7f4c90102da0 2024-09-10T11:02:48.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c95dad640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901120e0 con 0x7f4c9010c180 2024-09-10T11:02:48.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901036c0 0x7f4c90198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:48.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c8effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901036c0 0x7f4c90198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48246/0 (socket says 172.21.15.7:48246) 2024-09-10T11:02:48.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.313+0000 7f4c8effd640 1 -- 172.21.15.7:0/2027011577 learned_addr learned my addr 172.21.15.7:0/2027011577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T11:02:48.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8f7fe640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c90102da0 0x7f4c90198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:48.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8ffff640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9010c180 0x7f4c9019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:48.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8effd640 1 -- 172.21.15.7:0/2027011577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9010c180 msgr2=0x7f4c9019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:48.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8effd640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9010c180 0x7f4c9019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:48.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8effd640 1 -- 172.21.15.7:0/2027011577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c90102da0 msgr2=0x7f4c90198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:48.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8effd640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c90102da0 0x7f4c90198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:48.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8effd640 1 -- 172.21.15.7:0/2027011577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9019d3f0 con 0x7f4c901036c0 2024-09-10T11:02:48.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8f7fe640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c90102da0 0x7f4c90198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:02:48.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8effd640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901036c0 0x7f4c90198980 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400d950 tx=0x7f4c8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:48.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8cff9640 1 -- 172.21.15.7:0/2027011577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84017070 con 0x7f4c901036c0 2024-09-10T11:02:48.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8cff9640 1 -- 172.21.15.7:0/2027011577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c84009c30 con 0x7f4c901036c0 2024-09-10T11:02:48.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c95dad640 1 -- 172.21.15.7:0/2027011577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9019d6e0 con 0x7f4c901036c0 2024-09-10T11:02:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c8cff9640 1 -- 172.21.15.7:0/2027011577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400f6b0 con 0x7f4c901036c0 2024-09-10T11:02:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.314+0000 7f4c95dad640 1 -- 172.21.15.7:0/2027011577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c901a4fc0 con 0x7f4c901036c0 2024-09-10T11:02:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.315+0000 7f4c95dad640 1 -- 172.21.15.7:0/2027011577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c90103180 con 0x7f4c901036c0 2024-09-10T11:02:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.316+0000 7f4c8cff9640 1 -- 172.21.15.7:0/2027011577 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4c840109d0 con 0x7f4c901036c0 2024-09-10T11:02:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.317+0000 7f4c8cff9640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4c780a2d70 0x7f4c780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:02:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.317+0000 7f4c8f7fe640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4c780a2d70 0x7f4c780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:02:48.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.317+0000 7f4c8cff9640 1 -- 172.21.15.7:0/2027011577 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8245+0+0 (secure 0 0 0) 0x7f4c840cee80 con 0x7f4c901036c0 2024-09-10T11:02:48.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.318+0000 7f4c8f7fe640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4c780a2d70 0x7f4c780a5230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009230 tx=0x7f4c800058d0 comp rx=0 tx=0).ready entity=mgr.15132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:02:48.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.321+0000 7f4c8cff9640 1 -- 172.21.15.7:0/2027011577 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c84009de0 con 0x7f4c901036c0 2024-09-10T11:02:48.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.496+0000 7f4c95dad640 1 -- 172.21.15.7:0/2027011577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4c9002d7c0 con 0x7f4c901036c0 2024-09-10T11:02:48.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.919+0000 7f4c8cff9640 1 -- 172.21.15.7:0/2027011577 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) ==== 169+0+0 (secure 0 0 0) 0x7f4c8409b560 con 0x7f4c901036c0 2024-09-10T11:02:48.921 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-10T11:02:48.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.922+0000 7f4c95dad640 1 -- 172.21.15.7:0/2027011577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4c780a2d70 msgr2=0x7f4c780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:48.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.922+0000 7f4c95dad640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4c780a2d70 0x7f4c780a5230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009230 tx=0x7f4c800058d0 comp rx=0 tx=0).stop 2024-09-10T11:02:48.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.922+0000 7f4c95dad640 1 -- 172.21.15.7:0/2027011577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901036c0 msgr2=0x7f4c90198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:02:48.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.922+0000 7f4c95dad640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901036c0 0x7f4c90198980 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400d950 tx=0x7f4c8400de10 comp rx=0 tx=0).stop 2024-09-10T11:02:48.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.922+0000 7f4c95dad640 1 -- 172.21.15.7:0/2027011577 shutdown_connections 2024-09-10T11:02:48.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.922+0000 7f4c95dad640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9010c180 0x7f4c9019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:48.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.922+0000 7f4c95dad640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.7:6800/354580929,v1:172.21.15.7:6801/354580929] conn(0x7f4c780a2d70 0x7f4c780a5230 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009230 tx=0x7f4c800058d0 comp rx=0 tx=0).stop 2024-09-10T11:02:48.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.922+0000 7f4c95dad640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c901036c0 0x7f4c90198980 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:48.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.922+0000 7f4c95dad640 1 --2- 172.21.15.7:0/2027011577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c90102da0 0x7f4c90198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:02:48.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.922+0000 7f4c95dad640 1 -- 172.21.15.7:0/2027011577 >> 172.21.15.7:0/2027011577 conn(0x7f4c900fc7a0 msgr2=0x7f4c900fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:02:48.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.923+0000 7f4c95dad640 1 -- 172.21.15.7:0/2027011577 shutdown_connections 2024-09-10T11:02:48.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T11:02:48.923+0000 7f4c95dad640 1 -- 172.21.15.7:0/2027011577 wait complete. 2024-09-10T11:02:49.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:48 smithi103 ceph-mon[35160]: pgmap v539: 65 pgs: 65 active+clean; 312 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T11:02:49.172 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:48 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2027011577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T11:02:49.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:48 smithi007 ceph-mon[30873]: pgmap v539: 65 pgs: 65 active+clean; 312 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T11:02:49.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:48 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2027011577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T11:02:49.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:48 smithi133 ceph-mon[35282]: pgmap v539: 65 pgs: 65 active+clean; 312 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T11:02:49.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:48 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2027011577' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T11:02:49.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T11:02:49.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T11:02:49.651 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T11:02:49.693 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T11:02:49.693 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T11:02:49.712 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T11:02:49.727 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T11:02:49.748 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T11:02:49.748 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T11:02:49.749 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.a 2024-09-10T11:02:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:49 smithi133 ceph-mon[35282]: from='client.? 172.21.15.7:0/2027011577' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T11:02:49.975 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:49 smithi133 ceph-mon[35282]: osdmap e112: 12 total, 12 up, 12 in 2024-09-10T11:02:50.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:49 smithi007 systemd[1]: Stopping Ceph mon.a for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:02:50.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:49 smithi007 ceph-mon[30873]: from='client.? 172.21.15.7:0/2027011577' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T11:02:50.069 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:49 smithi007 ceph-mon[30873]: osdmap e112: 12 total, 12 up, 12 in 2024-09-10T11:02:50.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:49 smithi103 ceph-mon[35160]: from='client.? 172.21.15.7:0/2027011577' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T11:02:50.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:49 smithi103 ceph-mon[35160]: osdmap e112: 12 total, 12 up, 12 in 2024-09-10T11:02:50.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a[30869]: 2024-09-10T11:02:50.067+0000 7f79c36d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:02:50.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a[30869]: 2024-09-10T11:02:50.067+0000 7f79c36d2640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T11:02:50.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:50 smithi007 podman[92869]: 2024-09-10 11:02:50.15008934 +0000 UTC m=+0.209596975 container died bd02b0b05566704b5752f8296b1ec1caeb63c1b6cbd7fa6582570521d29ba87b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T11:02:50.933 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:50 smithi007 podman[92869]: 2024-09-10 11:02:50.758978694 +0000 UTC m=+0.818486325 container cleanup bd02b0b05566704b5752f8296b1ec1caeb63c1b6cbd7fa6582570521d29ba87b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T11:02:50.934 INFO:journalctl@ceph.mon.a.smithi007.stdout:Sep 10 11:02:50 smithi007 bash[92869]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-a 2024-09-10T11:02:51.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T11:02:51.259 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:02:51.260 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T11:02:51.260 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T11:02:51.260 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.b 2024-09-10T11:02:51.557 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:51 smithi103 systemd[1]: Stopping Ceph mon.b for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:02:51.922 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-b[35156]: 2024-09-10T11:02:51.553+0000 7f6f15371640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:02:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-b[35156]: 2024-09-10T11:02:51.553+0000 7f6f15371640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T11:02:51.923 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:51 smithi103 podman[72722]: 2024-09-10 11:02:51.799185235 +0000 UTC m=+0.347651555 container died 85482a8afe418a7c8f5ea77707c79947bf528a457a9a4e8f9697ce47b8144272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T11:02:52.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:52 smithi103 podman[72722]: 2024-09-10 11:02:52.145705786 +0000 UTC m=+0.694172104 container cleanup 85482a8afe418a7c8f5ea77707c79947bf528a457a9a4e8f9697ce47b8144272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-b, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:02:52.422 INFO:journalctl@ceph.mon.b.smithi103.stdout:Sep 10 11:02:52 smithi103 bash[72722]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-b 2024-09-10T11:02:52.571 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T11:02:52.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:02:52.614 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T11:02:52.614 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T11:02:52.614 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.c 2024-09-10T11:02:52.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:52 smithi133 systemd[1]: Stopping Ceph mon.c for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:02:53.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-c[35278]: 2024-09-10T11:02:52.949+0000 7f66d5be8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:02:53.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-c[35278]: 2024-09-10T11:02:52.949+0000 7f66d5be8640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T11:02:53.225 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:52 smithi133 podman[73616]: 2024-09-10 11:02:52.970355398 +0000 UTC m=+0.115018188 container died 973b4a04a53c042575cd49e1ae0231f4da213ba37e064379b287fda4549f4680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-c, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T11:02:53.509 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:53 smithi133 podman[73616]: 2024-09-10 11:02:53.316130824 +0000 UTC m=+0.460793686 container cleanup 973b4a04a53c042575cd49e1ae0231f4da213ba37e064379b287fda4549f4680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-c, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T11:02:53.509 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:53 smithi133 bash[73616]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-c 2024-09-10T11:02:53.736 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T11:02:53.760 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:53 smithi133 podman[73628]: 2024-09-10 11:02:53.508968089 +0000 UTC m=+0.527942946 container remove 973b4a04a53c042575cd49e1ae0231f4da213ba37e064379b287fda4549f4680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T11:02:53.760 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:53 smithi133 systemd[1]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-10T11:02:53.760 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:53 smithi133 systemd[1]: Stopped Ceph mon.c for 65af4d96-6f60-11ef-bcea-c7b262605968. 2024-09-10T11:02:53.760 INFO:journalctl@ceph.mon.c.smithi133.stdout:Sep 10 11:02:53 smithi133 systemd[1]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mon.c.service: Consumed 14.155s CPU time. 2024-09-10T11:02:53.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:02:53.787 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T11:02:53.788 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T11:02:53.788 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.x 2024-09-10T11:02:54.090 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 11:02:53 smithi007 systemd[1]: Stopping Ceph mgr.x for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:02:54.424 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 11:02:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x[31099]: 2024-09-10T11:02:54.088+0000 7fd60d365640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:02:54.424 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 11:02:54 smithi007 podman[92991]: 2024-09-10 11:02:54.119769814 +0000 UTC m=+0.124263764 container died 562a96b83dc9c9c174fedb37d9689f676e946a598b79afe4ca3bab29c04ff24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:02:54.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 11:02:54 smithi007 podman[92991]: 2024-09-10 11:02:54.506116002 +0000 UTC m=+0.510609954 container cleanup 562a96b83dc9c9c174fedb37d9689f676e946a598b79afe4ca3bab29c04ff24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:02:54.684 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Sep 10 11:02:54 smithi007 bash[92991]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-x 2024-09-10T11:02:54.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T11:02:54.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:02:54.982 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T11:02:54.982 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T11:02:54.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.y 2024-09-10T11:02:55.323 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 11:02:55 smithi103 systemd[1]: Stopping Ceph mgr.y for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:02:55.601 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 11:02:55 smithi103 podman[72843]: 2024-09-10 11:02:55.322657128 +0000 UTC m=+0.154813809 container died 70049c4f604ab28f5e9f8f20e1f193c10703dcbafe0e39e3eaabfbd21ea6dbea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T11:02:55.909 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 11:02:55 smithi103 podman[72843]: 2024-09-10 11:02:55.683075893 +0000 UTC m=+0.515232566 container cleanup 70049c4f604ab28f5e9f8f20e1f193c10703dcbafe0e39e3eaabfbd21ea6dbea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T11:02:55.910 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Sep 10 11:02:55 smithi103 bash[72843]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-y 2024-09-10T11:02:56.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T11:02:56.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:02:56.083 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T11:02:56.084 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T11:02:56.084 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.z 2024-09-10T11:02:56.395 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 11:02:56 smithi133 systemd[1]: Stopping Ceph mgr.z for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:02:56.688 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 11:02:56 smithi133 podman[73737]: 2024-09-10 11:02:56.394560028 +0000 UTC m=+0.114909424 container died fda7baca4a5d1a35cf4369cb8b1be257bb5f1631e7ff6dead3d06e24bdbd08c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T11:02:56.975 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 11:02:56 smithi133 podman[73737]: 2024-09-10 11:02:56.770356624 +0000 UTC m=+0.490706015 container cleanup fda7baca4a5d1a35cf4369cb8b1be257bb5f1631e7ff6dead3d06e24bdbd08c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T11:02:56.975 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Sep 10 11:02:56 smithi133 bash[73737]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mgr-z 2024-09-10T11:02:57.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T11:02:57.224 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:02:57.225 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T11:02:57.225 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T11:02:57.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.0 2024-09-10T11:02:57.537 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:02:57 smithi007 systemd[1]: Stopping Ceph osd.0 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:02:57.933 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:02:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0[44756]: 2024-09-10T11:02:57.536+0000 7f342d634640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:02:57.934 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:02:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0[44756]: 2024-09-10T11:02:57.536+0000 7f342d634640 -1 osd.0 112 *** Got signal Terminated *** 2024-09-10T11:02:57.934 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:02:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0[44756]: 2024-09-10T11:02:57.536+0000 7f342d634640 -1 osd.0 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:03:03.338 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:03:03 smithi007 podman[93111]: 2024-09-10 11:03:03.062257678 +0000 UTC m=+5.643818090 container died c6dd876c6dcb7fe92e1bd416a7388490b8beee8313c42b1c313be28cd268517f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T11:03:03.684 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:03:03 smithi007 podman[93111]: 2024-09-10 11:03:03.428193632 +0000 UTC m=+6.009754041 container cleanup c6dd876c6dcb7fe92e1bd416a7388490b8beee8313c42b1c313be28cd268517f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:03:03.684 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:03:03 smithi007 bash[93111]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0 2024-09-10T11:03:04.100 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:03:03 smithi007 podman[93126]: 2024-09-10 11:03:03.729983315 +0000 UTC m=+0.665233251 container remove c6dd876c6dcb7fe92e1bd416a7388490b8beee8313c42b1c313be28cd268517f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:03:04.375 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:03:04 smithi007 podman[93213]: 2024-09-10 11:03:04.002617287 +0000 UTC m=+0.016674413 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:03:04.376 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:03:04 smithi007 podman[93213]: 2024-09-10 11:03:04.122509304 +0000 UTC m=+0.136566417 container create d8e17de56aab5d4ae767d6ef6f603a4a0539be455a4a9a8b48154f1db0679c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:03:04.684 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:03:04 smithi007 podman[93213]: 2024-09-10 11:03:04.37519701 +0000 UTC m=+0.389254123 container init d8e17de56aab5d4ae767d6ef6f603a4a0539be455a4a9a8b48154f1db0679c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T11:03:04.685 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:03:04 smithi007 podman[93213]: 2024-09-10 11:03:04.379080608 +0000 UTC m=+0.393137721 container start d8e17de56aab5d4ae767d6ef6f603a4a0539be455a4a9a8b48154f1db0679c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T11:03:04.685 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:03:04 smithi007 podman[93213]: 2024-09-10 11:03:04.458760766 +0000 UTC m=+0.472817880 container attach d8e17de56aab5d4ae767d6ef6f603a4a0539be455a4a9a8b48154f1db0679c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T11:03:04.685 INFO:journalctl@ceph.osd.0.smithi007.stdout:Sep 10 11:03:04 smithi007 podman[93213]: 2024-09-10 11:03:04.577421953 +0000 UTC m=+0.591479076 container died d8e17de56aab5d4ae767d6ef6f603a4a0539be455a4a9a8b48154f1db0679c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T11:03:05.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T11:03:05.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:03:05.189 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T11:03:05.190 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T11:03:05.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.1 2024-09-10T11:03:05.480 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:05 smithi103 systemd[1]: Stopping Ceph osd.1 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:03:05.745 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:05 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1[39659]: 2024-09-10T11:03:05.477+0000 7f7ac9781640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:03:05.745 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:05 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1[39659]: 2024-09-10T11:03:05.477+0000 7f7ac9781640 -1 osd.1 112 *** Got signal Terminated *** 2024-09-10T11:03:05.745 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:05 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1[39659]: 2024-09-10T11:03:05.477+0000 7f7ac9781640 -1 osd.1 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:03:11.147 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:10 smithi103 podman[72969]: 2024-09-10 11:03:10.860438117 +0000 UTC m=+5.482167076 container died 6fe1f10dba82321e33527028facd59260ea459a5669cf78e70775ad528cae12a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T11:03:11.422 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:11 smithi103 podman[72969]: 2024-09-10 11:03:11.24600544 +0000 UTC m=+5.867734442 container cleanup 6fe1f10dba82321e33527028facd59260ea459a5669cf78e70775ad528cae12a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T11:03:11.422 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:11 smithi103 bash[72969]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1 2024-09-10T11:03:11.862 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:11 smithi103 podman[72998]: 2024-09-10 11:03:11.505552439 +0000 UTC m=+0.638976532 container remove 6fe1f10dba82321e33527028facd59260ea459a5669cf78e70775ad528cae12a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T11:03:12.134 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:11 smithi103 podman[73088]: 2024-09-10 11:03:11.764507448 +0000 UTC m=+0.015647839 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:03:12.135 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:11 smithi103 podman[73088]: 2024-09-10 11:03:11.881414309 +0000 UTC m=+0.132554686 container create 09929e4fb4ead744b8cdc0e4259dce0e5c68a425b7bad4ad768672da2ce19d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T11:03:12.135 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:12 smithi103 podman[73088]: 2024-09-10 11:03:12.133947462 +0000 UTC m=+0.385087839 container init 09929e4fb4ead744b8cdc0e4259dce0e5c68a425b7bad4ad768672da2ce19d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T11:03:12.422 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:12 smithi103 podman[73088]: 2024-09-10 11:03:12.137409311 +0000 UTC m=+0.388549697 container start 09929e4fb4ead744b8cdc0e4259dce0e5c68a425b7bad4ad768672da2ce19d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T11:03:12.422 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:12 smithi103 podman[73088]: 2024-09-10 11:03:12.225612224 +0000 UTC m=+0.476752613 container attach 09929e4fb4ead744b8cdc0e4259dce0e5c68a425b7bad4ad768672da2ce19d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:03:12.423 INFO:journalctl@ceph.osd.1.smithi103.stdout:Sep 10 11:03:12 smithi103 podman[73088]: 2024-09-10 11:03:12.328526264 +0000 UTC m=+0.579666644 container died 09929e4fb4ead744b8cdc0e4259dce0e5c68a425b7bad4ad768672da2ce19d5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T11:03:12.992 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T11:03:13.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:03:13.033 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T11:03:13.034 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T11:03:13.034 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.2 2024-09-10T11:03:13.350 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:13 smithi133 systemd[1]: Stopping Ceph osd.2 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:03:13.725 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:13 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2[40258]: 2024-09-10T11:03:13.348+0000 7f2c49c90640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:03:13.725 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:13 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2[40258]: 2024-09-10T11:03:13.348+0000 7f2c49c90640 -1 osd.2 112 *** Got signal Terminated *** 2024-09-10T11:03:13.726 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:13 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2[40258]: 2024-09-10T11:03:13.348+0000 7f2c49c90640 -1 osd.2 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:03:19.107 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:18 smithi133 podman[73873]: 2024-09-10 11:03:18.837349957 +0000 UTC m=+5.606169665 container died 0ac99fa985e7f5c083ed41d4bffa385463471ca1aa392d6179364bfe2bbd9c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T11:03:19.367 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:19 smithi133 podman[73873]: 2024-09-10 11:03:19.107012138 +0000 UTC m=+5.875831844 container cleanup 0ac99fa985e7f5c083ed41d4bffa385463471ca1aa392d6179364bfe2bbd9c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T11:03:19.367 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:19 smithi133 bash[73873]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2 2024-09-10T11:03:19.676 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:19 smithi133 podman[73890]: 2024-09-10 11:03:19.366678326 +0000 UTC m=+0.527970779 container remove 0ac99fa985e7f5c083ed41d4bffa385463471ca1aa392d6179364bfe2bbd9c5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T11:03:19.945 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:19 smithi133 podman[73980]: 2024-09-10 11:03:19.578299001 +0000 UTC m=+0.016022888 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:03:19.945 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:19 smithi133 podman[73980]: 2024-09-10 11:03:19.709230068 +0000 UTC m=+0.146953942 container create 5239e782ba739f93da217a5d034cc191dc5f59e5f4fce972ef817057a31d7092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T11:03:19.946 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:19 smithi133 podman[73980]: 2024-09-10 11:03:19.944860663 +0000 UTC m=+0.382584537 container init 5239e782ba739f93da217a5d034cc191dc5f59e5f4fce972ef817057a31d7092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T11:03:20.225 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:19 smithi133 podman[73980]: 2024-09-10 11:03:19.948583674 +0000 UTC m=+0.386307550 container start 5239e782ba739f93da217a5d034cc191dc5f59e5f4fce972ef817057a31d7092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T11:03:20.225 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:20 smithi133 podman[73980]: 2024-09-10 11:03:20.078210493 +0000 UTC m=+0.515934378 container attach 5239e782ba739f93da217a5d034cc191dc5f59e5f4fce972ef817057a31d7092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T11:03:20.532 INFO:journalctl@ceph.osd.2.smithi133.stdout:Sep 10 11:03:20 smithi133 podman[73980]: 2024-09-10 11:03:20.25902472 +0000 UTC m=+0.696748601 container died 5239e782ba739f93da217a5d034cc191dc5f59e5f4fce972ef817057a31d7092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T11:03:20.834 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T11:03:20.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:03:20.874 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T11:03:20.874 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T11:03:20.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.3 2024-09-10T11:03:21.183 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:20 smithi007 systemd[1]: Stopping Ceph osd.3 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:03:21.684 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:21 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3[51694]: 2024-09-10T11:03:21.206+0000 7f45891c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:03:21.684 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:21 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3[51694]: 2024-09-10T11:03:21.206+0000 7f45891c1640 -1 osd.3 112 *** Got signal Terminated *** 2024-09-10T11:03:21.684 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:21 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3[51694]: 2024-09-10T11:03:21.206+0000 7f45891c1640 -1 osd.3 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:03:23.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:23.560+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:24.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:23 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:23.809+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:24.183 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:23 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:23.864+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:24.684 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:24 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3[51694]: 2024-09-10T11:03:24.311+0000 7f45817be640 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:02.048068+0000 front 2024-09-10T11:03:02.048046+0000 (oldest deadline 2024-09-10T11:03:23.748221+0000) 2024-09-10T11:03:24.725 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:24.605+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:24.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:24.421+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:25.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:24 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:24.807+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:25.183 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:24 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:24.826+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:25.684 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:25 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3[51694]: 2024-09-10T11:03:25.339+0000 7f45817be640 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:02.048068+0000 front 2024-09-10T11:03:02.048046+0000 (oldest deadline 2024-09-10T11:03:23.748221+0000) 2024-09-10T11:03:25.725 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:25 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:25.628+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:25.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:25 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:25.404+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:26.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:25 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:25.845+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:26.183 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:25 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:25.842+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:26.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:25 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:25.989+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:26.646 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:26 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:26.379+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:26.932 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:26 smithi007 podman[93329]: 2024-09-10 11:03:26.659240399 +0000 UTC m=+5.571788349 container died 7ef99490c4b15b862db6485729ea3b1849bd013d0b2f265a79fcf95b3ecdd5d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T11:03:26.933 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:26 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:26.860+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:26.976 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:26 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:26.645+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:27.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:26 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:26.801+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:27.183 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:26 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:26.964+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:27.184 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:27 smithi007 podman[93329]: 2024-09-10 11:03:27.031550325 +0000 UTC m=+5.944098264 container cleanup 7ef99490c4b15b862db6485729ea3b1849bd013d0b2f265a79fcf95b3ecdd5d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-09-10T11:03:27.184 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:27 smithi007 bash[93329]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3 2024-09-10T11:03:27.657 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:27 smithi007 podman[93346]: 2024-09-10 11:03:27.307871262 +0000 UTC m=+0.639556460 container remove 7ef99490c4b15b862db6485729ea3b1849bd013d0b2f265a79fcf95b3ecdd5d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T11:03:27.670 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:27 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:27.411+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:27.918 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:27 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:27.817+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:27.918 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:27 smithi007 podman[93437]: 2024-09-10 11:03:27.559879127 +0000 UTC m=+0.017320739 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:03:27.919 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:27 smithi007 podman[93437]: 2024-09-10 11:03:27.658669961 +0000 UTC m=+0.116111562 container create b6c45277d72fc69e82d1b502f720e407c549653ea1a256f958b50289f8537317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T11:03:27.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:27 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:27.669+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:28.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:27 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:27.828+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:28.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:27 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:27.916+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:28.184 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:27 smithi007 podman[93437]: 2024-09-10 11:03:27.919399242 +0000 UTC m=+0.376840845 container init b6c45277d72fc69e82d1b502f720e407c549653ea1a256f958b50289f8537317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:03:28.184 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:27 smithi007 podman[93437]: 2024-09-10 11:03:27.922777255 +0000 UTC m=+0.380218859 container start b6c45277d72fc69e82d1b502f720e407c549653ea1a256f958b50289f8537317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T11:03:28.184 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:28 smithi007 podman[93437]: 2024-09-10 11:03:28.044958667 +0000 UTC m=+0.502400285 container attach b6c45277d72fc69e82d1b502f720e407c549653ea1a256f958b50289f8537317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:03:28.185 INFO:journalctl@ceph.osd.3.smithi007.stdout:Sep 10 11:03:28 smithi007 podman[93437]: 2024-09-10 11:03:28.115743376 +0000 UTC m=+0.573184978 container died b6c45277d72fc69e82d1b502f720e407c549653ea1a256f958b50289f8537317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T11:03:28.677 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:28 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:28.418+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:28.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T11:03:28.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:03:28.766 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T11:03:28.766 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T11:03:28.766 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.4 2024-09-10T11:03:28.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:28 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:28.676+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:29.058 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:28 smithi103 systemd[1]: Stopping Ceph osd.4 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:03:29.058 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:28 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:28.781+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:29.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:28 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:28.884+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:29.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:28 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:28.820+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:29.422 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:29 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4[46574]: 2024-09-10T11:03:29.057+0000 7fb577eed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:03:29.422 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:29 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4[46574]: 2024-09-10T11:03:29.057+0000 7fb577eed640 -1 osd.4 112 *** Got signal Terminated *** 2024-09-10T11:03:29.422 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:29 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4[46574]: 2024-09-10T11:03:29.057+0000 7fb577eed640 -1 osd.4 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:03:29.709 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:29 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:29.384+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:29.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:29 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:29.708+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:30.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:29 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:29.757+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:30.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:29 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:29.869+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:30.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:29 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:29.888+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:30.686 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:30 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:30.355+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:30.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:30 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:30.685+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:31.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:30 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:30.799+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:31.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:30 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:30.877+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:31.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:30 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:30.905+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:31.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:30 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:30.905+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:31.704 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:31 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:31.321+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:31.705 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:31 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:31.321+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:31.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:31 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:31.703+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:32.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:31 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:31.904+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:32.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:31 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:31.821+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:32.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:31 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:31.821+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:32.183 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:31 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:31.851+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:32.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:31 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:31.948+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:32.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:31 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:31.948+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:32.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:32 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:32.328+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:32.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:32 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:32.328+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:33.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:32 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:32.943+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:33.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:32 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:32.843+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:33.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:32 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:32.843+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:33.183 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:32 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:32.816+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:33.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:32 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:32.988+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:33.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:32 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:32.988+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:33.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:32 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:32.741+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:33.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:33 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:33.247+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:33.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:33 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:33.370+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:33.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:33 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:33.370+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:34.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:33 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:33.966+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:34.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:33 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:33.803+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:34.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:33 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:33.803+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:34.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:33 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:33.861+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:34.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:34 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:34.024+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:34.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:34 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:34.024+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:34.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:33 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:33.763+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:34.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:33 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:33.763+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.923006+0000 front 2024-09-10T11:03:09.922987+0000 (oldest deadline 2024-09-10T11:03:33.422947+0000) 2024-09-10T11:03:34.501 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:34 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:34.285+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:34.501 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:34 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:34.375+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:34.502 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:34 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:34.375+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:34.791 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:34 smithi103 podman[73190]: 2024-09-10 11:03:34.517115827 +0000 UTC m=+5.554364935 container died fdfec9708c9f2ccbe21a720df95768885bb6aa5e2cc915c9beeee7395a730821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T11:03:35.150 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:34 smithi103 podman[73190]: 2024-09-10 11:03:34.890256903 +0000 UTC m=+5.927506010 container cleanup fdfec9708c9f2ccbe21a720df95768885bb6aa5e2cc915c9beeee7395a730821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux ) 2024-09-10T11:03:35.151 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:34 smithi103 bash[73190]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4 2024-09-10T11:03:35.151 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:34 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:34.944+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:35.151 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:34 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:34.836+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:35.152 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:34 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:34.836+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:35.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:34 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:34.866+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:35.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:35 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:35.068+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:35.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:35 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:35.068+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:35.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:34 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:34.809+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:35.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:34 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:34.809+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.923006+0000 front 2024-09-10T11:03:09.922987+0000 (oldest deadline 2024-09-10T11:03:33.422947+0000) 2024-09-10T11:03:35.422 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:35 smithi103 podman[73205]: 2024-09-10 11:03:35.150069702 +0000 UTC m=+0.628536440 container remove fdfec9708c9f2ccbe21a720df95768885bb6aa5e2cc915c9beeee7395a730821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:03:35.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:35 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:35.312+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:35.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:35 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:35.382+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:35.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:35 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:35.382+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:35.762 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:35 smithi103 podman[73293]: 2024-09-10 11:03:35.403542743 +0000 UTC m=+0.015728589 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:03:35.762 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:35 smithi103 podman[73293]: 2024-09-10 11:03:35.517628685 +0000 UTC m=+0.129814509 container create eea5fbab2a157a8e7d6a89ba1507f465c53196305604ea13166762d260cc4a5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:03:36.117 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:35 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:35.843+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:36.117 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:35 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:35.843+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:36.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:35 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:35.957+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:36.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:35 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:35.866+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:36.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:35 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:35.866+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:36.173 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:35 smithi103 podman[73293]: 2024-09-10 11:03:35.761974994 +0000 UTC m=+0.374160817 container init eea5fbab2a157a8e7d6a89ba1507f465c53196305604ea13166762d260cc4a5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:03:36.173 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:35 smithi103 podman[73293]: 2024-09-10 11:03:35.76562249 +0000 UTC m=+0.377808314 container start eea5fbab2a157a8e7d6a89ba1507f465c53196305604ea13166762d260cc4a5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T11:03:36.173 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:35 smithi103 podman[73293]: 2024-09-10 11:03:35.836930765 +0000 UTC m=+0.449116610 container attach eea5fbab2a157a8e7d6a89ba1507f465c53196305604ea13166762d260cc4a5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T11:03:36.174 INFO:journalctl@ceph.osd.4.smithi103.stdout:Sep 10 11:03:35 smithi103 podman[73293]: 2024-09-10 11:03:35.94960863 +0000 UTC m=+0.561794472 container died eea5fbab2a157a8e7d6a89ba1507f465c53196305604ea13166762d260cc4a5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:03:36.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:35 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:35.781+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:36.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:35 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:35.781+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.923006+0000 front 2024-09-10T11:03:09.922987+0000 (oldest deadline 2024-09-10T11:03:33.422947+0000) 2024-09-10T11:03:36.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:36 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:36.115+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:36.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:36 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:36.115+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:36.572 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T11:03:36.612 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:03:36.612 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T11:03:36.612 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T11:03:36.613 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.5 2024-09-10T11:03:36.640 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:36 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:36.276+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:36.640 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:36 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:36.367+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:36.641 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:36 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:36.367+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:36.951 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:36 smithi133 systemd[1]: Stopping Ceph osd.5 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:03:36.952 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:36 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:36.786+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:36.952 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:36 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:36.786+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.923006+0000 front 2024-09-10T11:03:09.922987+0000 (oldest deadline 2024-09-10T11:03:33.422947+0000) 2024-09-10T11:03:36.952 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:36 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:36.949+0000 7fc68e1c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:03:36.952 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:36 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:36.949+0000 7fc68e1c5640 -1 osd.5 112 *** Got signal Terminated *** 2024-09-10T11:03:36.952 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:36 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:36.949+0000 7fc68e1c5640 -1 osd.5 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:03:37.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:36 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:36.930+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:37.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:36 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:36.892+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:37.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:36 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:36.892+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:37.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:37 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:37.089+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:37.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:37 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:37.089+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:37.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:36 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:36.861+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:37.185 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:36 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:36.861+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:37.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:37 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:37.301+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:37.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:37 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:37.414+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:37.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:37 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:37.414+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:38.127 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:37 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:37.820+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:38.127 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:37 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:37.820+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:38.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:37 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:37.958+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:38.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:37 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:37.881+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:38.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:37 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:37.881+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:38.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:37 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:37.764+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:38.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:37 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:37.764+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.923006+0000 front 2024-09-10T11:03:09.922987+0000 (oldest deadline 2024-09-10T11:03:33.422947+0000) 2024-09-10T11:03:38.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:38 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:38.126+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:38.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:38 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:38.126+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:38.723 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:38 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:38.299+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:38.724 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:38 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:38.366+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:38.724 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:38 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:38.366+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:38.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:38 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:38.721+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:38.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:38 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:38.721+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.923006+0000 front 2024-09-10T11:03:09.922987+0000 (oldest deadline 2024-09-10T11:03:33.422947+0000) 2024-09-10T11:03:39.097 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:38 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:38.791+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:39.097 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:38 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:38.791+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:39.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:38 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:38.981+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:39.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:38 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:38.844+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:39.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:38 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:38.844+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:39.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:39 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:39.095+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:39.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:39 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:39.095+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:39.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:39 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:39.252+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:39.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:39 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:39.410+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:39.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:39 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:39.410+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:40.116 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:39 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:39.834+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:40.117 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:39 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:39.834+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:40.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:40 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:40.010+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:40.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:39 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:39.892+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:40.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:39 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:39.892+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:40.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:39 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:39.723+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:40.225 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:39 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:39.723+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.923006+0000 front 2024-09-10T11:03:09.922987+0000 (oldest deadline 2024-09-10T11:03:33.422947+0000) 2024-09-10T11:03:40.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:40.115+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:40.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:40.115+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:40.698 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:40 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:40.242+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:40.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:40 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:40.368+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:40.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:40 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:40.368+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:40.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:40 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:40.697+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:40.976 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:40 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:40.697+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.923006+0000 front 2024-09-10T11:03:09.922987+0000 (oldest deadline 2024-09-10T11:03:33.422947+0000) 2024-09-10T11:03:41.156 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:40.825+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:41.157 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:40.825+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:41.157 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:40.825+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:41.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:41 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:41.025+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:41.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:41 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:41.025+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:41.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:40 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:40.890+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:41.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:40 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:40.890+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:41.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:40 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:40.890+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:41.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:41 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:41.155+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:41.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:41 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:41.155+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:41.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:41 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:41.155+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.296198+0000 front 2024-09-10T11:03:17.296353+0000 (oldest deadline 2024-09-10T11:03:40.796287+0000) 2024-09-10T11:03:41.657 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:41 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:41.248+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:41.658 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:41 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:41.399+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:41.658 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:41 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:41.399+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:41.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:41 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:41.656+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.021267+0000 front 2024-09-10T11:03:00.021113+0000 (oldest deadline 2024-09-10T11:03:23.521250+0000) 2024-09-10T11:03:41.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:41 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:41.656+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.923006+0000 front 2024-09-10T11:03:09.922987+0000 (oldest deadline 2024-09-10T11:03:33.422947+0000) 2024-09-10T11:03:41.976 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:41 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5[47214]: 2024-09-10T11:03:41.656+0000 7fc68a7ca640 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.123817+0000 front 2024-09-10T11:03:18.123843+0000 (oldest deadline 2024-09-10T11:03:41.024043+0000) 2024-09-10T11:03:42.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:42.000+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:42.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:42.000+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:42.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:41 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:41.901+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:42.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:41 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:41.901+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:42.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:41 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:41.901+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:42.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:41 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:41.836+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:42.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:41 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:41.836+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:42.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:41 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:41.836+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:42.677 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:42 smithi133 podman[74106]: 2024-09-10 11:03:42.359635627 +0000 UTC m=+5.549389102 container died 1ccc6f6269958c665099ddd848837426190db921224cfaba7b9eb990862314bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:03:42.678 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:42.226+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:42.678 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:42.226+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:42.678 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:42.391+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:42.678 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:42 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:42.391+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:42.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:42 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:42.196+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:42.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:42 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:42.196+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:42.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:42 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:42.196+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.296198+0000 front 2024-09-10T11:03:17.296353+0000 (oldest deadline 2024-09-10T11:03:40.796287+0000) 2024-09-10T11:03:42.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:42 smithi133 podman[74106]: 2024-09-10 11:03:42.758513835 +0000 UTC m=+5.948267309 container cleanup 1ccc6f6269958c665099ddd848837426190db921224cfaba7b9eb990862314bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T11:03:42.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:42 smithi133 bash[74106]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5 2024-09-10T11:03:43.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:43 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:43.037+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:43.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:43 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:43.037+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:43.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:42.897+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:43.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:42.897+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:43.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:42 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:42.897+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:43.183 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:42 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:42.819+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:43.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:42 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:42.819+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:43.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:42 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:42.819+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:43.336 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:43 smithi133 podman[74123]: 2024-09-10 11:03:43.005252712 +0000 UTC m=+0.641873448 container remove 1ccc6f6269958c665099ddd848837426190db921224cfaba7b9eb990862314bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T11:03:43.336 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:43 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:43.222+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:43.336 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:43 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:43.222+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:43.655 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:43 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:43.429+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:43.655 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:43 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:43.429+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:43.656 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:43 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:43.429+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:43.656 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:43 smithi133 podman[74211]: 2024-09-10 11:03:43.237948225 +0000 UTC m=+0.016077676 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:03:43.656 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:43 smithi133 podman[74211]: 2024-09-10 11:03:43.335600302 +0000 UTC m=+0.113729737 container create 0cb4e17874985736a09c076b10d356b6574430d51b9bc164945f64d3bb7051f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T11:03:43.656 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:43 smithi133 podman[74211]: 2024-09-10 11:03:43.571489131 +0000 UTC m=+0.349618565 container init 0cb4e17874985736a09c076b10d356b6574430d51b9bc164945f64d3bb7051f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T11:03:43.656 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:43 smithi133 podman[74211]: 2024-09-10 11:03:43.574847358 +0000 UTC m=+0.352976804 container start 0cb4e17874985736a09c076b10d356b6574430d51b9bc164945f64d3bb7051f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:03:43.656 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:43 smithi133 podman[74211]: 2024-09-10 11:03:43.654828984 +0000 UTC m=+0.432958427 container attach 0cb4e17874985736a09c076b10d356b6574430d51b9bc164945f64d3bb7051f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T11:03:43.683 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:43.241+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:43.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:43.241+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:43.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:43.241+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.296198+0000 front 2024-09-10T11:03:17.296353+0000 (oldest deadline 2024-09-10T11:03:40.796287+0000) 2024-09-10T11:03:43.975 INFO:journalctl@ceph.osd.5.smithi133.stdout:Sep 10 11:03:43 smithi133 podman[74211]: 2024-09-10 11:03:43.762987104 +0000 UTC m=+0.541116555 container died 0cb4e17874985736a09c076b10d356b6574430d51b9bc164945f64d3bb7051f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:03:44.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:43.853+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:44.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:43.853+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:44.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:43 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:43.853+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:44.333 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:44 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:44.228+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:44.333 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:44 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:44.228+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:44.358 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T11:03:44.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:44 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:44.085+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:44.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:44 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:44.085+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:44.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:43 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:43.943+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:44.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:43 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:43.943+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:44.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:43 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:43.943+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:44.437 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:03:44.437 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T11:03:44.437 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T11:03:44.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.6 2024-09-10T11:03:44.494 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:44.240+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:44.494 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:44.240+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:44.495 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:44.240+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.296198+0000 front 2024-09-10T11:03:17.296353+0000 (oldest deadline 2024-09-10T11:03:40.796287+0000) 2024-09-10T11:03:44.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:44 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:44.414+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:44.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:44 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:44.414+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:44.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:44 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:44.414+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:44.753 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:44 smithi007 systemd[1]: Stopping Ceph osd.6 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:03:44.753 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:44.751+0000 7efcc019e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:03:44.753 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:44.751+0000 7efcc019e640 -1 osd.6 112 *** Got signal Terminated *** 2024-09-10T11:03:45.184 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:44.751+0000 7efcc019e640 -1 osd.6 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:03:45.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:44.851+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:45.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:44.851+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:45.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:44 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:44.851+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:45.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:45 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:45.103+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:45.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:45 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:45.103+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:45.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:44 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:44.983+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:45.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:44 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:44.983+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:45.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:44 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:44.983+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:45.528 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:45 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:45.247+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:45.528 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:45 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:45.247+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:45.529 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:45 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:45.388+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:45.529 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:45 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:45.388+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:45.529 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:45 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:45.388+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:45.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:45.274+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:45.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:45.274+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:45.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:45.274+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.296198+0000 front 2024-09-10T11:03:17.296353+0000 (oldest deadline 2024-09-10T11:03:40.796287+0000) 2024-09-10T11:03:46.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:45.900+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:46.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:45.900+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:46.184 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:45.900+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:46.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:46 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:46.115+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:46.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:46 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:46.115+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:46.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:45 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:45.989+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:46.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:45 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:45.989+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:46.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:45 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:45.989+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:46.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:46 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:46.241+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:46.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:46 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:46.241+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:46.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:46 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:46.241+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.296198+0000 front 2024-09-10T11:03:17.296353+0000 (oldest deadline 2024-09-10T11:03:40.796287+0000) 2024-09-10T11:03:46.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:46 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:46.266+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:46.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:46 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:46.266+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:46.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:46 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:46.435+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:46.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:46 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:46.435+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:46.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:46 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:46.435+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:47.205 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:46 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:46.947+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:47.205 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:46 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:46.947+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:47.205 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:46 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:46.947+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:47.280 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:47 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:47.069+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:47.281 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:47 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:47.069+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:47.281 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:47 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:47.024+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:47.281 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:47 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:47.024+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:47.281 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:47 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:47.024+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:47.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:47 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:47.204+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:47.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:47 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:47.204+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:47.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:47 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:47.204+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.296198+0000 front 2024-09-10T11:03:17.296353+0000 (oldest deadline 2024-09-10T11:03:40.796287+0000) 2024-09-10T11:03:47.684 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:47 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:47.204+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:24.198183+0000 front 2024-09-10T11:03:24.198159+0000 (oldest deadline 2024-09-10T11:03:46.999060+0000) 2024-09-10T11:03:47.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:47 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:47.272+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:47.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:47 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:47.272+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:47.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:47 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:47.407+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:47.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:47 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:47.407+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:47.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:47 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:47.407+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:48.172 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:48 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:48.070+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:48.173 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:48 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:48.070+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:48.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:48 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:48.010+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:48.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:48 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:48.010+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:48.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:48 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:48.010+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:48.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:47 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:47.995+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:48.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:47 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:47.995+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:48.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:47 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:47.995+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:48.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:48 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:48.213+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:48.435 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:48 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:48.213+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:48.435 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:48 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:48.213+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.296198+0000 front 2024-09-10T11:03:17.296353+0000 (oldest deadline 2024-09-10T11:03:40.796287+0000) 2024-09-10T11:03:48.435 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:48 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:48.213+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:24.198183+0000 front 2024-09-10T11:03:24.198159+0000 (oldest deadline 2024-09-10T11:03:46.999060+0000) 2024-09-10T11:03:48.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:48.260+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:48.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:48.260+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:48.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:48.260+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:48.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:48.435+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:48.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:48.435+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:48.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:48 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:48.435+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:49.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:49.074+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:49.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:49.074+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:49.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:49.026+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:49.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:49.026+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:49.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:49 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:49.026+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:49.433 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:49.003+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:49.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:49.003+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:49.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:49.003+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:49.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:49.248+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:01.890648+0000 front 2024-09-10T11:03:01.890714+0000 (oldest deadline 2024-09-10T11:03:23.590805+0000) 2024-09-10T11:03:49.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:49.248+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:08.594120+0000 front 2024-09-10T11:03:08.594106+0000 (oldest deadline 2024-09-10T11:03:30.894202+0000) 2024-09-10T11:03:49.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:49.248+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.296198+0000 front 2024-09-10T11:03:17.296353+0000 (oldest deadline 2024-09-10T11:03:40.796287+0000) 2024-09-10T11:03:49.435 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6[58633]: 2024-09-10T11:03:49.248+0000 7efcb879b640 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:24.198183+0000 front 2024-09-10T11:03:24.198159+0000 (oldest deadline 2024-09-10T11:03:46.999060+0000) 2024-09-10T11:03:49.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:49.269+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:49.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:49.269+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:49.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:49.269+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:49.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:49.458+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:49.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:49.458+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:49.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:49 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:49.458+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:50.328 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:49.955+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:50.329 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:49.955+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:50.329 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:49 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:49.955+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:50.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:50.029+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:50.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:50.029+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:50.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:50.025+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:50.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:50.025+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:50.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:50.025+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:50.611 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:50 smithi007 podman[93552]: 2024-09-10 11:03:50.327939792 +0000 UTC m=+5.679725366 container died 0313a75fd994d9dda9f86b6e1d35914b41eceb5dd94b1759ad2da58250c34e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:03:50.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:50 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:50.247+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:50.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:50 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:50.247+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:50.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:50 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:50.247+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:50.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:50 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:50.426+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:50.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:50 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:50.426+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:50.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:50 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:50.426+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:50.937 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:50 smithi007 podman[93552]: 2024-09-10 11:03:50.701206733 +0000 UTC m=+6.052992314 container cleanup 0313a75fd994d9dda9f86b6e1d35914b41eceb5dd94b1759ad2da58250c34e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:03:50.938 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:50 smithi007 bash[93552]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6 2024-09-10T11:03:51.304 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:50 smithi007 podman[93567]: 2024-09-10 11:03:50.969313415 +0000 UTC m=+0.633779240 container remove 0313a75fd994d9dda9f86b6e1d35914b41eceb5dd94b1759ad2da58250c34e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T11:03:51.305 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:50.954+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:51.305 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:50.954+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:51.305 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:50 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:50.954+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:51.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:51.019+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:51.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:51.019+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:51.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:51.019+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.648743+0000 front 2024-09-10T11:03:25.648756+0000 (oldest deadline 2024-09-10T11:03:50.948729+0000) 2024-09-10T11:03:51.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:50.977+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:51.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:50.977+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:51.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:50.977+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:51.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:50.977+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:03:51.606 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:51 smithi007 podman[93656]: 2024-09-10 11:03:51.207037578 +0000 UTC m=+0.016490197 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:03:51.607 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:51 smithi007 podman[93656]: 2024-09-10 11:03:51.36217964 +0000 UTC m=+0.171632298 container create 772c3139abcdde08166469239fa421d4d219e5585cdc1b65be0d42e3a84cc7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T11:03:51.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:51.295+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:51.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:51.295+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:51.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:51.295+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:51.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:51.400+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:51.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:51.400+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:51.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:51 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:51.400+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:51.934 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:51 smithi007 podman[93656]: 2024-09-10 11:03:51.606198711 +0000 UTC m=+0.415651322 container init 772c3139abcdde08166469239fa421d4d219e5585cdc1b65be0d42e3a84cc7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:03:51.934 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:51 smithi007 podman[93656]: 2024-09-10 11:03:51.609642619 +0000 UTC m=+0.419095224 container start 772c3139abcdde08166469239fa421d4d219e5585cdc1b65be0d42e3a84cc7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T11:03:51.935 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:51 smithi007 podman[93656]: 2024-09-10 11:03:51.706310743 +0000 UTC m=+0.515763362 container attach 772c3139abcdde08166469239fa421d4d219e5585cdc1b65be0d42e3a84cc7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T11:03:51.935 INFO:journalctl@ceph.osd.6.smithi007.stdout:Sep 10 11:03:51 smithi007 podman[93656]: 2024-09-10 11:03:51.803534507 +0000 UTC m=+0.612987117 container died 772c3139abcdde08166469239fa421d4d219e5585cdc1b65be0d42e3a84cc7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T11:03:52.377 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:51 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:51.995+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:52.378 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:51 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:51.995+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:52.378 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:51 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:51.995+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:52.378 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:51 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:51.995+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:03:52.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T11:03:52.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:51.978+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:52.423 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:51.978+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:52.423 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:51.978+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.648743+0000 front 2024-09-10T11:03:25.648756+0000 (oldest deadline 2024-09-10T11:03:50.948729+0000) 2024-09-10T11:03:52.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:51.939+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:52.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:51.939+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:52.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:51.939+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:52.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:51 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:51.939+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:03:52.448 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:03:52.449 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T11:03:52.449 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T11:03:52.449 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.7 2024-09-10T11:03:52.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:52.270+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:52.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:52.270+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:52.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:52.270+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:52.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:52.354+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:52.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:52.354+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:52.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:52 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:52.354+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:52.773 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:52 smithi103 systemd[1]: Stopping Ceph osd.7 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:03:53.173 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:52.767+0000 7fe958aaf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:03:53.173 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:52.767+0000 7fe958aaf640 -1 osd.7 112 *** Got signal Terminated *** 2024-09-10T11:03:53.174 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:52.767+0000 7fe958aaf640 -1 osd.7 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:03:53.174 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:52.976+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:53.174 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:52.976+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:53.174 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:52.976+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.648743+0000 front 2024-09-10T11:03:25.648756+0000 (oldest deadline 2024-09-10T11:03:50.948729+0000) 2024-09-10T11:03:53.175 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:52.989+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:53.175 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:52.989+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:53.175 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:52.989+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:53.175 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:52 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:52.989+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:03:53.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:53 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:53.031+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:53.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:53 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:53.031+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:53.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:53 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:53.031+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:53.435 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:53 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:53.031+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:03:53.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:53 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:53.297+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:53.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:53 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:53.297+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:53.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:53 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:53.297+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:53.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:53 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:53.330+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:53.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:53 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:53.330+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:53.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:53 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:53.330+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:54.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:53 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:53.992+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:54.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:53 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:53.992+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:54.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:53 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:53.992+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.648743+0000 front 2024-09-10T11:03:25.648756+0000 (oldest deadline 2024-09-10T11:03:50.948729+0000) 2024-09-10T11:03:54.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:53 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:53.976+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:54.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:53 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:53.976+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:54.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:53 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:53.976+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:54.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:53 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:53.976+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:03:54.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:54.070+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:54.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:54.070+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:54.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:54.070+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:54.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:54 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:54.070+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:03:54.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:54 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:54.341+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:54.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:54 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:54.341+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:54.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:54 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:54.341+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:54.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:54 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:54.327+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:54.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:54 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:54.327+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:54.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:54 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:54.327+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:55.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:55 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:55.009+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:55.423 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:55 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:55.009+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:55.423 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:55 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:55.009+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.648743+0000 front 2024-09-10T11:03:25.648756+0000 (oldest deadline 2024-09-10T11:03:50.948729+0000) 2024-09-10T11:03:55.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:54 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:54.976+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:55.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:54 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:54.976+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:55.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:54 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:54.976+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:55.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:54 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:54.976+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:03:55.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:55.051+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:55.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:55.051+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:55.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:55.051+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:55.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:55 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:55.051+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:03:55.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:55 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:55.357+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:55.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:55 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:55.357+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:55.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:55 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:55.357+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:55.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:55 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:55.335+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:55.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:55 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:55.335+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:55.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:55 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:55.335+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:56.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:56 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:56.059+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:56.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:56 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:56.059+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:56.422 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:56 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:56.059+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.648743+0000 front 2024-09-10T11:03:25.648756+0000 (oldest deadline 2024-09-10T11:03:50.948729+0000) 2024-09-10T11:03:56.423 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:56 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:56.059+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:30.949149+0000 front 2024-09-10T11:03:30.949173+0000 (oldest deadline 2024-09-10T11:03:55.049411+0000) 2024-09-10T11:03:56.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:55 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:55.944+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:56.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:55 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:55.944+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:56.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:55 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:55.944+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:56.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:55 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:55.944+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:03:56.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:56 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:56.082+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:56.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:56 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:56.082+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:56.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:56 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:56.082+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:56.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:56 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:56.082+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:03:56.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:56 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:56.082+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:03:56.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:56 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:56.338+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:56.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:56 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:56.338+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:56.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:56 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:56.338+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:56.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:56 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:56.316+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:56.727 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:56 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:56.316+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:56.727 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:56 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:56.316+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:56.727 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:56 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:56.316+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:32.624271+0000 front 2024-09-10T11:03:32.624204+0000 (oldest deadline 2024-09-10T11:03:55.424701+0000) 2024-09-10T11:03:57.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:56 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:56.962+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:57.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:56 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:56.962+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:57.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:56 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:56.962+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:57.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:56 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:56.962+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:03:57.423 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:57 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:57.053+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.945807+0000 front 2024-09-10T11:03:09.945834+0000 (oldest deadline 2024-09-10T11:03:31.645946+0000) 2024-09-10T11:03:57.423 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:57 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:57.053+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:16.946965+0000 front 2024-09-10T11:03:16.946942+0000 (oldest deadline 2024-09-10T11:03:40.947566+0000) 2024-09-10T11:03:57.423 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:57 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:57.053+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.648743+0000 front 2024-09-10T11:03:25.648756+0000 (oldest deadline 2024-09-10T11:03:50.948729+0000) 2024-09-10T11:03:57.424 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:57 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7[53549]: 2024-09-10T11:03:57.053+0000 7fe9550b4640 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:30.949149+0000 front 2024-09-10T11:03:30.949173+0000 (oldest deadline 2024-09-10T11:03:55.049411+0000) 2024-09-10T11:03:57.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:57.109+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:57.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:57.109+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:57.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:57.109+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:57.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:57.109+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:03:57.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:57 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:57.109+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:03:57.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:57 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:57.290+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:57.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:57 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:57.290+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:57.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:57 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:57.290+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:57.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:57 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:57.355+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:57.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:57 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:57.355+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:57.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:57 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:57.355+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:57.727 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:57 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:57.355+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:32.624271+0000 front 2024-09-10T11:03:32.624204+0000 (oldest deadline 2024-09-10T11:03:55.424701+0000) 2024-09-10T11:03:58.212 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:57 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:57.927+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:58.212 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:57 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:57.927+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:58.212 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:57 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:57.927+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:58.212 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:57 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:57.927+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:03:58.213 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:57 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:57.927+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:03:58.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:58 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:58.105+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:58.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:58 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:58.105+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:58.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:58 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:58.105+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:58.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:58 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:58.105+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:03:58.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:58 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:58.105+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:03:58.519 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:58 smithi103 podman[73408]: 2024-09-10 11:03:58.210643034 +0000 UTC m=+5.564585445 container died bbd596177d29d0a816238f50bf678fb7aecf76183f326c9835c1e0b0cbbbe90c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:03:58.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:58 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:58.373+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:58.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:58 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:58.373+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:58.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:58 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:58.373+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:58.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:58 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:58.373+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:32.624271+0000 front 2024-09-10T11:03:32.624204+0000 (oldest deadline 2024-09-10T11:03:55.424701+0000) 2024-09-10T11:03:58.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:58 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:58.334+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:58.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:58 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:58.334+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:58.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:58 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:58.334+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:58.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:58 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:58.334+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:03:58.836 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:58 smithi103 podman[73408]: 2024-09-10 11:03:58.609816377 +0000 UTC m=+5.963758786 container cleanup bbd596177d29d0a816238f50bf678fb7aecf76183f326c9835c1e0b0cbbbe90c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:03:58.837 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:58 smithi103 bash[73408]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7 2024-09-10T11:03:59.143 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:58 smithi103 podman[73426]: 2024-09-10 11:03:58.83606695 +0000 UTC m=+0.624509889 container remove bbd596177d29d0a816238f50bf678fb7aecf76183f326c9835c1e0b0cbbbe90c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T11:03:59.143 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:58 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:58.948+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:03:59.143 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:58 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:58.948+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:03:59.143 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:58 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:58.948+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:03:59.144 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:58 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:58.948+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:03:59.144 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:58 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:58.948+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:03:59.394 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:59 smithi103 podman[73515]: 2024-09-10 11:03:59.045202154 +0000 UTC m=+0.015911382 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:03:59.394 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:59 smithi103 podman[73515]: 2024-09-10 11:03:59.171208794 +0000 UTC m=+0.141918075 container create d50bd9ec5c80bf0fd177dcd67efdf8c7dbe20200676ece9f593169bd77ec57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T11:03:59.394 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:59 smithi103 podman[73515]: 2024-09-10 11:03:59.389741574 +0000 UTC m=+0.360450805 container init d50bd9ec5c80bf0fd177dcd67efdf8c7dbe20200676ece9f593169bd77ec57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:03:59.394 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:59 smithi103 podman[73515]: 2024-09-10 11:03:59.393226259 +0000 UTC m=+0.363935481 container start d50bd9ec5c80bf0fd177dcd67efdf8c7dbe20200676ece9f593169bd77ec57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T11:03:59.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:59 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:59.138+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:03:59.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:59 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:59.138+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:03:59.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:59 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:59.138+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:03:59.435 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:59 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:59.138+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:03:59.435 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:03:59 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:03:59.138+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:03:59.672 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:59 smithi103 podman[73515]: 2024-09-10 11:03:59.472956414 +0000 UTC m=+0.443665652 container attach d50bd9ec5c80bf0fd177dcd67efdf8c7dbe20200676ece9f593169bd77ec57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T11:03:59.672 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:03:59 smithi103 podman[73515]: 2024-09-10 11:03:59.584131086 +0000 UTC m=+0.554840322 container died d50bd9ec5c80bf0fd177dcd67efdf8c7dbe20200676ece9f593169bd77ec57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T11:03:59.728 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:59 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:59.369+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:03:59.729 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:59 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:59.369+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:03:59.729 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:59 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:59.369+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:03:59.729 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:03:59 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:03:59.369+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:32.624271+0000 front 2024-09-10T11:03:32.624204+0000 (oldest deadline 2024-09-10T11:03:55.424701+0000) 2024-09-10T11:03:59.729 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:59 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:59.313+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:03:59.730 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:59 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:59.313+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:03:59.730 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:59 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:59.313+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:03:59.730 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:03:59 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:03:59.313+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:00.179 INFO:journalctl@ceph.osd.7.smithi103.stdout:Sep 10 11:04:00 smithi103 podman[73538]: 2024-09-10 11:04:00.176232045 +0000 UTC m=+0.586473047 container remove d50bd9ec5c80bf0fd177dcd67efdf8c7dbe20200676ece9f593169bd77ec57ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T11:04:00.179 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:59 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:59.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:00.179 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:59 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:59.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:00.179 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:59 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:59.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:00.179 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:59 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:59.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:00.180 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:03:59 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:03:59.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:00.196 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T11:04:00.238 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:00.238 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T11:04:00.239 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T11:04:00.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.8 2024-09-10T11:04:00.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:00.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:00.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:00.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:00.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:00.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:00.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:00.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:00.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:00 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:00.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:00.545 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:00.263+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:00.545 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:00.263+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:00.545 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:00.263+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:00.546 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:00.263+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:00.546 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:00 smithi133 systemd[1]: Stopping Ceph osd.8 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:04:00.546 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:00.365+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:04:00.546 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:00.365+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:04:00.546 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:00.365+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:04:00.546 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:00.365+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:32.624271+0000 front 2024-09-10T11:03:32.624204+0000 (oldest deadline 2024-09-10T11:03:55.424701+0000) 2024-09-10T11:04:00.547 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:00.543+0000 7f00b7e7f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:04:00.547 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:00.543+0000 7f00b7e7f640 -1 osd.8 112 *** Got signal Terminated *** 2024-09-10T11:04:00.547 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:00 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:00.543+0000 7f00b7e7f640 -1 osd.8 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:04:01.402 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:00 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:00.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:01.402 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:00 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:00.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:01.402 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:00 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:00.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:01.403 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:00 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:00.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:01.403 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:00 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:00.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:01.433 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:01 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:01.176+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:01.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:01 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:01.176+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:01.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:01 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:01.176+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:01.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:01 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:01.176+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:01.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:01 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:01.176+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:01.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:01 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:01.401+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:04:01.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:01 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:01.401+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:04:01.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:01 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:01.401+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:04:01.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:01 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:01.401+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:32.624271+0000 front 2024-09-10T11:03:32.624204+0000 (oldest deadline 2024-09-10T11:03:55.424701+0000) 2024-09-10T11:04:01.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:01 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:01.246+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:01.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:01 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:01.246+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:01.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:01 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:01.246+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:01.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:01 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:01.246+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:02.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:01 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:01.958+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:02.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:01 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:01.958+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:02.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:01 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:01.958+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:02.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:01 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:01.958+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:02.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:01 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:01.958+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:02.475 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:02 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:02.383+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:04:02.476 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:02 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:02.383+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:04:02.476 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:02 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:02.383+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:04:02.476 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:02 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:02.383+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:32.624271+0000 front 2024-09-10T11:03:32.624204+0000 (oldest deadline 2024-09-10T11:03:55.424701+0000) 2024-09-10T11:04:02.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:02 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:02.208+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:02.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:02 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:02.208+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:02.477 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:02 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:02.208+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:02.477 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:02 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:02.208+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:02.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:02 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:02.193+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:02.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:02 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:02.193+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:02.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:02 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:02.193+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:02.685 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:02 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:02.193+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:02.685 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:02 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:02.193+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:03.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:02 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:02.979+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:03.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:02 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:02.979+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:03.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:02 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:02.979+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:03.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:02 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:02.979+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:03.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:02 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:02.979+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:03.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:03 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:03.207+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:03.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:03 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:03.207+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:03.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:03 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:03.207+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:03.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:03 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:03.207+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:03.685 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:03 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:03.207+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:03.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:03 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:03.424+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:04:03.725 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:03 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:03.424+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:04:03.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:03 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:03.424+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:04:03.726 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:03 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:03.424+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:32.624271+0000 front 2024-09-10T11:03:32.624204+0000 (oldest deadline 2024-09-10T11:03:55.424701+0000) 2024-09-10T11:04:03.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:03 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:03.249+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:03.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:03 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:03.249+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:03.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:03 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:03.249+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:03.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:03 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:03.249+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:03.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:03 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:03.249+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:04.425 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:03 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:03.952+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:04.425 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:03 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:03.952+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:04.425 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:03 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:03.952+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:04.426 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:03 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:03.952+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:04.426 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:03 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:03.952+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:04.426 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:03 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:03.952+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:04.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:04 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:04.158+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:04.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:04 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:04.158+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:04.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:04 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:04.158+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:04.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:04 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:04.158+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:04.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:04 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:04.158+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:04.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:04 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:04.158+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:40.055540+0000 front 2024-09-10T11:03:40.055520+0000 (oldest deadline 2024-09-10T11:04:04.155524+0000) 2024-09-10T11:04:04.629 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:04 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:04.416+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:04:04.630 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:04 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:04.416+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:04:04.630 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:04 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:04.416+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:04:04.630 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:04 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:04.416+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:32.624271+0000 front 2024-09-10T11:03:32.624204+0000 (oldest deadline 2024-09-10T11:03:55.424701+0000) 2024-09-10T11:04:04.630 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:04 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:04.242+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:04.630 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:04 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:04.242+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:04.631 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:04 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:04.242+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:04.631 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:04 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:04.242+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:04.631 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:04 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:04.242+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:05.323 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:04 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:04.942+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:05.324 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:04 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:04.942+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:05.324 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:04 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:04.942+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:05.324 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:04 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:04.942+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:05.324 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:04 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:04.942+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:05.324 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:04 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:04.942+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:05.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:05 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:05.181+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:05.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:05 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:05.181+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:05.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:05 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:05.181+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:05.435 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:05 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:05.181+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:05.435 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:05 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:05.181+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:05.435 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:05 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:05.181+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:40.055540+0000 front 2024-09-10T11:03:40.055520+0000 (oldest deadline 2024-09-10T11:04:04.155524+0000) 2024-09-10T11:04:05.642 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:05 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:05.450+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.121084+0000 front 2024-09-10T11:03:09.121190+0000 (oldest deadline 2024-09-10T11:03:32.621137+0000) 2024-09-10T11:04:05.643 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:05 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:05.450+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.522286+0000 front 2024-09-10T11:03:15.522309+0000 (oldest deadline 2024-09-10T11:03:41.422412+0000) 2024-09-10T11:04:05.643 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:05 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:05.450+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:21.422862+0000 front 2024-09-10T11:03:21.422931+0000 (oldest deadline 2024-09-10T11:03:47.322963+0000) 2024-09-10T11:04:05.643 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:05 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8[53884]: 2024-09-10T11:04:05.450+0000 7f00b047c640 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:32.624271+0000 front 2024-09-10T11:03:32.624204+0000 (oldest deadline 2024-09-10T11:03:55.424701+0000) 2024-09-10T11:04:05.643 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:05 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:05.260+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:05.643 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:05 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:05.260+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:05.643 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:05 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:05.260+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:05.644 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:05 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:05.260+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:05.644 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:05 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:05.261+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:06.225 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:06 smithi133 podman[74313]: 2024-09-10 11:04:06.034343148 +0000 UTC m=+5.602046669 container died eedbb2bab8d405c676074f38df251aa7647156c23e47cd6de65cde0b055d7139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T11:04:06.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:05 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:05.990+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:06.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:05 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:05.990+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:06.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:05 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:05.990+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:06.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:05 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:05.990+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:06.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:05 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:05.990+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:06.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:05 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:05.990+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:06.433 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:06 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:06.139+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:06.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:06 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:06.139+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:06.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:06 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:06.139+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:06.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:06 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:06.139+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:06.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:06 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:06.139+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:06.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:06 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:06.139+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:40.055540+0000 front 2024-09-10T11:03:40.055520+0000 (oldest deadline 2024-09-10T11:04:04.155524+0000) 2024-09-10T11:04:06.570 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:06 smithi133 podman[74313]: 2024-09-10 11:04:06.384967314 +0000 UTC m=+5.952670853 container cleanup eedbb2bab8d405c676074f38df251aa7647156c23e47cd6de65cde0b055d7139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T11:04:06.570 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:06 smithi133 bash[74313]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8 2024-09-10T11:04:06.570 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:06 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:06.243+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:06.571 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:06 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:06.243+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:06.571 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:06 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:06.243+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:06.571 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:06 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:06.243+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:06.571 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:06 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:06.243+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:06.963 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:06 smithi133 podman[74333]: 2024-09-10 11:04:06.569541187 +0000 UTC m=+0.529997633 container remove eedbb2bab8d405c676074f38df251aa7647156c23e47cd6de65cde0b055d7139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T11:04:07.225 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:07 smithi133 podman[74427]: 2024-09-10 11:04:06.932166243 +0000 UTC m=+0.015575377 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:04:07.225 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:07 smithi133 podman[74427]: 2024-09-10 11:04:07.053984745 +0000 UTC m=+0.137393905 container create 95fe7c9c65c4154ce3079c6402ee438d95543c0451a8573e24c085f9b87f06d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T11:04:07.225 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:07 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:07.199+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:07.226 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:07 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:07.199+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:07.226 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:07 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:07.199+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:07.226 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:07 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:07.199+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:07.226 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:07 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:07.199+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:07.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:06 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:06.968+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:07.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:06 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:06.968+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:07.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:06 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:06.968+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:07.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:06 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:06.968+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:07.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:06 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:06.968+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:07.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:06 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:06.968+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:07.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:07 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:07.101+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:07.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:07 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:07.101+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:07.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:07 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:07.101+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:07.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:07 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:07.101+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:07.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:07 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:07.101+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:07.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:07 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:07.101+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:40.055540+0000 front 2024-09-10T11:03:40.055520+0000 (oldest deadline 2024-09-10T11:04:04.155524+0000) 2024-09-10T11:04:07.497 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:07 smithi133 podman[74427]: 2024-09-10 11:04:07.297847271 +0000 UTC m=+0.381256388 container init 95fe7c9c65c4154ce3079c6402ee438d95543c0451a8573e24c085f9b87f06d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS) 2024-09-10T11:04:07.497 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:07 smithi133 podman[74427]: 2024-09-10 11:04:07.301274653 +0000 UTC m=+0.384683782 container start 95fe7c9c65c4154ce3079c6402ee438d95543c0451a8573e24c085f9b87f06d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:04:07.497 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:07 smithi133 podman[74427]: 2024-09-10 11:04:07.381088632 +0000 UTC m=+0.464497772 container attach 95fe7c9c65c4154ce3079c6402ee438d95543c0451a8573e24c085f9b87f06d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:04:08.043 INFO:journalctl@ceph.osd.8.smithi133.stdout:Sep 10 11:04:07 smithi133 podman[74427]: 2024-09-10 11:04:07.496288625 +0000 UTC m=+0.579697761 container died 95fe7c9c65c4154ce3079c6402ee438d95543c0451a8573e24c085f9b87f06d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T11:04:08.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T11:04:08.110 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:08.111 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T11:04:08.111 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T11:04:08.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.9 2024-09-10T11:04:08.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:07 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:07.941+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:08.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:07 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:07.941+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:08.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:07 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:07.941+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:08.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:07 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:07.941+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:08.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:07 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:07.941+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:08.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:07 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:07.941+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:08.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:08 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:08.116+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:08.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:08 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:08.116+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:08.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:08 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:08.116+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:08.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:08 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:08.116+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:08.435 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:08 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:08.116+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:08.435 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:08 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:08.116+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:40.055540+0000 front 2024-09-10T11:03:40.055520+0000 (oldest deadline 2024-09-10T11:04:04.155524+0000) 2024-09-10T11:04:08.435 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:08 smithi007 systemd[1]: Stopping Ceph osd.9 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:04:08.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:08 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:08.228+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:08.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:08 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:08.228+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:08.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:08 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:08.228+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:08.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:08 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:08.228+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:08.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:08 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:08.228+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:08.934 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:08 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:08.504+0000 7fb9e522f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:04:08.934 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:08 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:08.504+0000 7fb9e522f640 -1 osd.9 112 *** Got signal Terminated *** 2024-09-10T11:04:08.934 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:08 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:08.504+0000 7fb9e522f640 -1 osd.9 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:04:09.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:08 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:08.931+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:09.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:08 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:08.931+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:09.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:08 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:08.931+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:09.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:08 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:08.931+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:09.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:08 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:08.931+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:09.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:08 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:08.931+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:09.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:09 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:09.146+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:09.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:09 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:09.146+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:09.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:09 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:09.146+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:09.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:09 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:09.146+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:09.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:09 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:09.146+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:09.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:09 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:09.146+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:40.055540+0000 front 2024-09-10T11:03:40.055520+0000 (oldest deadline 2024-09-10T11:04:04.155524+0000) 2024-09-10T11:04:09.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:09 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:09.257+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:09.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:09 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:09.257+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:09.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:09 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:09.257+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:09.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:09 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:09.257+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:09.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:09 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:09.257+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:10.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:09 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:09.964+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:10.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:09 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:09.964+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:10.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:09 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:09.964+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:10.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:09 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:09.964+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:10.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:09 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:09.964+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:10.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:09 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:09.964+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:10.433 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:10 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:10.137+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:10.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:10 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:10.137+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:10.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:10 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:10.137+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:10.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:10 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:10.137+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:10.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:10 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:10.137+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:10.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:10 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:10.137+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:40.055540+0000 front 2024-09-10T11:03:40.055520+0000 (oldest deadline 2024-09-10T11:04:04.155524+0000) 2024-09-10T11:04:10.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:10 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:10.264+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:10.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:10 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:10.264+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:10.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:10 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:10.264+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:10.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:10 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:10.264+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:10.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:10 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:10.264+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:11.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:10 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:10.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:11.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:10 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:10.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:11.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:10 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:10.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:11.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:10 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:10.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:11.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:10 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:10.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:11.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:10 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:10.963+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:11.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:11 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:11.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:11.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:11 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:11.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:11.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:11 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:11.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:11.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:11 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:11.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:11.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:11 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:11.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:11.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:11 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:11.165+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:40.055540+0000 front 2024-09-10T11:03:40.055520+0000 (oldest deadline 2024-09-10T11:04:04.155524+0000) 2024-09-10T11:04:11.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:11 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:11.232+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:11.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:11 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:11.232+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:11.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:11 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:11.232+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:11.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:11 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:11.232+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:11.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:11 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:11.232+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:12.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:11 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:11.914+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:12.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:11 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:11.914+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:12.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:11 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:11.914+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:12.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:11 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:11.914+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:12.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:11 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:11.914+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:12.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:11 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:11.914+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:12.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:11 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:11.915+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:47.606510+0000 front 2024-09-10T11:03:47.606366+0000 (oldest deadline 2024-09-10T11:04:11.706225+0000) 2024-09-10T11:04:12.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:12 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:12.172+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:12.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:12 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:12.172+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:12.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:12 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:12.172+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:12.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:12 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:12.172+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:12.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:12 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:12.172+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:12.434 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:12 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:12.172+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:40.055540+0000 front 2024-09-10T11:03:40.055520+0000 (oldest deadline 2024-09-10T11:04:04.155524+0000) 2024-09-10T11:04:12.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:12 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:12.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:12.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:12 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:12.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:12.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:12 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:12.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:12.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:12 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:12.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:12.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:12 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:12.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:13.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:12 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:12.909+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:13.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:12 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:12.909+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:13.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:12 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:12.909+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:13.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:12 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:12.909+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:13.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:12 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:12.909+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:13.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:12 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:12.909+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:13.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:12 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:12.909+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:47.606510+0000 front 2024-09-10T11:03:47.606366+0000 (oldest deadline 2024-09-10T11:04:11.706225+0000) 2024-09-10T11:04:13.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:13 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:13.158+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:13.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:13 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:13.158+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:13.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:13 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:13.158+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:13.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:13 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:13.158+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:13.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:13 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:13.158+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:13.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:13 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:13.158+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:13.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:13 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:13.217+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:59.749031+0000 front 2024-09-10T11:02:59.749109+0000 (oldest deadline 2024-09-10T11:03:25.549722+0000) 2024-09-10T11:04:13.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:13 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:13.217+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:09.551342+0000 front 2024-09-10T11:03:09.551380+0000 (oldest deadline 2024-09-10T11:03:35.451559+0000) 2024-09-10T11:04:13.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:13 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:13.217+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:15.452321+0000 front 2024-09-10T11:03:15.452221+0000 (oldest deadline 2024-09-10T11:03:40.152317+0000) 2024-09-10T11:04:13.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:13 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:13.217+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.453393+0000 front 2024-09-10T11:03:25.453476+0000 (oldest deadline 2024-09-10T11:03:51.353423+0000) 2024-09-10T11:04:13.685 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:13 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:13.217+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:31.353921+0000 front 2024-09-10T11:03:31.353956+0000 (oldest deadline 2024-09-10T11:03:55.954500+0000) 2024-09-10T11:04:13.685 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:13 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9[65660]: 2024-09-10T11:04:13.217+0000 7fb9dd82c640 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:40.055540+0000 front 2024-09-10T11:03:40.055520+0000 (oldest deadline 2024-09-10T11:04:04.155524+0000) 2024-09-10T11:04:14.380 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:14 smithi007 podman[93772]: 2024-09-10 11:04:14.105044568 +0000 UTC m=+5.782101788 container died 59e6e1583b4de8e1b457ec7fb687d250049262ed5ae7b79d1313d4f9bb7dc08c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T11:04:14.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:13 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:13.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:14.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:13 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:13.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:14.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:13 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:13.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:14.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:13 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:13.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:14.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:13 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:13.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:14.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:13 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:13.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:14.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:13 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:13.926+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:47.606510+0000 front 2024-09-10T11:03:47.606366+0000 (oldest deadline 2024-09-10T11:04:11.706225+0000) 2024-09-10T11:04:14.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:14 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:14.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:14.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:14 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:14.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:14.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:14 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:14.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:14.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:14 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:14.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:14.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:14 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:14.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:14.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:14 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:14.203+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:14.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:14 smithi007 podman[93772]: 2024-09-10 11:04:14.461783224 +0000 UTC m=+6.138840445 container cleanup 59e6e1583b4de8e1b457ec7fb687d250049262ed5ae7b79d1313d4f9bb7dc08c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T11:04:14.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:14 smithi007 bash[93772]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9 2024-09-10T11:04:15.102 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:14 smithi007 podman[93789]: 2024-09-10 11:04:14.729753198 +0000 UTC m=+0.620349610 container remove 59e6e1583b4de8e1b457ec7fb687d250049262ed5ae7b79d1313d4f9bb7dc08c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T11:04:15.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:14 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:14.917+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:15.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:14 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:14.917+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:15.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:14 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:14.917+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:15.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:14 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:14.917+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:15.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:14 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:14.917+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:15.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:14 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:14.917+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:15.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:14 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:14.917+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:47.606510+0000 front 2024-09-10T11:03:47.606366+0000 (oldest deadline 2024-09-10T11:04:11.706225+0000) 2024-09-10T11:04:15.408 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:15 smithi007 podman[93879]: 2024-09-10 11:04:15.004584534 +0000 UTC m=+0.016637670 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:04:15.409 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:15 smithi007 podman[93879]: 2024-09-10 11:04:15.10542974 +0000 UTC m=+0.117482862 container create 9fc683d3723bd5aea9fe17d8cb663ec0a8d43684e2b16ac088264fa97fff0f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_CLEAN=True) 2024-09-10T11:04:15.409 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:15 smithi007 podman[93879]: 2024-09-10 11:04:15.324741832 +0000 UTC m=+0.336794957 container init 9fc683d3723bd5aea9fe17d8cb663ec0a8d43684e2b16ac088264fa97fff0f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T11:04:15.409 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:15 smithi007 podman[93879]: 2024-09-10 11:04:15.328058692 +0000 UTC m=+0.340111821 container start 9fc683d3723bd5aea9fe17d8cb663ec0a8d43684e2b16ac088264fa97fff0f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T11:04:15.409 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:15 smithi007 podman[93879]: 2024-09-10 11:04:15.408119518 +0000 UTC m=+0.420172639 container attach 9fc683d3723bd5aea9fe17d8cb663ec0a8d43684e2b16ac088264fa97fff0f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:04:15.684 INFO:journalctl@ceph.osd.9.smithi007.stdout:Sep 10 11:04:15 smithi007 podman[93879]: 2024-09-10 11:04:15.51948517 +0000 UTC m=+0.531538296 container died 9fc683d3723bd5aea9fe17d8cb663ec0a8d43684e2b16ac088264fa97fff0f54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T11:04:15.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:15 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:15.233+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:15.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:15 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:15.233+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:15.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:15 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:15.233+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:15.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:15 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:15.233+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:15.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:15 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:15.233+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:15.728 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:15 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:15.233+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:16.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T11:04:16.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:16.191 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T11:04:16.191 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T11:04:16.191 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.10 2024-09-10T11:04:16.220 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:15 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:15.928+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:16.220 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:15 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:15.928+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:16.220 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:15 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:15.928+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:16.221 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:15 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:15.928+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:16.221 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:15 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:15.928+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:16.221 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:15 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:15.928+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:16.221 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:15 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:15.928+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:47.606510+0000 front 2024-09-10T11:03:47.606366+0000 (oldest deadline 2024-09-10T11:04:11.706225+0000) 2024-09-10T11:04:16.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:16 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:16.215+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:16.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:16 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:16.215+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:16.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:16 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:16.215+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:16.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:16 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:16.215+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:16.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:16 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:16.215+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:16.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:16 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:16.215+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:16.496 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 systemd[1]: Stopping Ceph osd.10 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:04:16.922 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:16.495+0000 7f7b64533640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:04:16.922 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:16.495+0000 7f7b64533640 -1 osd.10 112 *** Got signal Terminated *** 2024-09-10T11:04:16.922 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:16.495+0000 7f7b64533640 -1 osd.10 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:04:17.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:16.971+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:17.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:16.971+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:17.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:16.971+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:17.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:16.971+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:17.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:16.971+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:17.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:16.971+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:17.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:16 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:16.971+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:47.606510+0000 front 2024-09-10T11:03:47.606366+0000 (oldest deadline 2024-09-10T11:04:11.706225+0000) 2024-09-10T11:04:17.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:17.248+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:17.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:17.248+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:17.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:17.248+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:17.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:17.248+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:17.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:17.248+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:17.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:17.248+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:18.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:17 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:17.954+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:18.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:17 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:17.954+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:18.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:17 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:17.954+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:18.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:17 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:17.954+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:18.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:17 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:17.954+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:18.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:17 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:17.954+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:18.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:17 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:17.954+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:47.606510+0000 front 2024-09-10T11:03:47.606366+0000 (oldest deadline 2024-09-10T11:04:11.706225+0000) 2024-09-10T11:04:18.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:18 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:18.281+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:18.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:18 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:18.281+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:18.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:18 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:18.281+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:18.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:18 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:18.281+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:18.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:18 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:18.281+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:18.476 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:18 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:18.281+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:19.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:18 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:18.983+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:19.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:18 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:18.983+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:19.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:18 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:18.983+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:19.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:18 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:18.983+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:19.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:18 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:18.983+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:19.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:18 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:18.983+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:19.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:18 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:18.983+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:47.606510+0000 front 2024-09-10T11:03:47.606366+0000 (oldest deadline 2024-09-10T11:04:11.706225+0000) 2024-09-10T11:04:19.728 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:19 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:19.321+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:19.728 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:19 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:19.321+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:19.728 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:19 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:19.321+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:19.728 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:19 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:19.321+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:19.728 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:19 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:19.321+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:19.729 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:19 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:19.321+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:20.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:20.027+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:20.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:20.027+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:20.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:20.027+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:20.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:20.027+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:20.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:20.027+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:20.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:20.027+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:20.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:20 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:20.027+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:47.606510+0000 front 2024-09-10T11:03:47.606366+0000 (oldest deadline 2024-09-10T11:04:11.706225+0000) 2024-09-10T11:04:20.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:20.316+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:20.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:20.316+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:20.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:20.316+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:20.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:20.316+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:20.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:20.316+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:20.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:20.316+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:20.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:20.316+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-10T11:03:57.021998+0000 front 2024-09-10T11:03:57.021934+0000 (oldest deadline 2024-09-10T11:04:19.321790+0000) 2024-09-10T11:04:21.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:21 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:21.041+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:02:57.498255+0000 front 2024-09-10T11:02:57.498231+0000 (oldest deadline 2024-09-10T11:03:23.398382+0000) 2024-09-10T11:04:21.422 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:21 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:21.041+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.899407+0000 front 2024-09-10T11:03:06.899443+0000 (oldest deadline 2024-09-10T11:03:31.599600+0000) 2024-09-10T11:04:21.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:21 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:21.041+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:17.500795+0000 front 2024-09-10T11:03:17.500730+0000 (oldest deadline 2024-09-10T11:03:40.400805+0000) 2024-09-10T11:04:21.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:21 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:21.041+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-09-10T11:03:25.702075+0000 front 2024-09-10T11:03:25.702002+0000 (oldest deadline 2024-09-10T11:03:50.401900+0000) 2024-09-10T11:04:21.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:21 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:21.041+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:33.803173+0000 front 2024-09-10T11:03:33.803199+0000 (oldest deadline 2024-09-10T11:03:57.903383+0000) 2024-09-10T11:04:21.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:21 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:21.041+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.404577+0000 front 2024-09-10T11:03:41.404606+0000 (oldest deadline 2024-09-10T11:04:03.104446+0000) 2024-09-10T11:04:21.423 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:21 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10[60241]: 2024-09-10T11:04:21.041+0000 7f7b5cb30640 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:47.606510+0000 front 2024-09-10T11:03:47.606366+0000 (oldest deadline 2024-09-10T11:04:11.706225+0000) 2024-09-10T11:04:21.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:21.323+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:21.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:21.323+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:21.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:21.323+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:21.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:21.323+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:21.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:21.323+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:21.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:21.323+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:21.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:21 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:21.323+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-10T11:03:57.021998+0000 front 2024-09-10T11:03:57.021934+0000 (oldest deadline 2024-09-10T11:04:19.321790+0000) 2024-09-10T11:04:22.247 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:21 smithi103 podman[73630]: 2024-09-10 11:04:21.971384232 +0000 UTC m=+5.579591732 container died 5e60b3da221de34ffa25af045ae32c98cb6ab6099263a05620f0855f788d05d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T11:04:22.589 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:22 smithi103 podman[73630]: 2024-09-10 11:04:22.337237147 +0000 UTC m=+5.945444637 container cleanup 5e60b3da221de34ffa25af045ae32c98cb6ab6099263a05620f0855f788d05d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:04:22.589 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:22 smithi103 bash[73630]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10 2024-09-10T11:04:22.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:22 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:22.290+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:22.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:22 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:22.290+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:22.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:22 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:22.290+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:22.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:22 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:22.290+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:22.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:22 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:22.290+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:22.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:22 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:22.290+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:22.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:22 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:22.290+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-10T11:03:57.021998+0000 front 2024-09-10T11:03:57.021934+0000 (oldest deadline 2024-09-10T11:04:19.321790+0000) 2024-09-10T11:04:22.905 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:22 smithi103 podman[73646]: 2024-09-10 11:04:22.588171919 +0000 UTC m=+0.612454894 container remove 5e60b3da221de34ffa25af045ae32c98cb6ab6099263a05620f0855f788d05d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:04:23.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:22 smithi103 podman[73732]: 2024-09-10 11:04:22.807032531 +0000 UTC m=+0.016105585 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:04:23.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:22 smithi103 podman[73732]: 2024-09-10 11:04:22.914017939 +0000 UTC m=+0.123091030 container create af8751e82f6f4074deb967d782d0883af4fc92cdd60d8cb98ba1c3bebc51a25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T11:04:23.172 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:23 smithi103 podman[73732]: 2024-09-10 11:04:23.124835777 +0000 UTC m=+0.333908826 container init af8751e82f6f4074deb967d782d0883af4fc92cdd60d8cb98ba1c3bebc51a25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T11:04:23.173 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:23 smithi103 podman[73732]: 2024-09-10 11:04:23.128080968 +0000 UTC m=+0.337154020 container start af8751e82f6f4074deb967d782d0883af4fc92cdd60d8cb98ba1c3bebc51a25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:04:23.669 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:23 smithi103 podman[73732]: 2024-09-10 11:04:23.194450622 +0000 UTC m=+0.403523690 container attach af8751e82f6f4074deb967d782d0883af4fc92cdd60d8cb98ba1c3bebc51a25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T11:04:23.669 INFO:journalctl@ceph.osd.10.smithi103.stdout:Sep 10 11:04:23 smithi103 podman[73732]: 2024-09-10 11:04:23.316683484 +0000 UTC m=+0.525756537 container died af8751e82f6f4074deb967d782d0883af4fc92cdd60d8cb98ba1c3bebc51a25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T11:04:23.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:23.337+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:23.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:23.337+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:23.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:23.337+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:23.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:23.337+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:23.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:23.337+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:23.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:23.337+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:23.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:23 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:23.337+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-10T11:03:57.021998+0000 front 2024-09-10T11:03:57.021934+0000 (oldest deadline 2024-09-10T11:04:19.321790+0000) 2024-09-10T11:04:24.016 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T11:04:24.095 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:24.095 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T11:04:24.095 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T11:04:24.096 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.11 2024-09-10T11:04:24.374 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 systemd[1]: Stopping Ceph osd.11 for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:04:24.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:24.373+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:24.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:24.373+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:24.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:24.373+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:24.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:24.373+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:24.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:24.373+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:24.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:24.373+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:24.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:24.373+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-10T11:03:57.021998+0000 front 2024-09-10T11:03:57.021934+0000 (oldest deadline 2024-09-10T11:04:19.321790+0000) 2024-09-10T11:04:24.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:24.394+0000 7fc923b81640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:04:24.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:24.394+0000 7fc923b81640 -1 osd.11 112 *** Got signal Terminated *** 2024-09-10T11:04:24.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:24 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:24.394+0000 7fc923b81640 -1 osd.11 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:04:25.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:25 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:25.387+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:25.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:25 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:25.387+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:25.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:25 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:25.387+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:25.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:25 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:25.387+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:25.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:25 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:25.387+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:25.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:25 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:25.387+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:25.727 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:25 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:25.387+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-10T11:03:57.021998+0000 front 2024-09-10T11:03:57.021934+0000 (oldest deadline 2024-09-10T11:04:19.321790+0000) 2024-09-10T11:04:26.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:26 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:26.419+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:26.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:26 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:26.419+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:26.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:26 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:26.419+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:26.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:26 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:26.419+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:26.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:26 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:26.419+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:26.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:26 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:26.419+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:26.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:26 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:26.419+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-10T11:03:57.021998+0000 front 2024-09-10T11:03:57.021934+0000 (oldest deadline 2024-09-10T11:04:19.321790+0000) 2024-09-10T11:04:27.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:27 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:27.384+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:27.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:27 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:27.384+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:27.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:27 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:27.384+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:27.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:27 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:27.384+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:27.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:27 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:27.384+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:27.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:27 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:27.384+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:27.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:27 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:27.384+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-10T11:03:57.021998+0000 front 2024-09-10T11:03:57.021934+0000 (oldest deadline 2024-09-10T11:04:19.321790+0000) 2024-09-10T11:04:28.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:28 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:28.363+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:28.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:28 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:28.363+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:28.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:28 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:28.363+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:28.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:28 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:28.363+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:28.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:28 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:28.363+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:28.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:28 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:28.363+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:28.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:28 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:28.363+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-10T11:03:57.021998+0000 front 2024-09-10T11:03:57.021934+0000 (oldest deadline 2024-09-10T11:04:19.321790+0000) 2024-09-10T11:04:29.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:29 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:29.342+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-09-10T11:03:00.710737+0000 front 2024-09-10T11:03:00.710716+0000 (oldest deadline 2024-09-10T11:03:24.210840+0000) 2024-09-10T11:04:29.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:29 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:29.342+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-09-10T11:03:06.511864+0000 front 2024-09-10T11:03:06.511797+0000 (oldest deadline 2024-09-10T11:03:31.211989+0000) 2024-09-10T11:04:29.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:29 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:29.342+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-09-10T11:03:18.114266+0000 front 2024-09-10T11:03:18.114244+0000 (oldest deadline 2024-09-10T11:03:42.714998+0000) 2024-09-10T11:04:29.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:29 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:29.342+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-09-10T11:03:29.716908+0000 front 2024-09-10T11:03:29.716999+0000 (oldest deadline 2024-09-10T11:03:57.917264+0000) 2024-09-10T11:04:29.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:29 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:29.342+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-09-10T11:03:41.318642+0000 front 2024-09-10T11:03:41.318663+0000 (oldest deadline 2024-09-10T11:04:02.418809+0000) 2024-09-10T11:04:29.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:29 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:29.342+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-09-10T11:03:48.220925+0000 front 2024-09-10T11:03:48.221056+0000 (oldest deadline 2024-09-10T11:04:12.920988+0000) 2024-09-10T11:04:29.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:29 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:29.342+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-09-10T11:03:57.021998+0000 front 2024-09-10T11:03:57.021934+0000 (oldest deadline 2024-09-10T11:04:19.321790+0000) 2024-09-10T11:04:29.726 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:29 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11[60627]: 2024-09-10T11:04:29.342+0000 7fc91c17e640 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6820 osd.8 since back 2024-09-10T11:04:05.524276+0000 front 2024-09-10T11:04:05.524302+0000 (oldest deadline 2024-09-10T11:04:29.024295+0000) 2024-09-10T11:04:30.225 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:29 smithi133 podman[74532]: 2024-09-10 11:04:29.94894961 +0000 UTC m=+5.656818601 container died 4cc791ae6f00cc4f9a45eb01b4455a2a588f75ff8f99033a5b7cbe72fe3f2b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T11:04:30.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:30 smithi133 podman[74532]: 2024-09-10 11:04:30.260730461 +0000 UTC m=+5.968599448 container cleanup 4cc791ae6f00cc4f9a45eb01b4455a2a588f75ff8f99033a5b7cbe72fe3f2b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T11:04:30.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:30 smithi133 bash[74532]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11 2024-09-10T11:04:30.725 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:30 smithi133 podman[74550]: 2024-09-10 11:04:30.470444487 +0000 UTC m=+0.520461790 container remove 4cc791ae6f00cc4f9a45eb01b4455a2a588f75ff8f99033a5b7cbe72fe3f2b99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:04:31.174 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:30 smithi133 podman[74637]: 2024-09-10 11:04:30.77952854 +0000 UTC m=+0.016748835 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:04:31.174 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:30 smithi133 podman[74637]: 2024-09-10 11:04:30.879493616 +0000 UTC m=+0.116713956 container create caa699ef9585bb145693a34894cc444322f1983a05a36fcf7f2d897465bde96c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:04:31.174 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:31 smithi133 podman[74637]: 2024-09-10 11:04:31.106673224 +0000 UTC m=+0.343893519 container init caa699ef9585bb145693a34894cc444322f1983a05a36fcf7f2d897465bde96c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T11:04:31.175 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:31 smithi133 podman[74637]: 2024-09-10 11:04:31.10985839 +0000 UTC m=+0.347078686 container start caa699ef9585bb145693a34894cc444322f1983a05a36fcf7f2d897465bde96c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T11:04:31.175 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:31 smithi133 podman[74637]: 2024-09-10 11:04:31.173274679 +0000 UTC m=+0.410495004 container attach caa699ef9585bb145693a34894cc444322f1983a05a36fcf7f2d897465bde96c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T11:04:31.475 INFO:journalctl@ceph.osd.11.smithi133.stdout:Sep 10 11:04:31 smithi133 podman[74637]: 2024-09-10 11:04:31.306316737 +0000 UTC m=+0.543537035 container died caa699ef9585bb145693a34894cc444322f1983a05a36fcf7f2d897465bde96c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T11:04:31.867 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T11:04:31.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:31.906 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T11:04:31.906 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T11:04:31.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.a 2024-09-10T11:04:32.214 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 11:04:31 smithi007 systemd[1]: Stopping Ceph mds.a for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:04:32.684 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 11:04:32 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-a[72531]: 2024-09-10T11:04:32.211+0000 7f48d85bf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:04:32.684 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 11:04:32 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-a[72531]: 2024-09-10T11:04:32.211+0000 7f48d85bf640 -1 mds.a *** got signal Terminated *** 2024-09-10T11:04:34.915 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 11:04:34 smithi007 podman[93979]: 2024-09-10 11:04:34.644490617 +0000 UTC m=+2.539376738 container died 064f0afb982e97ca509425b94ccfb0aa634862fd8d2ade74cfae80f440681f29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T11:04:35.184 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 11:04:35 smithi007 podman[93979]: 2024-09-10 11:04:35.018986989 +0000 UTC m=+2.913873108 container cleanup 064f0afb982e97ca509425b94ccfb0aa634862fd8d2ade74cfae80f440681f29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T11:04:35.184 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 11:04:35 smithi007 bash[93979]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-a 2024-09-10T11:04:35.444 INFO:journalctl@ceph.mds.a.smithi007.stdout:Sep 10 11:04:35 smithi007 podman[93993]: 2024-09-10 11:04:35.290905224 +0000 UTC m=+0.636066977 container remove 064f0afb982e97ca509425b94ccfb0aa634862fd8d2ade74cfae80f440681f29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T11:04:35.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T11:04:35.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:35.497 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T11:04:35.497 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T11:04:35.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.d 2024-09-10T11:04:35.819 INFO:journalctl@ceph.mds.d.smithi007.stdout:Sep 10 11:04:35 smithi007 systemd[1]: Stopping Ceph mds.d for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:04:36.183 INFO:journalctl@ceph.mds.d.smithi007.stdout:Sep 10 11:04:35 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-d[71409]: 2024-09-10T11:04:35.817+0000 7f0375cab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:04:36.184 INFO:journalctl@ceph.mds.d.smithi007.stdout:Sep 10 11:04:35 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-d[71409]: 2024-09-10T11:04:35.817+0000 7f0375cab640 -1 mds.d *** got signal Terminated *** 2024-09-10T11:04:40.111 INFO:journalctl@ceph.mds.d.smithi007.stdout:Sep 10 11:04:39 smithi007 podman[94099]: 2024-09-10 11:04:39.833241356 +0000 UTC m=+4.112699832 container died 4f30e13f3f4e0ef8c6efcd63a6cf84adbd0d1a961e824f8cfe8c6eb9a04ea9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:04:40.433 INFO:journalctl@ceph.mds.d.smithi007.stdout:Sep 10 11:04:40 smithi007 podman[94099]: 2024-09-10 11:04:40.193406464 +0000 UTC m=+4.472864939 container cleanup 4f30e13f3f4e0ef8c6efcd63a6cf84adbd0d1a961e824f8cfe8c6eb9a04ea9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T11:04:40.434 INFO:journalctl@ceph.mds.d.smithi007.stdout:Sep 10 11:04:40 smithi007 bash[94099]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-d 2024-09-10T11:04:40.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T11:04:40.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:40.664 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T11:04:40.664 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T11:04:40.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.g 2024-09-10T11:04:40.998 INFO:journalctl@ceph.mds.g.smithi007.stdout:Sep 10 11:04:40 smithi007 systemd[1]: Stopping Ceph mds.g for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:04:41.434 INFO:journalctl@ceph.mds.g.smithi007.stdout:Sep 10 11:04:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-g[72178]: 2024-09-10T11:04:40.996+0000 7fc37ad3d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:04:41.434 INFO:journalctl@ceph.mds.g.smithi007.stdout:Sep 10 11:04:40 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-g[72178]: 2024-09-10T11:04:40.996+0000 7fc37ad3d640 -1 mds.g *** got signal Terminated *** 2024-09-10T11:04:44.434 INFO:journalctl@ceph.mds.g.smithi007.stdout:Sep 10 11:04:44 smithi007 podman[94229]: 2024-09-10 11:04:44.168632034 +0000 UTC m=+3.277078331 container died 28b0843720eed0c20b92e5d6fbabe80f1892ff9e352271a0405931cd3ca7f1d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T11:04:44.798 INFO:journalctl@ceph.mds.g.smithi007.stdout:Sep 10 11:04:44 smithi007 podman[94229]: 2024-09-10 11:04:44.53047291 +0000 UTC m=+3.638919196 container cleanup 28b0843720eed0c20b92e5d6fbabe80f1892ff9e352271a0405931cd3ca7f1d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:04:44.799 INFO:journalctl@ceph.mds.g.smithi007.stdout:Sep 10 11:04:44 smithi007 bash[94229]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-g 2024-09-10T11:04:44.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T11:04:45.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:45.010 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T11:04:45.010 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T11:04:45.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.j 2024-09-10T11:04:45.352 INFO:journalctl@ceph.mds.j.smithi007.stdout:Sep 10 11:04:45 smithi007 systemd[1]: Stopping Ceph mds.j for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:04:45.684 INFO:journalctl@ceph.mds.j.smithi007.stdout:Sep 10 11:04:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-j[70999]: 2024-09-10T11:04:45.351+0000 7f853c5ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:04:45.684 INFO:journalctl@ceph.mds.j.smithi007.stdout:Sep 10 11:04:45 smithi007 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-j[70999]: 2024-09-10T11:04:45.351+0000 7f853c5ef640 -1 mds.j *** got signal Terminated *** 2024-09-10T11:04:49.619 INFO:journalctl@ceph.mds.j.smithi007.stdout:Sep 10 11:04:49 smithi007 podman[94351]: 2024-09-10 11:04:49.358005363 +0000 UTC m=+4.111065600 container died 5f3188481593127170025eed3e235830114db0e0048f6ecdd98fa87e7c02b049 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T11:04:49.934 INFO:journalctl@ceph.mds.j.smithi007.stdout:Sep 10 11:04:49 smithi007 podman[94351]: 2024-09-10 11:04:49.675584481 +0000 UTC m=+4.428644715 container cleanup 5f3188481593127170025eed3e235830114db0e0048f6ecdd98fa87e7c02b049 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T11:04:49.934 INFO:journalctl@ceph.mds.j.smithi007.stdout:Sep 10 11:04:49 smithi007 bash[94351]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-j 2024-09-10T11:04:50.328 INFO:journalctl@ceph.mds.j.smithi007.stdout:Sep 10 11:04:50 smithi007 podman[94362]: 2024-09-10 11:04:50.043705878 +0000 UTC m=+0.677625365 container remove 5f3188481593127170025eed3e235830114db0e0048f6ecdd98fa87e7c02b049 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T11:04:50.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T11:04:50.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:50.378 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T11:04:50.378 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T11:04:50.379 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.b 2024-09-10T11:04:50.672 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 11:04:50 smithi103 systemd[1]: Stopping Ceph mds.b for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:04:51.172 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 11:04:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b[65911]: 2024-09-10T11:04:50.803+0000 7fe0b3dca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:04:51.172 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 11:04:50 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b[65911]: 2024-09-10T11:04:50.803+0000 7fe0b3dca640 -1 mds.b *** got signal Terminated *** 2024-09-10T11:04:55.139 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 11:04:54 smithi103 podman[73838]: 2024-09-10 11:04:54.810790063 +0000 UTC m=+4.228072817 container died 0aeddf290aee43d07cf712323d32fcd69a4515346ce38006ab163ac7419bf798 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:04:55.422 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 11:04:55 smithi103 podman[73838]: 2024-09-10 11:04:55.203753552 +0000 UTC m=+4.621036306 container cleanup 0aeddf290aee43d07cf712323d32fcd69a4515346ce38006ab163ac7419bf798 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T11:04:55.422 INFO:journalctl@ceph.mds.b.smithi103.stdout:Sep 10 11:04:55 smithi103 bash[73838]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-b 2024-09-10T11:04:55.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T11:04:55.622 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:55.623 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T11:04:55.623 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T11:04:55.623 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.e 2024-09-10T11:04:56.172 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:04:55 smithi103 systemd[1]: Stopping Ceph mds.e for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:04:56.172 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:04:55 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e[65303]: 2024-09-10T11:04:55.942+0000 7f9b065b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:04:56.173 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:04:55 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e[65303]: 2024-09-10T11:04:55.942+0000 7f9b065b3640 -1 mds.e *** got signal Terminated *** 2024-09-10T11:04:59.242 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:04:58 smithi103 podman[73973]: 2024-09-10 11:04:58.966403352 +0000 UTC m=+3.129293648 container died 26f870833e690ef1b84f25ba7ffdde23952ee8e878ac4047c14eb63ca9953bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T11:04:59.609 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:04:59 smithi103 podman[73973]: 2024-09-10 11:04:59.341000647 +0000 UTC m=+3.503890942 container cleanup 26f870833e690ef1b84f25ba7ffdde23952ee8e878ac4047c14eb63ca9953bc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T11:04:59.609 INFO:journalctl@ceph.mds.e.smithi103.stdout:Sep 10 11:04:59 smithi103 bash[73973]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-e 2024-09-10T11:04:59.766 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T11:04:59.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:04:59.800 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T11:04:59.800 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T11:04:59.800 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.h 2024-09-10T11:05:00.129 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 11:04:59 smithi103 systemd[1]: Stopping Ceph mds.h for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:05:00.422 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 11:05:00 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h[65605]: 2024-09-10T11:05:00.127+0000 7f79438b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:05:00.422 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 11:05:00 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h[65605]: 2024-09-10T11:05:00.127+0000 7f79438b5640 -1 mds.h *** got signal Terminated *** 2024-09-10T11:05:03.355 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 11:05:02 smithi103 podman[74091]: 2024-09-10 11:05:02.970335395 +0000 UTC m=+2.948402164 container died 95c1fb125d95c1a226be3cc6eb77133a64fda04ca881833d3f93eb8827ee792d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:05:03.672 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 11:05:03 smithi103 podman[74091]: 2024-09-10 11:05:03.435913376 +0000 UTC m=+3.413980130 container cleanup 95c1fb125d95c1a226be3cc6eb77133a64fda04ca881833d3f93eb8827ee792d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T11:05:03.672 INFO:journalctl@ceph.mds.h.smithi103.stdout:Sep 10 11:05:03 smithi103 bash[74091]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-h 2024-09-10T11:05:03.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T11:05:03.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:05:03.898 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T11:05:03.898 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T11:05:03.898 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.k 2024-09-10T11:05:04.307 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 11:05:03 smithi103 systemd[1]: Stopping Ceph mds.k for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:05:04.672 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 11:05:04 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-k[64992]: 2024-09-10T11:05:04.306+0000 7f3c9cc17640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:05:04.672 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 11:05:04 smithi103 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-k[64992]: 2024-09-10T11:05:04.306+0000 7f3c9cc17640 -1 mds.k *** got signal Terminated *** 2024-09-10T11:05:07.033 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 11:05:06 smithi103 podman[74211]: 2024-09-10 11:05:06.715344062 +0000 UTC m=+2.599638834 container died 17da4644861975fe7e3146e9df7080c77f4e4446c72dd8c6732d9125d2ff7f43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T11:05:07.341 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 11:05:07 smithi103 podman[74211]: 2024-09-10 11:05:07.123355217 +0000 UTC m=+3.007649991 container cleanup 17da4644861975fe7e3146e9df7080c77f4e4446c72dd8c6732d9125d2ff7f43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-k, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:05:07.342 INFO:journalctl@ceph.mds.k.smithi103.stdout:Sep 10 11:05:07 smithi103 bash[74211]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-k 2024-09-10T11:05:07.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T11:05:07.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:05:07.579 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T11:05:07.579 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T11:05:07.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.c 2024-09-10T11:05:07.943 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 11:05:07 smithi133 systemd[1]: Stopping Ceph mds.c for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:05:08.195 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 11:05:07 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c[65774]: 2024-09-10T11:05:07.941+0000 7f9c37a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:05:08.195 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 11:05:07 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c[65774]: 2024-09-10T11:05:07.941+0000 7f9c37a00640 -1 mds.c *** got signal Terminated *** 2024-09-10T11:05:11.369 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 11:05:10 smithi133 podman[74737]: 2024-09-10 11:05:10.976250166 +0000 UTC m=+3.200892391 container died 87822b8b05ce593d8c9ec0a04115bf7cc56750e2f1fc8e95d4a323791051c2f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:05:11.709 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 11:05:11 smithi133 podman[74737]: 2024-09-10 11:05:11.449204637 +0000 UTC m=+3.673846851 container cleanup 87822b8b05ce593d8c9ec0a04115bf7cc56750e2f1fc8e95d4a323791051c2f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T11:05:11.709 INFO:journalctl@ceph.mds.c.smithi133.stdout:Sep 10 11:05:11 smithi133 bash[74737]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-c 2024-09-10T11:05:11.841 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T11:05:11.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:05:11.877 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T11:05:11.877 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T11:05:11.877 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.f 2024-09-10T11:05:12.225 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 11:05:11 smithi133 systemd[1]: Stopping Ceph mds.f for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:05:12.226 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 11:05:12 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: 2024-09-10T11:05:12.178+0000 7fa9686f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:05:12.226 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 11:05:12 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f[65468]: 2024-09-10T11:05:12.178+0000 7fa9686f6640 -1 mds.f *** got signal Terminated *** 2024-09-10T11:05:16.475 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 11:05:16 smithi133 podman[74871]: 2024-09-10 11:05:16.185260003 +0000 UTC m=+4.087121402 container died 78ce81a2e7d0250238fd69ec042aab2d4aa46259fa6fd1a80815fa82e5682af4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T11:05:16.821 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 11:05:16 smithi133 podman[74871]: 2024-09-10 11:05:16.578069534 +0000 UTC m=+4.479930933 container cleanup 78ce81a2e7d0250238fd69ec042aab2d4aa46259fa6fd1a80815fa82e5682af4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T11:05:16.822 INFO:journalctl@ceph.mds.f.smithi133.stdout:Sep 10 11:05:16 smithi133 bash[74871]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-f 2024-09-10T11:05:16.969 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T11:05:17.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:05:17.003 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T11:05:17.003 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T11:05:17.003 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.i 2024-09-10T11:05:17.475 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 11:05:17 smithi133 systemd[1]: Stopping Ceph mds.i for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:05:17.475 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 11:05:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-i[66074]: 2024-09-10T11:05:17.323+0000 7fad19688640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:05:17.476 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 11:05:17 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-i[66074]: 2024-09-10T11:05:17.323+0000 7fad19688640 -1 mds.i *** got signal Terminated *** 2024-09-10T11:05:19.266 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 11:05:18 smithi133 podman[74995]: 2024-09-10 11:05:18.982472234 +0000 UTC m=+1.761373732 container died 354565dfd37317baf857afc12e4acefc7a880984699f095de4651e5a7e52b1e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T11:05:19.642 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 11:05:19 smithi133 podman[74995]: 2024-09-10 11:05:19.373804433 +0000 UTC m=+2.152705977 container cleanup 354565dfd37317baf857afc12e4acefc7a880984699f095de4651e5a7e52b1e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T11:05:19.642 INFO:journalctl@ceph.mds.i.smithi133.stdout:Sep 10 11:05:19 smithi133 bash[74995]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-i 2024-09-10T11:05:19.776 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T11:05:19.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:05:19.809 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T11:05:19.809 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T11:05:19.809 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.l 2024-09-10T11:05:20.225 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:05:19 smithi133 systemd[1]: Stopping Ceph mds.l for 65af4d96-6f60-11ef-bcea-c7b262605968... 2024-09-10T11:05:20.225 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:05:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l[65164]: 2024-09-10T11:05:20.135+0000 7fea99b39640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:05:20.225 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:05:20 smithi133 ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l[65164]: 2024-09-10T11:05:20.135+0000 7fea99b39640 -1 mds.l *** got signal Terminated *** 2024-09-10T11:05:23.286 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:05:22 smithi133 podman[75115]: 2024-09-10 11:05:22.975139027 +0000 UTC m=+2.943218021 container died 185a2aa8bf29858bc3e92a2e5e68e4fa44c15a301e2a08c1bc40e6ab92f28869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux ) 2024-09-10T11:05:23.286 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:05:23 smithi133 podman[75115]: 2024-09-10 11:05:23.285403859 +0000 UTC m=+3.253482839 container cleanup 185a2aa8bf29858bc3e92a2e5e68e4fa44c15a301e2a08c1bc40e6ab92f28869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:05:23.611 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:05:23 smithi133 bash[75115]: ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l 2024-09-10T11:05:23.612 INFO:journalctl@ceph.mds.l.smithi133.stdout:Sep 10 11:05:23 smithi133 podman[75130]: 2024-09-10 11:05:23.503466579 +0000 UTC m=+0.523036128 container remove 185a2aa8bf29858bc3e92a2e5e68e4fa44c15a301e2a08c1bc40e6ab92f28869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-65af4d96-6f60-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:05:23.630 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-65af4d96-6f60-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T11:05:23.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T11:05:23.663 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T11:05:23.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T11:05:23.876 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T11:05:26.492 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T11:05:26.701 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T11:05:29.195 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T11:05:29.396 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T11:05:31.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T11:05:31.876 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T11:05:31.909 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T11:05:31.936 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T11:05:31.938 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034/remote/smithi007/crash 2024-09-10T11:05:31.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/crash -- . 2024-09-10T11:05:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T11:05:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now 2024-09-10T11:05:31.996 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034/remote/smithi103/crash 2024-09-10T11:05:31.997 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/crash -- . 2024-09-10T11:05:32.062 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T11:05:32.062 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-09-10T11:05:32.064 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034/remote/smithi133/crash 2024-09-10T11:05:32.065 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/crash -- . 2024-09-10T11:05:32.103 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T11:05:32.104 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2024-09-10T11:05:32.104 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T11:05:32.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-10T11:05:32.144 INFO:tasks.cephadm:Compressing logs... 2024-09-10T11:05:32.144 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T11:05:32.187 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T11:05:32.189 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T11:05:32.213 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T11:05:32.213 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T11:05:32.214 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T11:05:32.214 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T11:05:32.214 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T11:05:32.216 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T11:05:32.216 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T11:05:32.216 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T11:05:32.217 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T11:05:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T11:05:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:: No such file or directory 2024-09-10T11:05:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T11:05:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T11:05:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T11:05:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T11:05:32.218 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T11:05:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-volume.log: gzip/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.b.log: -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T11:05:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T11:05:32.220 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T11:05:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mon.a.log: 84.6% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T11:05:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.g.log: 90.4% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T11:05:32.221 INFO:teuthology.orchestra.run.smithi007.stderr: 94.5% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T11:05:32.223 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T11:05:32.223 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T11:05:32.227 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.2% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T11:05:32.227 INFO:teuthology.orchestra.run.smithi103.stderr: 94.5% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T11:05:32.227 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.log 2024-09-10T11:05:32.227 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T11:05:32.228 INFO:teuthology.orchestra.run.smithi007.stderr: 94.6% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T11:05:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T11:05:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.log 2024-09-10T11:05:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T11:05:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T11:05:32.229 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T11:05:32.229 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mgr.x.log: 87.1% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T11:05:32.230 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.log: /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mgr.y.log: 87.2% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T11:05:32.231 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T11:05:32.231 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T11:05:32.231 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T11:05:32.231 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T11:05:32.231 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T11:05:32.231 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T11:05:32.232 INFO:teuthology.orchestra.run.smithi103.stderr: 90.9% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T11:05:32.234 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T11:05:32.234 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T11:05:32.234 INFO:teuthology.orchestra.run.smithi007.stderr: 94.5%/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.j.log: -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.d.log.gzgzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T11:05:32.234 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T11:05:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T11:05:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T11:05:32.236 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.11.log: -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T11:05:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.log 2024-09-10T11:05:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T11:05:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T11:05:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T11:05:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T11:05:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T11:05:32.237 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T11:05:32.238 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.3.log: 94.7% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T11:05:32.240 INFO:teuthology.orchestra.run.smithi103.stderr: 92.2% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T11:05:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T11:05:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T11:05:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T11:05:32.246 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/cephadm.log: 93.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T11:05:32.260 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T11:05:32.337 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T11:05:32.366 INFO:teuthology.orchestra.run.smithi133.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T11:05:32.376 INFO:teuthology.orchestra.run.smithi133.stderr: 90.6% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T11:05:32.420 INFO:teuthology.orchestra.run.smithi133.stderr: 82.2% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T11:05:32.535 INFO:teuthology.orchestra.run.smithi007.stderr: 90.2% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T11:05:32.554 INFO:teuthology.orchestra.run.smithi133.stderr: 90.8% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T11:05:32.789 INFO:teuthology.orchestra.run.smithi103.stderr: 91.8% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T11:05:32.872 INFO:teuthology.orchestra.run.smithi133.stderr: 87.2% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T11:05:35.022 INFO:teuthology.orchestra.run.smithi007.stderr: 92.3% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T11:05:35.923 INFO:teuthology.orchestra.run.smithi103.stderr: 94.4% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T11:05:37.856 INFO:teuthology.orchestra.run.smithi103.stderr: 95.1% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T11:05:43.885 INFO:teuthology.orchestra.run.smithi103.stderr: 94.1% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T11:05:43.887 INFO:teuthology.orchestra.run.smithi007.stderr: 94.1% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T11:05:44.002 INFO:teuthology.orchestra.run.smithi007.stderr: 93.8% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T11:05:45.660 INFO:teuthology.orchestra.run.smithi007.stderr: 93.9% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T11:05:46.080 INFO:teuthology.orchestra.run.smithi103.stderr: 94.0% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T11:05:46.785 INFO:teuthology.orchestra.run.smithi007.stderr: 93.9% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T11:05:46.788 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T11:05:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m14.587s 2024-09-10T11:05:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m52.727s 2024-09-10T11:05:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m1.824s 2024-09-10T11:05:46.936 INFO:teuthology.orchestra.run.smithi103.stderr: 93.9% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T11:05:48.226 INFO:teuthology.orchestra.run.smithi103.stderr: 94.0% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T11:05:56.052 INFO:teuthology.orchestra.run.smithi133.stderr: 91.7% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T11:06:29.005 INFO:teuthology.orchestra.run.smithi133.stderr: 94.1% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T11:06:40.206 INFO:teuthology.orchestra.run.smithi133.stderr: 95.3% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T11:07:09.274 INFO:teuthology.orchestra.run.smithi133.stderr: 95.4% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T11:07:11.159 INFO:teuthology.orchestra.run.smithi133.stderr: 94.0% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T11:07:23.327 INFO:teuthology.orchestra.run.smithi103.stderr: 93.3% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T11:07:23.327 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-10T11:07:23.327 INFO:teuthology.orchestra.run.smithi103.stderr:real 1m51.120s 2024-09-10T11:07:23.328 INFO:teuthology.orchestra.run.smithi103.stderr:user 2m51.370s 2024-09-10T11:07:23.328 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m5.556s 2024-09-10T11:08:14.465 INFO:teuthology.orchestra.run.smithi133.stderr: 94.0% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T11:08:53.357 INFO:teuthology.orchestra.run.smithi133.stderr: 94.1% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T11:21:29.543 INFO:teuthology.orchestra.run.smithi133.stderr: 93.4% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T11:22:34.486 INFO:teuthology.orchestra.run.smithi133.stderr: 93.4% -- replaced with /var/log/ceph/65af4d96-6f60-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T11:22:34.487 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-09-10T11:22:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:real 17m2.283s 2024-09-10T11:22:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:user 6m29.979s 2024-09-10T11:22:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m18.966s 2024-09-10T11:22:34.489 INFO:tasks.cephadm:Archiving logs... 2024-09-10T11:22:34.490 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034/remote/smithi007/log 2024-09-10T11:22:34.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T11:22:41.482 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034/remote/smithi103/log 2024-09-10T11:22:41.483 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T11:23:05.599 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034/remote/smithi133/log 2024-09-10T11:23:05.600 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T11:24:02.397 INFO:tasks.cephadm:Removing cluster... 2024-09-10T11:24:02.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 --force 2024-09-10T11:24:02.611 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T11:24:02.951 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 --force 2024-09-10T11:24:03.155 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T11:24:03.646 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 65af4d96-6f60-11ef-bcea-c7b262605968 --force 2024-09-10T11:24:04.004 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 65af4d96-6f60-11ef-bcea-c7b262605968 2024-09-10T11:24:04.667 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T11:24:04.667 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T11:24:04.688 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T11:24:04.704 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T11:24:04.720 INFO:tasks.cephadm:Teardown complete 2024-09-10T11:24:04.721 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T11:24:04.732 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T11:24:04.732 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T11:24:04.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T11:24:04.747 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T11:24:04.878 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T11:24:04.888 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T11:24:04.889 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:24:04.892 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:24:04.894 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:24:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:24:04.907 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:24:04.908 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:24:04.910 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:24:04.910 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-10T11:24:04.910 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 -270us[ -268us] +/- 51ms 2024-09-10T11:24:04.911 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -85us[ -85us] +/- 61ms 2024-09-10T11:24:04.911 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +467us[ +470us] +/- 68ms 2024-09-10T11:24:04.911 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:24:04.912 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:24:04.912 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-10T11:24:04.913 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 -258us[ -257us] +/- 51ms 2024-09-10T11:24:04.913 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 -67us[ -67us] +/- 61ms 2024-09-10T11:24:04.913 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +484us[ +484us] +/- 68ms 2024-09-10T11:24:04.913 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:24:04.947 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:24:04.948 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-09-10T11:24:04.948 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 -251us[ -252us] +/- 51ms 2024-09-10T11:24:04.948 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -56us[ -57us] +/- 61ms 2024-09-10T11:24:04.948 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +485us[ +484us] +/- 68ms 2024-09-10T11:24:04.948 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:24:04.949 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T11:24:04.959 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T11:24:04.959 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T11:24:04.997 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T11:24:05.081 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T11:24:05.150 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T11:24:05.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:24:05.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:24:05.259 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:24:05.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:24:05.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:24:05.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:24:05.315 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T11:24:05.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T11:24:05.381 INFO:teuthology.task.internal:Duration was 3282.997208 seconds 2024-09-10T11:24:05.381 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T11:24:05.413 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T11:24:05.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T11:24:05.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T11:24:05.419 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T11:24:05.455 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:24:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:24:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:24:05.979 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T11:24:05.980 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T11:24:05.980 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T11:24:06.004 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-09-10T11:24:06.004 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T11:24:06.027 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T11:24:06.027 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T11:24:06.057 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T11:24:06.057 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T11:24:06.060 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T11:24:06.070 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T11:24:06.125 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T11:24:06.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T11:24:06.132 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T11:24:06.137 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T11:24:06.620 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T11:24:06.633 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T11:24:06.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T11:24:06.660 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T11:24:06.685 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T11:24:06.711 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T11:24:06.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:24:06.725 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:24:06.728 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:24:06.747 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-09-10T11:24:06.749 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-09-10T11:24:06.787 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-09-10T11:24:06.809 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:24:06.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:24:06.824 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:24:06.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:24:06.839 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:24:06.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:24:06.863 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T11:24:06.875 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T11:24:06.876 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034/remote/smithi007 2024-09-10T11:24:06.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T11:24:06.922 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034/remote/smithi103 2024-09-10T11:24:06.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T11:24:06.961 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899034/remote/smithi133 2024-09-10T11:24:06.961 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T11:24:06.999 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T11:24:06.999 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T11:24:07.001 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T11:24:07.005 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T11:24:07.047 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T11:24:07.061 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T11:24:07.061 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T11:24:07.121 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T11:24:07.122 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T11:24:07.124 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T11:24:07.126 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T11:24:07.137 INFO:teuthology.orchestra.run.smithi007.stdout: 265336 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 11:24 /home/ubuntu/cephtest 2024-09-10T11:24:07.139 INFO:teuthology.orchestra.run.smithi103.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 11:24 /home/ubuntu/cephtest 2024-09-10T11:24:07.141 INFO:teuthology.orchestra.run.smithi133.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 11:24 /home/ubuntu/cephtest 2024-09-10T11:24:07.148 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T11:24:07.160 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T11:24:07.228 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 3282.9972081184387 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi007 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=0974d9d0be4c4dc3b6304d41fbd461a8 status: fail success: false 2024-09-10T11:24:07.228 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T11:24:07.311 INFO:teuthology.run:FAIL